What is a data feed?

To make smart search work on your site, you must provide our developers with a data feed. Thanks to that data, Multisearch will find what the user is searching for in a split of a second. Often our managers are asked the question — what is a product feed and where to get it? In this article, we will answer your questions about this file in detail to avoid difficulties during feed generation, even if you have not worked with unloading before.

XML and YML: what is it and what is their mission?

Many file formats are used to describe the site content. For example XML, YML, Excel, TXT, CSV, and others. But each site or service has its own unload recommendations and restrictions. Multisearch also has it — we only need a data feed in XML/YML format.

XML/YML-files contain detailed descriptions of products, services, or articles of your site.

What are these formats?

• XML– is an eXtensible Markup Language, and in simple words – is a language for describing data. It doesn't do anything, but serves to structure, store, and transfer information between sites, platforms, or services.
• YML — is an isotope of the XML format from Yandex, which was created for placing products on Yandex Market. This markup language is clear to almost any system and has no significant differences from XML

Why XML/YML file but not Excel?

• These are the most common formats that stores are using to export data to price-aggregators and marketplaces.
• Such files are convenient due to certain standards. Specialists do not need to puzzle out what you transmit, under what name, and in what field —  everything is described.

Usually, the creation of the data feed is online stores' basic functionality, and the most popular CMSs have it. It's easy to generate the data feed if your site runs on a CMS. All you need to do is just to click on the appropriate button in the admin panel. But to get a feed from a self-developed website, you need the help of a programmer. In this case, the specialist will create the file manually or generate it by script. Next, we'll tell you what you need to add to the unloading file and show you how it looks. 

Everyone needs a data feed: online stores, blogs, and information portals.

Multisaerch requires XML or YML files for its work. A feed unload file can be divided into several files by categories or product availability, for example. The more information you provide in the feed, the better search algorithms will recognize user queries. Information in such files is written with help of attributes. 

What should the online store specify?

Here are the required attributes for the online store in case of Multisearch integration via API:

• category ID;
• category name;
• product ID;
• product name;
• availability;
• article number;
• characteristics of items;
• brand;
• product description.

In addition to these fields, we recommend entering: product type (special offer, new product, promotion, hit sales), product link, product photo link, price, old price, category link, product creation date, key fields, product rating.

Integration via the plugin requires more information, so be sure to specify the following:

• product ID;
• product name;
• article number;
• category ID; 
• category name;
• product link;
• availability;
• price;
• product characteristics;
• product photo link;
• brand;
• product description.

Additional attributes are: product description, old price, link to the category, product type, product creation date, key fields, product rating.

What information is required for informational sites?

Non-commercial sites that are using the API must specify the following:

• category ID;
• category name;
• article ID;
• article title;
• text of the article.

We recommend these attributes: article creation date, the key fields, article rating.

If you are using a plugin for an information site, make sure to make the following changes:

• article ID;
• article title;
• Article category ID;
• category name;  
• article link

These are additional fields: article text, article category link, article creation date, key fields, article rating, photo link.

Farther we will show you examples of feed for online store and for informational site. 

Not obvious, but important questions about product updating and the number of feeds.

How often do I need to update the feed unload file? We recommend updating the data feed once an hour. 

Do I need to add missing products to the feed? Here the reasons why we recommend specifying all the products, even those that are not in stock:

• the displaying of “not available” products affect the growth of conversion;
• a missing item can reduce the bounce rate if the user can go to similar products or contact the Sales Department to clarify the date when the product will be available.

These positions can only be displayed in a case of 0 output results. You can read more about this in the article “ "Product out of stock: delete or leave on the site and in the search" .

There are a huge number of products in the assortment. How can I simplify the data transfer process? We allow you to transfer only those items that have changed. To optimize the updating, you need to create 2 feeds:

• full feed, in which you need to transmit all information about the product. Nightly update;
• a “quick” feed, in which you need to specify only the price and availability. Hourly update.

It speeds up the updating of information and reduces the load on your site.

Is there any validation of the feed? Yes. If the number of products in the new feed decreases by more than 10%, we will send you a notification with a confirmation request. In this case, the number of products saves from the previous successful indexing.

How many feeds do I need for a multilingual site? Each language version of the site requires a separate feed data unloading.  

Getting acquainted with the main elements of unloading

Each platform has its own requirements and features for data feed unloading. We will tell you about the main elements of XML markup in order to help you to describe your site data as best as possible.

The <categories> block contains parameters for <category> categories. It specifies the URL attribute with a link to the required category. For example: <category id="14" url="https://domain.com/laptop/">Laptops</category>. This link is used to go from categories in the search results to categories in the online store.

The <offers> block stores information about the product. The following parameters are used for this element:

• <label> with the value of the product type, for example, new, hit, discount, promotion. This parameter is used for prioritization of displaying the special offers in a search results;  
• <oldprice> with the value of the old price before the discount. It is needed for displaying the crossed out price and the new price;
• <presence> with the value of product availability, for example, warehouse, in stock, temporarily out of stock, check availability. It is needed to show results of availability from first to last;
• <createdAt> – the creation date of the product. It used for new products displaying;
• <vendorCode> - product article;
• <picture> – image, you need necessarily to specify the location of the file with size near 160x160px.

Additional tags and their functions

If an element has multiple values, enter them sequentially. For example:

• <categoryId>14</category>
• <categoryId>25</category>
• <label>New</label>
• <label>Hit</label>

To set up additional results ranking, use the <ordering> parameter with a numeric value. This tag is useful if your site has ratings/internal weight/score/product popularity. You need to add this data to the feed unloading file to get smart search to take these words into account in search results.

Keywords or search queries need to be prioritized in search results. To do so, specify them by using the <keywords> parameter. For example:
<keywords>Office laptop, Popular author</keywords> 
or
<keyword>Office laptop</keyword>
<keyword>Popular author</keyword>

You can use the vendor code to search for different codes <code>4205D</code>.

To display additional information in the search results under the product name we need the <snippet>tag Code: 4205D</snippet>.

6 common errors in feed unloading

Errors in the feed that we receive from clients are not uncommon. Some of them are critical - not allowing to index the file and stopping the work of smart search. No closing tag for the category block — is an example of such a mistake. If an error occurs in the offer, the search will not find this position. For example, if 20 of 100 products contain such errors, then only 80 items will be indexing. That's why we share with you the most common mistakes.
 
Yandex writes that problems may be as follows:

1. YML doesn't match XML specifications — to check it, change the extension to .xml and look this file in the browser, you can also run the file through the validator;
2. Format requirements are non-compliance: incorrect use of elements, incorrect order, empty elements, invalid characters;
3. incorrect encoding;
4. ID repeating;
5. URL that leads to a nonexistent file or contains invalid characters;
6. The categories in the category block are not described or the category's IDs in the products are not specified. 

Similar problems occur in XML files as well. If you can't figure out the cause of the problem, you can contact Yandex support or write to us. 

The main theses of the article briefly: 

• the more information you pass in the feed, the better search understands user queries;
• products marked as "out of stock" must be entered in the feed;
• multilingual sites need to generate a separate feed unloading for each language; 
• create multiple feeds if you have a large assortment;
• the more frequently the feed is updated, the more relevant the information on the site is.