Sorting category on position with null or missing values

[bartdelvaux]

When I have a child category which is a deepest node in a category tree, the sorting on position is not sorting as I expect.

Preconditions

Magento Version : CE 2.2

ElasticSuite Version : 2.5.13

Environment : Production mode

Steps to reproduce

1. Create a category tree
2. Set some product positions in a specific category. Make sure you have also positions with the number 0
3. If you look for a specific product en ElasticSearch, the category data for a product with a high position looks like this:

         "category" : [
           {
             "category_id" : 2,
             "position" : 10003,
             "store_id" : "1",
             "visibility" : "4"
           },
           {
             "category_id" : 12,
             "position" : 10003,
             "store_id" : "1",
             "visibility" : "4",
             "name" : "Cat level 1"
           },
           {
             "category_id" : 139,
             "position" : 3,
             "is_parent" : true,
             "store_id" : "1",
             "visibility" : "4",
             "name" : "Deepest level"
           }
         ],

   A product with a position of 0 looks like this:

         "category" : [
           {
             "category_id" : 2,
             "position" : 10000,
             "store_id" : "1",
             "visibility" : "4"
           },
           {
             "category_id" : 12,
             "position" : 10000,
             "store_id" : "1",
             "visibility" : "4",
             "name" : "Cat level 1"
           },
           {
             "category_id" : 139,
             "is_parent" : true,
             "store_id" : "1",
             "visibility" : "4",
             "name" : "Deepest level"
           }
         ],

   In the deepest level, the position is not set.

Expected result

When sorting ascending (from low to high), I expect that the products with a missing position are ordered before the products with a (higher) position. This is also how sorting works in MySQL (null values are sorted first).

Actual result

1. When sorting ascending (from 0 to high), the products with a position of 0 are sorted after the products with a higher position
2. When sorting descending (high to low), the products with a position of 0 are sorted before the products with a higher position.

This is only the case when you have a missing position in a category and you are sorting that category.

What could be wrong:

When I debug the ElasticSearch query, I see the following JSON for the category sorting:

  "sort": [
    {
      "category.position": {
        "order": "desc",
        "missing": "_first",
        "unmapped_type": "string",
        "nested_path": "category",
        "mode": "min",
        "nested_filter": {
          "terms": {
            "category.category_id": [
              "139"
            ],
            "boost": 1
          }
        }
      }
    }
  ],

I see the line "missing": "_first" in the JSON. When sorting desc (high to low), I expect that the missing values are sorted last.

When sorting asc, I see in the JSON "missing": "_last". In that case, I expect that the missing values are sorted first.

In the class Smile\ElasticsuiteCore\Search\Adapter\Elasticsuite\Request\SortOrder\Builder, when I change on line 73 _first with _last, the sorting works as I expect.

[afoucret]

Having no position and having the position 0 is a different semantic. In the first case it means product are not ordered and should be displayed after ordered products when default sort order is applied.

Additionally ElasticSuite, does not use Magento standard positioning but it's own (the one you see in the merchandising tool in category admin). No product can get the position 0 using this tool.

=> It works as designed and as expected