Magento 2.4 catalog products sort by position is not working

[esimov]

Preconditions (*)

Magento v2.4.1

Steps to reproduce (*)

I'm trying to sort the products by position, but unfortunately it's not working. I tried to debug it and printing out the request under the Elasticsearch7 Adapter class, but I was getting an empty array for position sorting:

// vendor/magento/module-elasticsearch-7/SearchAdapter/Adapter.php:115:

array (size=0)
  empty

If I'm selecting the price or name attribute as sorting option then I'm receiving the results correctly:

// vendor/magento/module-elasticsearch-7/SearchAdapter/Adapter.php:115:

array (size=1)
  0 => 
    array (size=1)
      'price_0_1' => 
        array (size=1)
          'order' => string 'asc' (length=3)

It's also working sorting by name:

array (size=1)
  0 => 
    array (size=1)
      'name.sort_name' => 
        array (size=1)
          'order' => string 'asc' (length=3)

Can investigate the reason why it's not working?

---

Please provide Severity assessment for the Issue as Reporter. This information will help during Confirmation and Issue triage processes.

• [x] Severity: S0 - Affects critical data or functionality and leaves users without workaround.
• [ ] Severity: S1 - Affects critical data or functionality and forces users to employ a workaround.
• [ ] Severity: S2 - Affects non-critical data or functionality and forces users to employ a workaround.
• [ ] Severity: S3 - Affects non-critical data or functionality and does not force users to employ a workaround.
• [ ] Severity: S4 - Affects aesthetics, professional look and feel, “quality” or “usability”.

[m2-assistant[bot]]

Hi @esimov. Thank you for your report. To help us process this issue please make sure that you provided the following information:

• Summary of the issue
• Information on your environment
• Steps to reproduce
• Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento give me 2.4-develop instance - upcoming 2.4.x release

For more details, please, review the Magento Contributor Assistant documentation.

Please, add a comment to assign the issue: @magento I am working on this

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

  :warning: According to the Magento Contribution requirements, all issues must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.

:clock10: You can find the schedule on the Magento Community Calendar page.

:telephone_receiver: The triage of issues happens in the queue order. If you want to speed up the delivery of your contribution, please join the Community Contributions Triage session to discuss the appropriate ticket.

:movie_camera: You can find the recording of the previous Community Contributions Triage on the Magento Youtube Channel

:pencil2: Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel

[m2-assistant[bot]]

Hi @engcom-Alfa. Thank you for working on this issue. In order to make sure that issue has enough information and ready for development, please read and check the following instruction: :point_down:

• [ ] 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).

  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• [ ] 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• [ ] 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• [ ] 4. Verify that the issue is reproducible on 2.4-develop branch

  Details

  - Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

• [ ] 5. Add label Issue: Confirmed once verification is complete.

• [ ] 6. Make sure that automatic system confirms that report has been added to the backlog.

[engcom-Alfa]

Hi @esimov .

Unfortunatelly, we are not able to reproduce the issue on a fresh 2.4-develop instance. Sorting by position works correctly on the catalog products page.

Precondition: Have several products with pre-set positions. Like:

Manual testing scenario:

1. Go to Storefront -> catalog with products;
2. Select Position from Sort By drop-down menu

Actual Result: :heavy_check_mark: Products were sorted by position correctly

@esimov Maybe we missed something? Could you take a look? Thanks!

[esimov]

Unfortunately this is not working for me. I have tried with an earlier version 2.3.2 where is still possible to use MySQL as a search engine. Using MySQL the sorting functionality is working correctly, not the case with Elasticsearch.

MySQL

Elasticsearch

Is something that I'm missing?

[engcom-Alfa]

@magento give me 2.4-develop instance

[magento-deployment-service[bot]]

Hi @engcom-Alfa. Thank you for your request. I'm working on Magento instance for you.

[magento-deployment-service[bot]]

Hi @engcom-Alfa, here is your Magento Instance: https://9309f22232cbcd05dc7ea3cc7b3d680f-2-4-develop.instances.magento-community.engineering Admin access: https://9309f22232cbcd05dc7ea3cc7b3d680f-2-4-develop.instances.magento-community.engineering/admin_f433 Login: 6e4d44e7 Password: 58d75c0d6ffe

[engcom-Alfa]

@esimov On a 2.4-develop instance products sort by position works as expected (Elasticsearch). Looks like the problem has already been fixed.

Actual Result: :heavy_check_mark:

@esimov Are you able to reproduce the issue on a 2.4-develop test instance? Thanks!

[esimov]

This means that it will be fixed on the next stable release which will be v2.4.2?

[engcom-Alfa]

@esimov The problem has already been solved in the 2.4.1 version.

Actual Result: :heavy_check_mark: and so this problem should not be in the 2.4.2 version.

[esimov]

@engcom-Alfa I'm afraid is not. I have tried with M2.4.1, but it's still not working, but it's working with the latest dev version. I have also tried with a previous version where it's still not working.

[engcom-Alfa]

@esimov If the problem is reproduced on the 2.4.1 version and not reproduced on the 2.4-develop version then most likely this problem should not be in the release 2.4.2. Can we close this issue? Thanks!

[MarksAfanasevics]

@esimov we have similar problems with our 2.4.1 Magento, sorting by position works really weird way. Can you tell which exactly variable is empty in you case, in module-elasticsearch-7/SearchAdapter/Adapter.php?

[sunilit42]

@MarksAfanasevics i m faced similar issue and finally debug more than 1 day and found issue due to WeltPixel_GoogleTagManager

Hope it will help you

[MarksAfanasevics]

@sunilit42, we don't use currently WeltPixel_GoogleTagManager, we removed it some time ago, but we still use WeltPixel_OwlCarouselSlider with WeltPixel_Backend, can you give me more technical information what was wrong and how you fixed it?

[esimov]

@MarksAfanasevics when sorting by the position attribute and printing out the query result from the SearchAdapter class (line 115) is resulting always in an empty array.

[sunilit42]

@MarkADurham in our case we are WeltPixel_GoogleTagManager and it is calling WeltPixel\GoogleTagManager\Block\Category.php and that create issue, but after upgrade it is fixed

So in your case may be some other module go to this file Magento\Elasticsearch7\SearchAdapter\Adapter.php

public function query(RequestInterface $request) : QueryResponse
    {
        foreach(debug_backtrace() as $t) {       
        if(isset($t['file']))       
   echo $t['file'] . ' line ' . $t['line'] . ' calls ' . $t['function'] . "()<br/>";
}; exit

and debug which module called other than magento

[MarksAfanasevics]

@esimov In my case 115 line don't have any info. I checked $query variable, there is no order data, only sort data, which always is empty (other sorting works as desired).

Array
(
    [index] => magento2_product_1
    [type] => document
    [body] => Array
        (
            [from] => 0
            [size] => 10000
            [stored_fields] => Array
                (
                    [0] => _id
                    [1] => _score
                )

            [sort] => Array
                (
                )

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 14 days if no further activity occurs. Is this issue still relevant? If so, what is blocking it? Is there anything you can do to help move it forward? Thank you for your contributions!

[kamorana]

Is there any progress on this issue? We are facing the same problem on 2.4.1-p1.

[neddy236]

Problem still exists in 2.4.2

[heyepic]

Just updated to 2.4.2-p1 and I'm having the same issue. (ElasticSearch 7)

[daanggc]

Same issue as this one: https://github.com/magento/magento2/issues/32522

[daanggc]

Related issue to this one: https://github.com/magento/magento2/issues/31043

[tabunsa]

In our case we are running Magento 2.4.2 and Porto theme (latest version 4.0.0). Porto theme is coming with Mageplaza_Layered_Navigation + Mageplaza_Ajax_Layer extensions which interfere with the product sort on category pages.

We found the problem in: app/code/Mageplaza/LayeredNavigation/Model/ResourceModel/Fulltext/Collection.php The product collection gets sorted in the ElasticSearchApply()-function in the following way: $this->getSelect()->where('e.entity_id IN (?)', $ids); $this->getSelect()->reset(Select::ORDER); $this->getSelect()->order(new Zend_Db_Expr('FIELD(e.entity_id,' . implode(',', $ids) . ')'));

So no matter what you set in the category page toolbar (position, name, price, ascending or descending) it always sorts the products by the entity_id attribute.

Its not enough if you disable Mageplaza_Layered_Navigation in the backend (I don't know why right now).

The solution for us was to disable these extensions completely like this: bin/magento module:disable Mageplaza_LayeredNavigation --clear-static-content bin/magento module:disable Mageplaza_AjaxLayer --clear-static-content bin/magento setup:static-content:deploy bin/magento setup:di:compile

Now all the standard sort options work perfectly fine. Even though this is not a solution for the main problem some are facing I hope this maybe helps someone with a similar problem caused by this theme/extension :)

[amitvkhajanchi]

I am using Porto theme too and had encountered this problem on Magento 2.4.2. After above comment by @tabunsa, I looked up release notes for mageplaza's layered navigation module and ajax module and realized that they had newer releases that were compatible with 2.4.2 and they fixed the bug. I used composer update mageplaza/* to update the the latest 4.1.1 module for layered navigation the issue is now resolved for anyone else who encountered this problem.

[heyepic]

I am using Porto theme too and had encountered this problem on Magento 2.4.2. After above comment by @tabunsa, I looked up release notes for mageplaza's layered navigation module and ajax module and realized that they had newer releases that were compatible with 2.4.2 and they fixed the bug. I used composer update mageplaza/* to update the the latest 4.1.1 module for layered navigation the issue is now resolved for anyone else who encountered this problem.

Awesome - thanks, will grab that and update. Appreciate the heads up as I'm on Porto.

[devchris79]

I am using Porto and Magento 2.4.2, the above fix (Mageplaza core 1.4.11) still doesn't correct the sort by position functionality. Name and price sorting appears to work fine.

@amitvkhajanchi @heyepic do you see issues still?

[magento-deployment-service[bot]]

Hi @teodoramoldovan. Thank you for your request. I'm working on Magento instance for you.

[magento-deployment-service[bot]]

Hi @teodoramoldovan, unfortunately there is no ability to deploy Magento instance at the moment. Please try again later.

[pmonosolo]

Same issue is happening on 2.4.3. Any custom attributes or plugins that use the sorting are completely broken. Price sorting works fine.

[devchris79]

This fixed for me :-)

vendor/magento/module-catalog/Block/Product/ProductList/toolbar.php at line 221 Remove the if its a position sort function, it works fine with the else function! So you get:

public function setCollection($collection)
    {
        $this->_collection = $collection;

        $this->_collection->setCurPage($this->getCurrentPage());

        // we need to set pagination only if passed value integer and more that 0
        $limit = (int)$this->getLimit();
        if ($limit) {
            $this->_collection->setPageSize($limit);
        }
        if ($this->getCurrentOrder()) {
                $this->_collection->setOrder($this->getCurrentOrder(), $this->getCurrentDirection());
        }
        return $this;
    }

[m2-assistant[bot]]

Hi @engcom-Delta. Thank you for working on this issue. In order to make sure that issue has enough information and ready for development, please read and check the following instruction: :point_down:

• [ ] 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).

  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• [ ] 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• [ ] 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• [ ] 4. Verify that the issue is reproducible on 2.4-develop branch

  Details

  - Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

• [ ] 5. Add label Issue: Confirmed once verification is complete.

• [ ] 6. Make sure that automatic system confirms that report has been added to the backlog.

[engcom-Delta]

Hi @esimov, Issue is not reproducible in Magento 2.4 Develop instance , Hence closed .

[vipin12117]

Issue still exists in 2.4.3.

[vipin12117]

Hi @esimov, Issue is not reproducible in Magento 2.4 Develop instance , Hence closed .

Still issue exists

[devchris79]

@magento give me 2.4-develop instance

[magento-deployment-service[bot]]

Hi @devchris79. Thank you for your request. I'm working on Magento instance for you.

[magento-deployment-service[bot]]

Hi @devchris79, here is your Magento Instance: https://9309f22232cbcd05dc7ea3cc7b3d680f-2-4-develop.instances.magento-community.engineering Admin access: https://9309f22232cbcd05dc7ea3cc7b3d680f-2-4-develop.instances.magento-community.engineering/admin_200d Login: 5dfa60de Password: cea783f1433f

[devchris79]

The issue does appear to now be fixed in the latest 2.4 develop instance.

My 2.4.3 site works with my earlier post on a change to the setCollection function, although that function doesn't appear to be changed in the 2.4 develop code. I expect something has been corrected/changed in the collection that is sent to the function.

[vipin12117]

Can you tell me a file which I can i check and fix ?

On Mon, Nov 1, 2021 at 3:53 PM devchris79 @.***> wrote:

The issue does appear to now be fixed in the latest 2.4 develop instance.

My 2.4.3 site works with my earlier post on a change to the setCollection function, although that function doesn't appear to be changed in the 2.4 develop code. I expect something has been corrected/changed in the collection that is sent to the function.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/magento/magento2/issues/31758#issuecomment-956112718, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABXRUHP3QZT2VJWAPOZKRMLUJZTDZANCNFSM4WIRC25Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- Thanks & Regards, Vipin Garg Mobile: +91 9785118324

[devchris79]

@vipin12117

I have commented the file changes instead of removing them so you can see the changes.

vendor/magento/module-catalog/Block/Product/ProductList/toolbar.php at line 221

    public function setCollection($collection)
    {
        $this->_collection = $collection;

        $this->_collection->setCurPage($this->getCurrentPage());

        // we need to set pagination only if passed value integer and more that 0
        $limit = (int)$this->getLimit();
        if ($limit) {
            $this->_collection->setPageSize($limit);
        }
        if ($this->getCurrentOrder()) {
/*    
             if (($this->getCurrentOrder()) == 'position') {
                $this->_collection->addAttributeToSort(
                    $this->getCurrentOrder(),
                    $this->getCurrentDirection()
                );
            } else {
*/
                $this->_collection->setOrder($this->getCurrentOrder(), $this->getCurrentDirection());
//            }
        }
        return $this;
    }

[vipin12117]

Array --   | (   | [index] => sketchbubble_product_1   | [type] => document   | [body] => Array   | (   | [from] => 0   | [size] => 9   | [stored_fields] => Array   | (   | [0] => _id   | [1] => _score   | )   |     | [sort] => Array   | (   | )   |     | [query] => Array   | (   | [bool] => Array   | (   | [must] => Array   | (   | [0] => Array   | (   | [term] => Array   | (   | [category_ids] => 27   | )   |     | )   |     | [1] => Array   | (   | [terms] => Array   | (   | [visibility] => Array   | (   | [0] => 2   | [1] => 4   | )   |     | )   |     | )   |     | )   |     | )   |     | )   |     | [aggregations] => Array   | (   | [price_bucket] => Array   | (   | [extended_stats] => Array   | (   | [field] => price_0_1   | )   |     | )   |     | [category_bucket] => Array   | (   | [terms] => Array   | (   | [field] => category_ids   | [size] => 500   | )   |     | )   |     | [color_bucket] => Array   | (   | [terms] => Array   | (   | [field] => color   | [size] => 500   | )   |     | )   |     | [manufacturer_bucket] => Array   | (   | [terms] => Array   | (   | [field] => manufacturer   | [size] => 500   | )   |     | )   |     | )   |     | )   |     | )

[vipin12117]

updates ?

[vipin12117]

Any solution ?

[amokato]

So I guess this bug is still present in 2.4.3-p1, right?

[vipin12117]

yes, correct.

On Thu, Nov 18, 2021 at 8:52 PM amorana @.***> wrote:

So I guess this bug is still present in 2.4.3-p1, right?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/magento/magento2/issues/31758#issuecomment-972966648, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABXRUHP4GBPJBLXMCUV5NGLUMUK45ANCNFSM4WIRC25Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- Thanks & Regards, Vipin Garg Mobile: +91 9785118324

[jornwildenbeest]

Samen problem here

[gmilosh]

Playing with negative values fixed the issue for me. Magento v2.3.7 Search Engine MySQL

[vipin12117]

Issue with elastic search not MySQL engine.

On Sun, 28 Nov 2021, 8:51 am Milos, @.***> wrote:

Playing with negative values fixed a issue for me. Magento v2.3.7 Search Engine MySQL

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/magento/magento2/issues/31758#issuecomment-980829985, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABXRUHOOLGMORVM6N3IUVFDUOGNZZANCNFSM4WIRC25Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.