"Pending, in queue" message remain even after bulk action - update of product attributes is done

[MrPotatox]

 - Summary of the issue,

After attempting to update product attributes in bulk I have a "stuck" system message of [Task "Update attributes for 679 selected products": 0 item(s) have been scheduled for update.] This has been in "Pending, in queue" status for about 2 weeks now.

Preconditions

Magento ver. 2.3.5-p2, 2.4-develop Using Magento Cloud Commerce (Valid for Magento Open Source as well)

Steps to reproduce

1. In Admin navigate to Catalog -> Products page.
2. Select all products.
3. In drop-down 'Actions' menu select "Update attributes"
4. Change desired value of any attribute (like description) and save.

Expected result

After all product attributes changed to the desired value, bulk action message should inform about task complete

Actual result

Bulk actions / system message still displaying at top of page.

[m2-assistant[bot]]

Hi @MrPotatox. Thank you for your report. To help us process this issue please make sure that you provided the following information:

• Summary of the issue
• Information on your environment
• Steps to reproduce
• Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento give me 2.4-develop instance - upcoming 2.4.x release

For more details, please, review the Magento Contributor Assistant documentation.

Please, add a comment to assign the issue: @magento I am working on this

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

  :warning: According to the Magento Contribution requirements, all issues must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.

:clock10: You can find the schedule on the Magento Community Calendar page.

:telephone_receiver: The triage of issues happens in the queue order. If you want to speed up the delivery of your contribution, please join the Community Contributions Triage session to discuss the appropriate ticket.

:movie_camera: You can find the recording of the previous Community Contributions Triage on the Magento Youtube Channel

:pencil2: Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel

[VladimirZaets]

@magento I am working on it

[jackrevate]

Having same issue with 2.4.0, upgraded from 2.3.5-p1 to 2.3.5-p2 then to 2.4.0.

It's really annoying, hoping this can be fixed asap.

[codemastering]

Having the same issue with 2.4.0, upgraded from 2.3.5 to 2.4.0, Thankfully, I took back up before upgrade to 2.4.0, Restored back to 2.3.5

Looking forward to any temporary fix for this issue.

Thanks

[perfpcs]

I am having this same problem with 3 different servers running M 2.4.0. CE All have stuck tasks that used to process in minutes sitting in the bulk task grid and notices for weeks. Using rabbitmq for message broker. So is someone working on this issue @VladimirZaets ? I can't see how anyone on M 2.4.0 is not having this problem. I would be happy even with a temporary fix if anyone has any ideas on this. How could M 2.4.0 even have been released with such a bad bug?? Thanks in advance if anyone can help.

[perfpcs]

To add further info for us it is various consumers and tasks getting stuck and not just the mass product attribute updates. For us we have 50-100 tasks involving Nosto extension and their product updates that are done via cron jobs:

Task "Sync 1500 Nosto products": 15 item(s) have been scheduled for update.

These seem to complete too because I have seen the number of tasks from 50-165 just for this Nosto task. So because the number goes up and down I have to think some of these are processing but then they are not updating status in the grid. I never see any of them "completed" like I used to see in previous Magento versions where I would dismiss the completed task notices. I have never had no unfinished tasks where the tasks are completed fairly fast and none show pending like is the case for 90% of the time in previous Magento versions (you see nothing pending for tasks in bulk actions grid).

I am curious @codemastering and @jackrevate are you guys using mysql or rabbitmq messaging and are you on Cloud Commerce or CE?

[TonMarton]

Hi, I am using mysql and CE (2.4) and having the exact same issue.

For me all the updates were completed (I changed price on 10 products only). But the system messages are stuck.

I don't know if it is helpful or not, but my queue_message table has the following contents: `	id	topic_name	body
10	product_action_attribute.update	{"id":null,"bulk_uuid":"95891348-0c1b-478e-9d69-8e36c356977d","topic_name":"product_action_attribute.update","serialized_data":"{\"meta_information\":\"Update product attributes\",\"product_ids\":[\"3\",\"4\",\"5\",\"7\",\"8\",\"9\",\"16\",\"17\",\"18\",\"19\"],\"store_id\":0,\"website_id\":\"0\",\"attributes\":{\"price\":\"99999\"}}","result_serialized_data":null,"status":4,"result_message":null,"error_code":null}	`

[perfpcs]

@TonMarton I think the more people that post this problem the better chance we have of Magento fixing it. But its appearing to be a widespread problem.

[perfpcs]

The more I look at this the more it appears that the bulk jobs are not actually stuck but are actually done and complete but the notice of completion is not getting back to the bulk task grid and therefore it never updates all these "pendings" to "completes". So it seems to be a communication issue to tell the bulk task notices that they are complete. You never see any bulk tasks listed as "complete" in M 2.4.0 like i used to see in M 2.3.5. Then I would click the "Dismiss all the completed tasks" link. It never gets to this state to dismiss any of them as the tasks are not updating that they are complete. So that is what the problem is as I see it. Anyone have any ideas how to resolve? Anyone having these same issues with M 2.4.0?

If anyone from team Magento feels I need to rather start a new post or topic on this I can since what I am experiencing is not exactly the same as the OP. Just let me know.

[codemastering]

100% agreed with @perfpcs

i am facing same issue with Magento 2.4

anyone, please provide the fix for this.

[HamishBlank]

Also seeing this issue in Magento 2.4, job has been carried out but notification is still being displayed as "Pending, in queue..."

[perfpcs]

@VladimirZaets ..any update to this issue since it appears to be widespead Magento bug?

[m2-assistant[bot]]

Hi @engcom-Oscar. Thank you for working on this issue. In order to make sure that issue has enough information and ready for development, please read and check the following instruction: :point_down:

• [ ] 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).

  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• [ ] 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• [ ] 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• [ ] 4. Verify that the issue is reproducible on 2.4-develop branch

  Details

  - Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

• [ ] 5. Add label Issue: Confirmed once verification is complete.

• [ ] 6. Make sure that automatic system confirms that report has been added to the backlog.

[magento-engcom-team]

:white_check_mark: Confirmed by @engcom-Oscar Thank you for verifying the issue. Based on the provided information internal tickets MC-37717 were created

Issue Available: @engcom-Oscar, You will be automatically unassigned. Contributors/Maintainers can claim this issue to continue. To reclaim and continue work, reassign the ticket to yourself.

[perfpcs]

@engcom-Oscar to be clear which issue are you addressing here to fix. Most of us posting here are not saying the tasks are really stuck but that they only appear to be stuck . We believe the tasks are actually complete but the notices at the top of admin and the bulk action log are not updating to "complete" for the tasks listed so we can dismiss them. This is true using mysql and RabbitMQ. If you need anymore details I'd be happy to provide. Thank you.

[perfpcs]

It is good to see the title changed and that the issue is confirmed. Looking forward to any patch or fix you can provide for this as soon as it has passed QA. Thank you for your help in resolving this problem.

[hostep]

Not 100% sure, but isn't this a duplicate of https://github.com/magento/magento2/issues/29718 which was fixed by https://github.com/magento/magento2/pull/29814?

[gwharton]

29814 fixes it for me. Although I did clear out old items from magento_bulk. New bulk attribute assignments completed properly, and I was able to dismiss the completed message as designed.

[perfpcs]

Hello guys, I read that other thread and am not seeing what exactly is the fix?? Could someone clarify what file(sd) need to be updated to fix this? thanks

[codemastering]

I am also having the same question as @perfpcs

Thanks.

[perfpcs]

I read that whole thread and must be blind as I see no fix mentioned. :(

[gwharton]

https://github.com/magento/magento2/pull/29814/files

[perfpcs]

Thank you @gwharton ! So its one file edit and 1 new file. I will test it out tonight. thanks again.

[gwharton]

Checkout how to apply patches with composer rather than editing core files directly.

[nuzil]

So I mark it fixed by https://github.com/magento/magento2/pull/29814/files then

[perfpcs]

For me this fix does not appear to solve anything. Now the message says the update has failed even though I can see the action completed successfully on the products. Then no tasks are showing complete still where I can dismiss them. They appear pending and then after some time or days they wont be in the bulk task grid anymore but new ones will have replaced. Those would be the Nosto consumers. I don't understand why it is not showing all the completed tasks like it used to be in 2.3.5

[perfpcs]

Do I need to create a new post since you closed this?

[perfpcs]

Could anyone from Magento team comment on what I should do here? Also could the others in this post ( @codemastering , @HamishBlank , @TonMarton, @jackrevate, @MrPotatox )that were having problems, has this fix worked for you or is your situation like mine where this fix does not solve anything? Thank you.

[engcom-Oscar]

Hi @perfpcs ! I think first of all you can try that fix that been mentioned above, but please notice that it for 2.4-develop code, so you need to upgrade and check if it works for you. If it doesn't fix your issue, you feel free to reopen the ticket - just update the description if you have more details how to reproduce your issue

[perfpcs]

Hi @engcom-Oscar I did try the fix on M 2.4 and it did not solve anything with the issue. Is "2.4-develop code" something else? How do you reopen the ticket? I can update a bit more but I pretty much have listed what is occurring in my above posts. Thank you.

[engcom-Oscar]

2.4-develop - is default branch from this repository https://github.com/magento/magento2. If you have not button 'Reopen' on the bottom of this ticket, just create a new post.

[gwharton]

I have 29814 applied ontop of 2.4.0. I've just created a new bulk attribute update task and it went through to completion successfully. Although I did get an error when displaying the log. See attached, but it atleast shows the task completed and was not stuck in pending.

[2020-09-29 08:19:20] main.CRITICAL: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry 'f694c386-9c78-4bb1-83a5-e29828dc5628' for key 'magento_acknowledged_bulk.MAGENTO_ACKNOWLEDGED_BULK_BULK_UUID', query was: INSERT INTOmagento_acknowledged_bulk(bulk_uuid) VALUES (?) [] []

This error may be because I have been clearing the bulk table, but not the acknowledged table. Will clear both and investigate further.

Note though that I'm not sure the 29814 fix will clear out old pending tasks. I had to manually clear them from magento_bulk table in SQL.

[perfpcs]

@gwharton did the notice and the grid list that task as completed and did you get to dismiss the notice by clicking the link?

[gwharton]

In your case Dismiss All Completed Tasks will do nothing as it doesn't think any of your tasks have completed. I went into the database and cleared all of the rows from the magento_bulk and magento_bulk_acknowledged tables. Then the grid is clear. Now new tasks that you create will go through to completed state, where the dismiss all tasks button should work. HOWEVER, I am repeatedly getting database integrity constraint violations when hitting the Dismiss All Completed Tasks button. I have no idea if fix 29814 fixes new jobs getting to the completed state, but also breaks the dismiss all completed tasks button with database integrity constraint violation or not! Actually, despite the error popping up on dismissing the tasks, the task is actually successfully dismissed and the next refresh of the grid shows the task has gone.

The only way I was able to get it to dismiss tasks from before 29814 was applied was to clear the database manually.

[perfpcs]

OK good news in my case. So I notice that 30 of the tasks of the 61 in the grid were dated Sept 26 and listed as "not started". The other tasks that apparently ran on Sept 28 and 29th were actually marked that the task was "completed" So I went in the database and deleted the 30 that were listed as "not started" from the magento_bulk table. After I did this the link that says "Dismiss all completed tasks" now in fact works again! My only gripe now is that it says "dismiss all tasks" but in fact it only dismisses like 5 at one time. So I had to click the link like 6 times and then, sure enough, all my tasks were completed and no longer listed in the notices at the top. So this fix in fact DOES WORK!:) Could someone though check how to make the link actually dismiss ALL like it says and not only 5 at one time? Thanks for everyone's help and actually posting this fix!

[forcecodema]

This issue still exists in 2.4.1.

[gwharton]

@forcecodema It will be fixed in 2.4.2 In the meantine, here is the fix https://github.com/magento/magento2/pull/29814/files

[magecoders]

I have the same issue in magento 2.3.5-p1, as per the fix https://github.com/magento/magento2/pull/29814/files, such column not exists in magento 2.3.5, How to fix that ?

[gwharton]

Unfortunately that patch is only compatible with 2.4.xxx codebase. Not sure of the fix for 2.3.xxx.

[hostep]

@magecoders: this isn't a bug in Magento 2.3.x, this bug only existed in Magento 2.4.0 and 2.4.1

If you are running against a queue problem, try searching around some more, there is a lot of information here on github and probably out on the web which can help you narrow down your particular issue.

[forcecodema]

@gwharton I changed the line in /magento_root/vendor/magento/module-asynchronous-operations/etc/db_schema.xml then run bin/magento setup:upgrade, but nothing changed...

[gwharton]

The change probably worked, its just that it only allows you to acknowledge new bulk tasks. I think existing bulk tasks that are stuck need to be manually cleared from the database unfortunately. Check and clear ut the offending items from magento_bulk, and hopefully new ones will work properly now.

[forcecodema]

I cleared all the messages. And when the new showed up i tried to dismiss them. Operation clears/acknowledges them in database (if you refresh the page they are gone), but the ui returns "Attention Something went wrong". It looks like the problem is with AJAX call?

/admin_xxxxxx/bulk/notification/dismiss/key/39e1eb81366237663820b4026c18b2a14d0e433c039549b46f4da305e2ea7d22/?isAjax=true

It returns an empty response - no errors in system.log

[forcecodema]

I found solution - the problem was JSON response in Dismiss.php action - I fixed it modifying the file as follows:

<?php
/**
 * Copyright © Magento, Inc. All rights reserved.
 * See COPYING.txt for license details.
 */
namespace Magento\AsynchronousOperations\Controller\Adminhtml\Notification;

use Magento\AsynchronousOperations\Model\BulkNotificationManagement;
use Magento\Backend\App\Action\Context;
use Magento\Backend\App\Action;
use Magento\Framework\Controller\Result\JsonFactory;

/**
 * Class Bulk Notification Dismiss Controller
 */
class Dismiss extends Action
{
    /**
     * @var BulkNotificationManagement
     */
    private $notificationManagement;

    /**
     * @var JsonFactory
     */
    protected $resultJsonFactory;

    /**
     * Class constructor.
     *
     * @param Context $context
     * @param BulkNotificationManagement $notificationManagement
     * @param JsonFactory $resultJsonFactory
     */
    public function __construct(
        Context $context,
        BulkNotificationManagement $notificationManagement,
        JsonFactory $resultJsonFactory
    ) {
        parent::__construct($context);
        $this->notificationManagement = $notificationManagement;
        $this->resultJsonFactory = $resultJsonFactory;
    }

    /**
     * @inheritDoc
     */
    protected function _isAllowed()
    {
        return $this->_authorization->isAllowed('Magento_Logging::system_magento_logging_bulk_operations');
    }

    /**
     * {@inheritdoc}
     */
    public function execute()
    {
        $bulkUuids = [];
        foreach ((array)$this->getRequest()->getParam('uuid', []) as $bulkUuid) {
            $bulkUuids[] = (string)$bulkUuid;
        }

        $isAcknowledged = $this->notificationManagement->acknowledgeBulks($bulkUuids);

        /** @var \Magento\Framework\Controller\Result\Json $result */
        $result = $this->resultJsonFactory->create(ResultFactory::TYPE_JSON);
        $response = new \Magento\Framework\DataObject();
        if (!$isAcknowledged) {
            $result->setHttpResponseCode(400);
            $response->setError(true);
        } else {
            $result->setHttpResponseCode(200);
            $response->setSuccess(true);
        }

        $result->setData($response);

        return $result;
    }
}

[bhaveshpp]

After update, Magento 2.3.4 to Magento 2.4 and I was facing this issue. Now the issue is solved after the update: https://github.com/magento/magento2/pull/29814/files Thank you @gwharton @nuzil

[bibz0r]

How can we apply this fix?

[gaiterjones]

@bibz0r for Magento 2.4.x in vendor/magento/module-asynchronous-operations/etc/db_schema.xml change

<column xsi:type="int" name="operation_key" padding="10" unsigned="true" nullable="false"

to

<column xsi:type="int" name="operation_key" padding="10" unsigned="true" nullable="true"

then run bin/magento s:up

Old jobs might well still be stuck but new jobs should complete.

[davirs]

I have same situation here with magento 2.4.0.

I did the fix in Dismiss.php as mentionated above and I did the fix inside db_schema.xml. After did this procidures I did a setup upgrade, I cleared cache, I forced cron:run... but the problem persists.

Someone can help me to clear this queue?:

there is some more thing to do to solve this problem? Thank you

[in-session]

The problem still exists even on Magento 2.4.1 with PHP 7.4

The cron is running so far, even starting it via SSH did not help.

Mentioned fix does not solve the problem: https://github.com/magento/magento2/pull/29814/files

[OvalMedia]

The fix works as the bulk actions get executed via cron. But the message queue in magento_bulk does not get cleared.