php-enqueue / enqueue-dev

Message Queue, Job Queue, Broadcasting, WebSockets packages for PHP, Symfony, Laravel, Magento. DEVELOPMENT REPOSITORY - provided by Forma-Pro
https://enqueue.forma-pro.com/
MIT License
2.17k stars 430 forks source link

MongoDB - Add combined index #1283

Closed ddziaduch closed 1 year ago

ddziaduch commented 1 year ago

Hey there 👋

In Printify we are utilizing this library quite heavily.

Recently we had an incident with the super high CPU and I/O usage in MongoDB.

With multiple queues and thousands of messages per queue, MongoDB is doing a full collection scan.

The combined index solves that.

ddziaduch commented 1 year ago

Hey @makasim !

I see that you have triggered the CI but it failed somehow. I've checked it but seems that nothing is related to my changes. Not sure what to do next 🤷‍♂️

makasim commented 1 year ago

I could merge only if CI is green. Someone has to fix it. I won't do it.

ddziaduch commented 1 year ago

@makasim can you rerun the CI?

andrewmy commented 1 year ago

https://github.com/php-enqueue/enqueue-dev/pull/1284 should fix the CI, @makasim please take a look

ddziaduch commented 1 year ago

Hey @makasim, I merged a fix from @andrewmy

This should unblock the CI.

Could you please run the CI once again?

Thanks

ddziaduch commented 1 year ago

@makasim my apologies for pinging you constantly, but this is quite a priority for Printify.

I've merged another change proposed by @andrewmy so maybe this time the CI will be green 🤞

ddziaduch commented 1 year ago

@makasim can you please create a new tag? I would like to upgrade dependencies in Printify projects. Thanks in advance 🙏🏻

makasim commented 1 year ago

I have some problems with git subtree split on my machine. I'll try to cut some time to fix it. Feel free to ping me from time to time.

makasim commented 1 year ago

I've recently migrated to mac m1 and it seems split-sh does not work on m1. So, currently, I cannot push changes from enqueue-dev repo to package repos.

I've tried to compile it at my machine, without luck so far. I'll try to work around it somehow, but it would take more time than I've thought.

makasim commented 1 year ago

done

https://github.com/php-enqueue/mongodb/commit/8b02e695c857f01b49bd61d139fefe78e7324ca3