Kombu 5.4.1 not working with Celery[SQS] 5.4.0

[Hawkline451]

Celery worker is running but is not picking any messages from the SQS queue.

Kombu 5.4.0 works fine.

[Nusnus]

The changes regarding SQS in v5.4.1 were:

• SQS: Fixes #2091 queue_delete() method doesn't actually delete the queue by @ivanprjcts in https://github.com/celery/kombu/pull/2099
• Validate SQS SSL certs by default by @poundifdef in https://github.com/celery/kombu/pull/2094

@Hawkline451 can you please share more info? Is there any error? Does it sometimes work sometimes doesn’t?

If I had to guess, I’d say it’s the SSL changes (validate_cert changed from default false to default true), but the original PR says

... I'm running python 3.12.4 and tested with the latest Celery+Kombu from github.

So I am not completely sure. @ivanprjcts @poundifdef can you please give it a look as well in case I am missing something?

Thank you!

[imran8294]

Hi @Nusnus Kombu 5.4.1 is not compatible with Celery 5.4.0. As a result, the messages are not being consumed. Although the Celery workers are initialized, they are unable to retrieve messages from the queue.

With kombu <= 5.4.0 its working as expected.

[Nusnus]

Hi @Nusnus Kombu 5.4.1 is not compatible with Celery 5.4.0. As a result, the messages are not being consumed. Although the Celery workers are initialized, they are unable to retrieve messages from the queue.

With kombu <= 5.4.0 its working as expected.

@thedrow @auvipy

Fyi.

As this is an SQS only issue I prefer not to yank the last release but revert the PR that caused it (once found) and release a new patch.

[auvipy]

Yeah that should work

[Hawkline451]

@Hawkline451 can you please share more info? Is there any error? Does it sometimes work sometimes doesn’t?

If I had to guess, I’d say it’s the SSL changes (validate_cert changed from default false to default true), but the original PR says

... I'm running python 3.12.4 and tested with the latest Celery+Kombu from github.

There is no errors or warnings, we test this running the worker in an EC2 server in debug.

Aditional info

boto3==1.35.5
botocore==1.35.5
celery==5.4.0
Django==4.2.13

Python 3.11

[ivanprjcts]

Hi @Nusnus ! I checked the issue using a local sample celery application using SQS, and it seems to be related to this commit.

I fixed it after reverting -> https://github.com/ivanprjcts/kombu/commits/bugfix/revert-cert-validation/

You can try it using: pip install git+https://github.com/ivanprjcts/kombu.git@fe5fb64627f17031799bc98be7cab14155eb0682#egg=kombu

[Nusnus]

Hi @Nusnus ! I checked the issue using a local sample celery application using SQS, and it seems to be related to this commit.

I fixed it after reverting -> https://github.com/ivanprjcts/kombu/commits/bugfix/revert-cert-validation/

You can try it using: pip install git+https://github.com/ivanprjcts/kombu.git@fe5fb64627f17031799bc98be7cab14155eb0682#egg=kombu

Thank you!

@poundifdef FYI - if you have a potential fix please submit a PR. If not, we will have to revert the change unfortunately 🙏

[poundifdef]

Let me take a look! Also, if you need to revert in the meantime, please go ahead and don't let me block this. I can always make a separate PR with a more complete update.

[Nusnus]

@poundifdef

Let me take a look! Also, if you need to revert in the meantime, please go ahead and don't let me block this. I can always make a separate PR with a more complete update.

Cool, thanks!

[Nusnus]

@Hawkline451 Change reverted in main. New patch version will be released soon.

[Hawkline451]

@Hawkline451 Change reverted in main. New patch version will be released soon.

Thanks

[pranavdxl]

Any ETA for release?

[Nusnus]

Any ETA for release?

Today 🙂

[mhdzumair]

+1

[Nusnus]

v5.4.2 Released. https://pypi.org/project/kombu/5.4.2/

@Hawkline451 @imran8294 @ivanprjcts @poundifdef @pranavdxl @mhdzumair