search

packit / packit-service

Packit provided as a service
https://packit.dev
MIT License
37 stars 48 forks source link

SQS maximum message size is limited to 256KB #811

Closed sentry-io[bot] closed 3 years ago

sentry-io[bot] commented 4 years ago

Sentry Issue: RED-HAT-0P-2V8

ClientError: An error occurred (InvalidParameterValue) when calling the SendMessage operation: One or more parameters are invalid. Reason: Message must be shorter than 262144 bytes.
sentry-io[bot] commented 4 years ago

Sentry issue: RED-HAT-0P-2VW

stale[bot] commented 3 years ago

This issue has been marked as stale because it hasn't seen any activity for the last 60 days.

Stale issues are closed after 14 days, unless the label is removed by a maintainer or someone comments on it.

This is done in order to ensure that open issues are still relevant.

Thank you for your contribution! :unicorn: :rocket: :robot:

(Note: issues labeled with pinned or EPIC are never marked as stale.)

lachmanfrantisek commented 3 years ago

@jpopelka this is still not fixed and related to the global worker architecture, isn't it?

jpopelka commented 3 years ago

Yes, we haven't investigated that yet and still need to know how to work around that if ever wanted to move to SQS.

stale[bot] commented 3 years ago

This issue has been marked as stale because it hasn't seen any activity for the last 60 days.

Stale issues are closed after 14 days, unless the label is removed by a maintainer or someone comments on it.

This is done in order to ensure that open issues are still relevant.

Thank you for your contribution! :unicorn: :rocket: :robot:

(Note: issues labeled with pinned or EPIC are never marked as stale.)

stale[bot] commented 3 years ago

This issue has been marked as stale because it hasn't seen any activity for the last 60 days.

Stale issues are closed after 14 days, unless the label is removed by a maintainer or someone comments on it.

This is done in order to ensure that open issues are still relevant.

Thank you for your contribution! :unicorn: :rocket: :robot:

(Note: issues labeled with pinned or EPIC are never marked as stale.)

jpopelka commented 3 years ago

I did some searching and all advice (example) I found suggest storing the message in a DB (S3) and send only pointer to the data in the actual message to SQS. I haven't found anything that would help us with this in Python, the official library for this purpose is in Java only. Even I found one post which points to a python example project which should do this, I haven't found anything like that in the code itself. Since in our case it's Celery sending/consuming the message we would probably need to extend some Celery classes.

stale[bot] commented 3 years ago

This issue has been marked as stale because it hasn't seen any activity for the last 60 days.

Stale issues are closed after 14 days, unless the label is removed by a maintainer or someone comments on it.

This is done in order to ensure that open issues are still relevant.

Thank you for your contribution! :unicorn: :rocket: :robot:

(Note: issues labeled with pinned or EPIC are never marked as stale.)

TomasTomecek commented 3 years ago

It seems we won't use the AWS' native services after all in the near future so I'm going to close this.