Too many requests

[huotarih]

[medusa-plugin-printful]: Hey! Initial Printful synchronization has been started with a batch size of 3! This might take a while.. [medusa-plugin-printful]: Printful Webhook Support is enabled! [medusa-plugin-printful]: Product HH Cap does not exist in Medusa! Preparing to create...️ [medusa-plugin-printful]: Product HH Mountain Mug does not exist in Medusa! Preparing to create...️ [medusa-plugin-printful]: Product HH Rubber Case for AirPods® does not exist in Medusa! Preparing to create...️ [medusa-plugin-printful]: Failed getting category from Printful: You've recently sent too many requests. Please try again after 60 seconds. [medusa-plugin-printful]: Error occurred while trying to create a product! Attempt 1 of 5. Error: You've recently sent too many requests. Please try again after 60 seconds. [medusa-plugin-printful]: Error occurred while trying to create a product! Attempt 1 of 5. Error: [object Object] [medusa-plugin-printful]: Failed getting category from Printful: You've recently sent too many requests. Please try again after 60 seconds. [medusa-plugin-printful]: Error occurred while trying to create a product! Attempt 1 of 5. Error: You've recently sent too many requests. Please try again after 60 seconds.

Everything fails because of request limit.

[fsyntax]

Hey there! Thanks for reporting. Im actively working on a alternative to handle the Printful API rate limit. The error occurs most likely due to products having lots of variants and the categories fetch not being optimized.

Until I got the new solution ready, I suggest turning down the batchSize option to 1 or 2. Hope this helps in the meanwhile!

[huotarih]

Thanks for your work! I tried with batch size one with a brand new api key but it still got flooded after one item with only 4 variants. Weird.

[fsyntax]

Thanks for your work! I tried with batch size one with a brand new api key but it still got flooded after one item with only 4 variants. Weird.

Hey! I've just released a new version wich introduces a queue system through redis BullMQ and some tweakings to the backoff strategy options. The batchSize option is now responsible for how much product jobs are going to be added at once to the queue. You can now basically turn this value much higher, as the worker only handles 1 concurrent job. It should definetely work better now!

[huotarih]

Looks promising! I still have an issue with my deployment failing because of the plugin for some reason, which is quite weird since i have another deployment that it doesn't fail with. However then other stuff keeps failing because of old build tools etc. With newest medusa 1.11 it gives this error: 2023-05-23T09:48:22.515414612Z {"level":"error","message":"Error starting server","stack":[{"columnNumber":23,"fileName":"/app/medusa/node_modules/medusa-plugin-printful/utils/printful-request.js","functionName":"new PrintfulClient","lineNumber":18,"methodName":null,"native":false,"typeName":null},{"columnNumber":28,"fileName":"/app/medusa/node_modules/medusa-plugin-printful/services/printful.js","functionName":"new PrintfulService","lineNumber":44,"methodName":null,"native":false,"typeName":null},{"columnNumber":112,"fileName":"/app/medusa/node_modules/@medusajs/medusa/dist/loaders/plugins.js","functionName":"container.register._g.<computed>.lifetime","lineNumber":441,"methodName":"lifetime","native":false,"typeName":"container.register._g.<computed>"},{"columnNumber":20,"fileName":"/app/medusa/node_modules/awilix/lib/resolvers.js","functionName":"Object.resolve","lineNumber":295,"methodName":"resolve","native":false,"typeName":"Object"},{"columnNumber":41,"fileName":"/app/medusa/node_modules/awilix/lib/container.js","functionName":"resolve","lineNumber":287,"methodName":null,"native":false,"typeName":null},{"columnNumber":33,"fileName":"/app/medusa/node_modules/awilix/lib/container.js","functionName":"Object.get","lineNumber":66,"methodName":"get","native":false,"typeName":"Object"},{"columnNumber":39,"fileName":"/app/medusa/node_modules/medusa-plugin-printful/services/printfulSync.js","functionName":"new PrintfulSyncService","lineNumber":37,"methodName":null,"native":false,"typeName":null},{"columnNumber":112,"fileName":"/app/medusa/node_modules/@medusajs/medusa/dist/loaders/plugins.js","functionName":"container.register._g.<computed>.lifetime","lineNumber":441,"methodName":"lifetime","native":false,"typeName":"container.register._g.<computed>"},{"columnNumber":20,"fileName":"/app/medusa/node_modules/awilix/lib/resolvers.js","functionName":"Object.resolve","lineNumber":295,"methodName":"resolve","native":false,"typeName":"Object"},{"columnNumber":41,"fileName":"/app/medusa/node_modules/awilix/lib/container.js","functionName":"resolve","lineNumber":287,"methodName":null,"native":false,"typeName":null}],"timestamp":"2023-05-23 09:48:22"}

[fsyntax]

Looks promising! I still have an issue with my deployment failing because of the plugin for some reason, which is quite weird since i have another deployment that it doesn't fail with. However then other stuff keeps failing because of old build tools etc. With newest medusa 1.11 it gives this error: 2023-05-23T09:48:22.515414612Z {"level":"error","message":"Error starting server","stack":[{"columnNumber":23,"fileName":"/app/medusa/node_modules/medusa-plugin-printful/utils/printful-request.js","functionName":"new PrintfulClient","lineNumber":18,"methodName":null,"native":false,"typeName":null},{"columnNumber":28,"fileName":"/app/medusa/node_modules/medusa-plugin-printful/services/printful.js","functionName":"new PrintfulService","lineNumber":44,"methodName":null,"native":false,"typeName":null},{"columnNumber":112,"fileName":"/app/medusa/node_modules/@medusajs/medusa/dist/loaders/plugins.js","functionName":"container.register._g.<computed>.lifetime","lineNumber":441,"methodName":"lifetime","native":false,"typeName":"container.register._g.<computed>"},{"columnNumber":20,"fileName":"/app/medusa/node_modules/awilix/lib/resolvers.js","functionName":"Object.resolve","lineNumber":295,"methodName":"resolve","native":false,"typeName":"Object"},{"columnNumber":41,"fileName":"/app/medusa/node_modules/awilix/lib/container.js","functionName":"resolve","lineNumber":287,"methodName":null,"native":false,"typeName":null},{"columnNumber":33,"fileName":"/app/medusa/node_modules/awilix/lib/container.js","functionName":"Object.get","lineNumber":66,"methodName":"get","native":false,"typeName":"Object"},{"columnNumber":39,"fileName":"/app/medusa/node_modules/medusa-plugin-printful/services/printfulSync.js","functionName":"new PrintfulSyncService","lineNumber":37,"methodName":null,"native":false,"typeName":null},{"columnNumber":112,"fileName":"/app/medusa/node_modules/@medusajs/medusa/dist/loaders/plugins.js","functionName":"container.register._g.<computed>.lifetime","lineNumber":441,"methodName":"lifetime","native":false,"typeName":"container.register._g.<computed>"},{"columnNumber":20,"fileName":"/app/medusa/node_modules/awilix/lib/resolvers.js","functionName":"Object.resolve","lineNumber":295,"methodName":"resolve","native":false,"typeName":"Object"},{"columnNumber":41,"fileName":"/app/medusa/node_modules/awilix/lib/container.js","functionName":"resolve","lineNumber":287,"methodName":null,"native":false,"typeName":null}],"timestamp":"2023-05-23 09:48:22"}

That seems like an odd error - I mainly tested & developed the plugin on latest Medusa versions, 1.11.0 included. Unfortunately I couldnt reproduce this issue, would you mind sharing your plugin options? Have you also tried deleting node_modules and reinstall tthe dependencies?

[huotarih]

Yeah! This is super weird because it only happens when building for production (staging), for which I use docker, so dependencies are installed fresh every time.

My config is as follows: options: { printfulAccessToken: process.env.PRINTFUL_ACCESS_TOKEN, storeId: process.env.PRINTFUL_STORE_ID, backendUrl: process.env.BACKEND_URL, redisHost: process.env.REDIS_HOST, redisPort: process.env.REDIS_PORT, enableWebhooks: process.env.NODE_ENV === "PRODUCTION" ? true : false, enableSync: false, batchSize: 5, productTags: true, productCategories: true, },

I need to investigate some more I guess. Right now I seem to be having an issue with the plugin not taking my environment variables into the redis address and port no matter what I put in the config.

Might be easier to take the redis url as a whole in the config and use that since it's being used already in the event bus and cache, or use the URL.hostname and URL.port from the url.

[fsyntax]

Yeah! This is super weird because it only happens when building for production (staging), for which I use docker, so dependencies are installed fresh every time.

My config is as follows: options: { printfulAccessToken: process.env.PRINTFUL_ACCESS_TOKEN, storeId: process.env.PRINTFUL_STORE_ID, backendUrl: process.env.BACKEND_URL, redisHost: process.env.REDIS_HOST, redisPort: process.env.REDIS_PORT, enableWebhooks: process.env.NODE_ENV === "PRODUCTION" ? true : false, enableSync: false, batchSize: 5, productTags: true, productCategories: true, },

I need to investigate some more I guess. Right now I seem to be having an issue with the plugin not taking my environment variables into the redis address and port no matter what I put in the config.

Might be easier to take the redis url as a whole in the config and use that since it's being used already in the event bus and cache, or use the URL.hostname and URL.port from the url.

Refactored it in the latest version. It still required to pass the REDIS_URL env variable in the plugin options. Im currently trying the plugin on a production store, without any errors so far!

Im closing this issue in favor of the integrated rate limit strategy - feel free to open another issue if you've got something 🙏