search

armandasalmd / money-can-fly

money-can-fly.vercel.app
1 stars 0 forks source link

Long import processes on production stop running #9

Closed armandasalmd closed 1 year ago

armandasalmd commented 1 year ago

It was found out that on production api endpoints run as serverless functions. Thus they have limited lifespan - around 30 seconds before it timesout.

Running most import files exceeds 30 second limit. Thus, transactions never get imported, and import status stay "running" forever.

Solution 1 - Try using job sceduler https://quirrel.dev/ Alternative solution - Create separate service that runs only this job. Possibly nodeJs app as we have Js code written.

armandasalmd commented 1 year ago

Quirrel job queing does not work, and requires huge amounts of work to implement.

📌 Instead, algorithm was so optimized that most imports fits under 10s serveless function timeout

Can close this for now! :)