BIDMCDigitalPsychiatry / LAMP-platform

The LAMP Platform (issues and documentation).
https://docs.lamp.digital/
13 stars 10 forks source link

Automations framework beta #382

Open avaidyam opened 3 years ago

avaidyam commented 3 years ago

LAMP-worker currently uses the filesystem (code here) but there should be zero usage of filesystem across any LAMP backend code as there is no guarantee that these files/folders will remain available across container restart or resets.

lukeoftheshire commented 3 years ago

Hi @Linoy339 glad to hear it's working! To be honest I didn't get the chance to change anything after our previous discussion so I'm not sure what resolved the issue...perhaps just time. But thank you for the article - it will be a good resource for the future!

One note - while containers are being spun up, they are quickly failing and being removed with error 137 in the same way that the lamp-worker was initially failing (a couple blank terminations and then an error 137)- it's tough to access the logs as the containers are removed immediately, or I would give any logging info.

Let me know what other info I might give that would be helpful

Linoy339 commented 3 years ago

@lukeoftheshire . Which Container you are mentioning about? Is it like some random containers are raising up and automatically getting stopped or removed?

lukeoftheshire commented 3 years ago

The containers made by LAMP-worker (or at least what I assume to be made by lamp worker). In our container tracking logs we see containers with names like 'jolly_edison' (which I believe is the default docker naming convention), with the same docker image the worker uses, being started and then terminated and removed. I can send the exact logs if needed.

Linoy339 commented 3 years ago

@lukeoftheshire. Its a new behavior/feature of LAMP-worker to spin new containers for the automation purpose. The new container thus created will be automatically stopped and removed immediately after its purpose.

lukeoftheshire commented 3 years ago

Okay, so this is intended behavior? This looks similar to the problems we had initially with the lamp worker containers but as long as this is intended that's okay for now then. Thanks @Linoy339 !!

Linoy339 commented 3 years ago

Yes. This is intended behavior of LAMP-worker.

Linoy339 commented 3 years ago

@lukeoftheshire. Can you please check portainer, as we are unable to access the stack or containers. It says Unable to retrieve stacks. Also can you please give the whole container logs for last 2 days?

lukeoftheshire commented 3 years ago

@Linoy339 Thank you for alerting us to this! We had to force an update to the portainer container. Things appear to be working on our end now - but please let us know if you continue to encounter issues.

I have attached a .txt file containig all the logs for the lamp_staging_worker container. Please let me know if there is anything else that would be helpful. lamp_staging_worker_logs.txt

Linoy339 commented 3 years ago

Thank you @lukeoftheshire

Linoy339 commented 2 years ago

@SuruPa00 . We have tested it weeks before and found working. We were also waiting for your team's feedback and move to production if everything goes well. Hope @avaidyam can give us a feedback and move to production this week itself if everything goes well

ZCOEngineer commented 2 years ago

@avaidyam do you have any feedback on this? from our side we believe this is complete.

avaidyam commented 2 years ago

@ZCOEngineer Our team (me + @lukeoftheshire) will take a closer look later this month and double check things. This issue has been moved to a lower priority for now.

ertjlane commented 1 year ago

@avaidyam Should this issue be placed on hold, or do we think we should try to wrap this up?