beamable / BeamableProduct

The beamable product suite including com.beamable, com.beamable.server, microservice base image, portal, the installer, and build scripts
Other
5 stars 0 forks source link

USAM - deploy sometimes break before uploading images #3585

Open gabrielbeamable opened 1 month ago

gabrielbeamable commented 1 month ago

In the deployment process, first we build all images and deploy to local, which means that we run those services and storages that are being deployed locally using docker. After that, we check if those services are indeed running by making a request to the /Docs endpoint of those services. Sometimes, for some unknown reason, those services are exiting before that request is being made, which causes the backend to return a 500 error like this:

CLI: stderr,[[/basic/1709927373427153.DE_1709927373427156.MacBook-Promicro_JustAnotherOne/admin/Docs]]request error with response code: 500 and message: No binding found for service 1709927373427153.de_1709927373427156.macbook-promicro_justanotherone.basic with routingKeys: ,

However, if you look into the logs of the service that exited, you will see this error:

2024-08-12 17:17:27 assertion failed [block != nullptr]: BasicBlock requested for unrecognized address
2024-08-12 17:17:27 (BuilderBase.h:550 block_for_offset)

As mentioned before this is not consistent, most of the times it works fine.

cdhanna commented 1 month ago

estimated this happens 3/10 times

cdhanna commented 1 month ago

Gabriel noticed this on 25.0.3, but Pedro and I are on 26.x, maybe its a docker version thing?