svante4 container state is broken

cypressf commented 2 years ago

it looks like the backend container process is stopped, but the process is still alive

podman ps -a
CONTAINER ID  IMAGE                                      COMMAND     CREATED      STATUS                     PORTS                                           NAMES
ca533d149e3d  localhost/podman-pause:4.0.2-1652749236                3 weeks ago  Up 3 weeks ago             0.0.0.0:8000->8000/tcp, 0.0.0.0:8002->4000/tcp  17d243513267-infra
66985c7dfb56  docker.io/library/postgres:latest          postgres    3 weeks ago  Up 3 weeks ago             0.0.0.0:8000->8000/tcp, 0.0.0.0:8002->4000/tcp  crm_db
e535e2296017  localhost/climate_risk_map_backend:latest  bash        3 weeks ago  Exited (1) 11 minutes ago  0.0.0.0:8000->8000/tcp, 0.0.0.0:8002->4000/tcp  crm_backend
curl 0.0.0.0:8000/state
[{"id":1,"name":"Alabama"},{"id":2,"name":"Alaska"},{"id":4,"name":"Arizona"},{"id":5,"name":"Arkansas"},{"id":6,"name":"California"},{"id":8,"name":"Colorado"},{"id":9,"name":"Connecticut"},{"id":10,"name":"Delaware"},{"id":11,"name":"D.C."},{"id":12,"name":"Florida"},{"id":13,"name":"Georgia"},{"id":15,"name":"Hawaii"},{"id":16,"name":"Idaho"},{"id":17,"name":"Illinois"},{"id":18,"name":"Indiana"},{"id":19,"name":"Iowa"},{"id":20,"name":"Kansas"},{"id":21,"name":"Kentucky"},{"id":22,"name":"Louisiana"},{"id":23,"name":"Maine"},{"id":24,"name":"Maryland"},{"id":25,"name":"Massachusetts"},{"id":26,"name":"Michigan"},{"id":27,"name":"Minnesota"},{"id":28,"name":"Mississippi"},{"id":29,"name":"Missouri"},{"id":30,"name":"Montana"},{"id":31,"name":"Nebraska"},{"id":32,"name":"Nevada"},{"id":33,"name":"New Hampshire"},{"id":34,"name":"New Jersey"},{"id":35,"name":"New Mexico"},{"id":36,"name":"New York"},{"id":37,"name":"North Carolina"},{"id":38,"name":"North Dakota"},{"id":39,"name":"Ohio"},{"id":40,"name":"Oklahoma"},{"id":41,"name":"Oregon"},{"id":42,"name":"Pennsylvania"},{"id":44,"name":"Rhode Island"},{"id":45,"name":"South Carolina"},{"id":46,"name":"South Dakota"},{"id":47,"name":"Tennessee"},{"id":48,"name":"Texas"},{"id":49,"name":"Utah"},{"id":50,"name":"Vermont"},{"id":51,"name":"Virginia"},{"id":53,"name":"Washington"},{"id":54,"name":"West Virginia"},{"id":55,"name":"Wisconsin"},{"id":56,"name":"Wyoming"},{"id":60,"name":"American Samoa"},{"id":66,"name":"Guam"},{"id":69,"name":"Northern Mariana Islands"},{"id":72,"name":"Puerto Rico"},{"id":74,"name":"U.S. Minor Outlying Islands"},{"id":78,"name":"U.S. Virgin Islands"}]

Despite the crm_backend container being in state 'exited' it still is responding to http requests, and the website continues to work. This is strange.

cypressf commented 2 years ago

Looking at the logs of the 'exited' crm_backend pod, they continue to be written to. There's also an error that looks like happened when the latest deploy occurred, but the process is still running.

podman logs crm_backend
[2022-07-15T17:20:25Z INFO  actix_web::middleware::logger] 10.0.2.100 "GET /data/65?source=11&start_date=2020-01-01&end_date=2020-12-31 HTTP/1.1" 200 35054 "https://est.mit.edu/5" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.114 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 0.024228
Error: Os { code: 98, kind: AddrInUse, message: "Address already in use" }
[2022-07-15T17:33:26Z INFO  actix_web::middleware::logger] 10.0.2.100 "GET /map-visualization?include_drafts=false HTTP/1.1" 200 155400 "https://est.mit.edu/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.5 Safari/605.1.15" 0.765390