Unable to use Carrier Home app with Infinitude on Raspberry Pi Docker container

I've got Infinitude running as a docker container on a Raspberry Pi 4 and set the proxy on my SYSTXCCITC01-B thermostat. I'm able to see the data coming into the Infinitude page and can integrate it with Home Assistant, but my android app Carrier Home reports that "This system has not connected to Wi-Fi recently" unless I disable the proxy.

Is this an expected result of using Infinitude or should it pass the same data along to Carrier? Originally I thought it may be related to me changing the port to 3001 since I had Grafana using 3000, but I swapped ports so Infinitude is running on the default port 3000 now and the app is still unable to communicate.

I'm using the following command on the RPI4 to start it: docker run --rm -v $PWD/state:/infinitude/state -e APP_SECRET='123456zxcv' -e PASS_REQS='1020' -e MODE='Production' -p 3000:3000 nebulous/infinitude

There is nothing obviously wrong with what you've done above. Infinitude should send unique requests up to Carrier's servers a maximum of once every 17 minutes(1020 seconds). If you watch the logs in development mode, there may be some clues. You should see messages like No cache for foo. Make Carrier request

I must admit that support for Carrier's apps has never been a priority for me since the point of Infinitude is to replicate the cloud service so I can control the stat without Carrier's cloud service ala HomeAssistant.

Both iOS apps working here. I have found after any setup changes it can take an hour or more for carriers web service to play nice. This includes where there is no infinitude at all.

Get Outlook for iOShttps://aka.ms/o0ukef

From: nebulous @.> Sent: Wednesday, August 4, 2021 9:57:22 AM To: nebulous/infinitude @.> Cc: Subscribed @.***> Subject: Re: [nebulous/infinitude] Unable to use Carrier Home app with Infinitude? (#121)

I must admit that support for Carrier's apps has never been a priority for me since the point of Infinitude is to replicate the cloud service so I can control the stat without Carrier's cloud service ala HomeAssistant.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnebulous%2Finfinitude%2Fissues%2F121%23issuecomment-892818341&data=04%7C01%7C%7C39e198fc097c45620efc08d95768edf7%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637636930440840013%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=MlyEvVxXy5cVOcrfqKKi1GBlqC9%2F2WXNtWV0ylhtZaY%3D&reserved=0, or unsubscribehttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FACWXVXAWYUMPHJX6VHQJUATT3FWPFANCNFSM5BRJCZKA&data=04%7C01%7C%7C39e198fc097c45620efc08d95768edf7%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637636930440850012%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=3zlot6G4vK1FmMZA69GWcRtZSqTHLPERrglOrR4A2Fo%3D&reserved=0.

I changed the proxy to point to a Windows PC running Docker Desktop and the same commands and then I could see the status through the Android app, so it does seem to be specific to my Pi4 only. I don't have another Pi4 around, but I could get a Pi3 setup with the same thing, to see if it is specific to that installation, or how I installed Docker or something.

I haven't tried running the container in development mode yet, but running "docker logs infinitude" on my current setup shows what nebulous indicated I would see


[2021-08-04 19:44:16.09231] [8] [info] systems-foo-status cached or passthru disabled
[2021-08-04 19:44:16.09308] [8] [info] /systems/foo/status
[2021-08-04 19:44:16.12130] [8] [info] Saving status
[2021-08-04 19:44:28.30183] [8] [info] No cache for systems-foo-status. Make Carrier request
[2021-08-04 19:44:28.38932] [8] [info] /systems/foo/status
[2021-08-04 19:44:28.40236] [8] [info] Saving status
[2021-08-04 19:44:40.35787] [8] [info] systems-foo-status cached or passthru disabled
[2021-08-04 19:44:40.35867] [8] [info] /systems/foo/status
[2021-08-04 19:44:40.37402] [8] [info] Saving status
[2021-08-04 19:44:52.56072] [8] [info] systems-foo-status cached or passthru disabled
[2021-08-04 19:44:52.56164] [8] [info] /systems/foo/status
[2021-08-04 19:44:52.57809] [8] [info] Saving status
[2021-08-04 19:45:04.75735] [8] [info] systems-foo-status cached or passthru disabled
[2021-08-04 19:45:04.75812] [8] [info] /systems/foo/status
[2021-08-04 19:45:04.77274] [8] [info] Saving status
[2021-08-04 19:45:16.94669] [8] [info] systems-foo-status cached or passthru disabled
[2021-08-04 19:45:16.94741] [8] [info] /systems/foo/status

I downloaded the latest Raspberry Pi image on a Pi3 and installed Docker using

curl -sSL https://get.docker.com | sh
sudo usermod -aG docker pi
logout
docker run -v $PWD/state:/infinitude/state  -e PASS_REQS='60' -e MODE='development' -p 3000:3000 nebulous/infinitude

After setting the proxy on the thermostat to this Pi3 IP, I have the same behavior, infinitude is receiving the data, but after more than 1 hour, the Carrier Home app doesn't think my thermostat is connected.

Here is some of the log output, showing "Make Carrier request" line mentioned earlier, replacing my serial with all 1s

[2021-08-05 15:00:08.80152] [8] [debug] 200 OK (0.253182s, 3.950/s)
[2021-08-05 15:00:10.17837] [8] [debug] GET "/api/status" (afd8fee7)
[2021-08-05 15:00:10.17941] [8] [debug] Routing to a callback
[2021-08-05 15:00:10.18209] [8] [debug] 200 OK (0.003644s, 274.424/s)
[2021-08-05 15:00:16.38397] [8] [info] No cache for systems-111111111111-odu_config. Make Carrier request
[2021-08-05 15:00:16.46356] [8] [info] /systems/111111111111/odu_config
[2021-08-05 15:00:16.47018] [8] [info] Saving odu_config
[2021-08-05 15:00:16.48025] [8] [debug] POST "/systems/111111111111/odu_config" (67814cd7)
[2021-08-05 15:00:16.48160] [8] [debug] Routing to a callback
[2021-08-05 15:00:16.48197] [8] [info] Unimplemented request: odu_config
[2021-08-05 15:00:16.48282] [8] [debug] 200 OK (0.002531s, 395.101/s)
[2021-08-05 15:00:16.87151] [8] [info] systems-111111111111-status cached or passthru disabled
[2021-08-05 15:00:16.87234] [8] [info] /systems/111111111111/status
[2021-08-05 15:00:16.88845] [8] [info] Saving status
[2021-08-05 15:00:16.90713] [8] [debug] POST "/systems/111111111111/status" (819ca88a)
[2021-08-05 15:00:16.90825] [8] [debug] Routing to a callback
[2021-08-05 15:00:16.91205] [8] [debug] 200 OK (0.00487s, 205.339/s)
[2021-08-05 15:00:29.48317] [8] [info] systems-111111111111-status cached or passthru disabled
[2021-08-05 15:00:29.48394] [8] [info] /systems/111111111111/status
[2021-08-05 15:00:29.49842] [8] [info] Saving status
[2021-08-05 15:00:29.51533] [8] [debug] POST "/systems/111111111111/status" (d1cec358)
[2021-08-05 15:00:29.51633] [8] [debug] Routing to a callback

I then went back to Docker on my Windows 10 desktop using the same command and updated the proxy to go to this IP and within 5 minutes the Carrier Home app showed connected. I didn't see anything significant in the logs other than a new entry for "Check Carrier/Bryant change flags"

[2021-08-05 16:29:59.46909] [8] [info] systems-111111111111-status cached or passthru disabled
[2021-08-05 16:29:59.46935] [8] [info] /systems/111111111111/status
[2021-08-05 16:29:59.47205] [8] [info] Saving status
[2021-08-05 16:29:59.51009] [8] [debug] POST "/systems/111111111111/status" (2074e014)
[2021-08-05 16:29:59.51039] [8] [debug] Routing to a callback
[2021-08-05 16:29:59.51829] [8] [debug] 200 OK (0.008126s, 123.062/s)
[2021-08-05 16:30:11.71740] [8] [info] systems-111111111111-status cached or passthru disabled
[2021-08-05 16:30:11.71814] [8] [info] /systems/111111111111/status
[2021-08-05 16:30:11.72812] [8] [info] Saving status
[2021-08-05 16:30:11.78462] [8] [debug] POST "/systems/111111111111/status" (850955c0)
[2021-08-05 16:30:11.78495] [8] [debug] Routing to a callback
[2021-08-05 16:30:11.79412] [8] [debug] 200 OK (0.009475s, 105.541/s)
[2021-08-05 16:30:23.73420] [8] [info] No cache for systems-111111111111-status. Make Carrier request
[2021-08-05 16:30:23.97561] [8] [info] /systems/111111111111/status
[2021-08-05 16:30:23.97807] [8] [info] Saving status
[2021-08-05 16:30:24.01747] [8] [debug] POST "/systems/111111111111/status" (9dc7b0fb)
[2021-08-05 16:30:24.01774] [8] [debug] Routing to a callback
[2021-08-05 16:30:24.02564] [8] [info] ********** Check Carrier/Bryant change flags ****************
[2021-08-05 16:30:24.02834] [8] [debug] 200 OK (0.010842s, 92.234/s)

The Desktop is hard wired, while the Pis are all wireless. Entering the container I was able to verify that I could resolve external IPs and ping public addresses, so they have outbound connectivity.

Any other hints? Seems to be only on the Pi devices that I'm having trouble. I mostly want to be able to enter Vacation mode if I'm away and use the Energy Usage part of the app.

File permissions? The state folder I sometimes have to wipe, since after multiple restarts of the container the Status sometimes shows an old timestamp, even though the actual JSON response has the proper date Firewall? I've tried disabling iptables on the Pi which can really break docker, but it doesn't seem like a regular requirement to modify it for containers

perl makes my head hurt, but in the debug log snippet from the Pi the "make carrier request" line is for odu_config, not for status.

If you're not getting the "change flags" line for the Pi, then the Pi is getting back different data from Carrier than the desktop is. You might get useful information by adding $c->app->log->info($data); to the try block around line 109 of infinitude and seeing what is logged on the desktop versus what is logged on the Pi.

If that raw response doesn't show the issue, maybe logging the request itself would show the issue.

Once you have a working instance, as you do, your debugging plan is to see what differs in the request and response between the working and nonworking instances. If the request differs, find out why. If the response differs, try to create a curl command that emulates the request and see if that gives different responses on the desktop host (non-container) versus the Pi host (non-container). If it doesn't, try running that curl within a container.

I added the line to the infinitude file and I do see more data in the logs, but nothing that stands out to me.

From the Pi container:

[2021-08-06 16:05:24.59080] [7] [info] Saving status
[2021-08-06 16:05:24.60275] [7] [debug] POST "/systems/11111111111/status" (b68cc7c7)
[2021-08-06 16:05:24.60390] [7] [debug] Routing to a callback
[2021-08-06 16:05:24.60963] [7] [debug] 200 OK (0.006846s, 146.071/s)
[2021-08-06 16:05:26.33094] [7] [debug] GET "/api/status" (911f7d6a)
[2021-08-06 16:05:26.33154] [7] [debug] Routing to a callback
[2021-08-06 16:05:26.33295] [7] [debug] 200 OK (0.00197s, 507.614/s)
[2021-08-06 16:05:36.57196] [7] [info] No cache for systems-11111111111-status. Make Carrier request
[2021-08-06 16:05:36.64593] [7] [info] /systems/11111111111/status
[2021-08-06 16:05:36.65330] [7] [info] <status version="1.41"><localTime>2021-08-06T11:04:36-06:01</localTime><oat>82</oat><mode>off</mode><cfgem>F</cfgem><cfgtype>heatcool</cfgtype><vacatrunning>off</vacatrunning><filtrlvl>0</filtrlvl><uvlvl>1</uvlvl><humlvl>1</humlvl><ventlvl>10</ventlvl><humid>off</humid><oprstsmsg/><idu><type>furnacemodulating</type><opstat>off</opstat><cfm>0</cfm></idu><odu><type>gs3ngipac</type><opstat>off</opstat><opmode>off</opmode></odu><zones><zone id="1"><name>ZONE 1</name><enabled>on</enabled><currentActivity>home</currentActivity><rt>74.0</rt><rh>50</rh><fan>off</fan><htsp>68.0</htsp><clsp>74.0</clsp><hold>off</hold><otmr/><zoneconditioning>idle</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="2"><name>Zone 2</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>50</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>idle</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="3"><name>Zone 3</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>50</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>idle</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="4"><name>Zone 4</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>50</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>idle</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="5"><name>Zone 5</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>50</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>idle</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="6"><name>Zone 6</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>50</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>idle</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="7"><name>Zone 7</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>50</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>idle</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="8"><name>Zone 8</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>50</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>idle</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone></zones></status>
[2021-08-06 16:05:36.66140] [7] [info] Saving status
[2021-08-06 16:05:36.67027] [7] [debug] POST "/systems/11111111111/status" (8182fad5)
[2021-08-06 16:05:36.67112] [7] [debug] Routing to a callback
[2021-08-06 16:05:36.67351] [7] [debug] 200 OK (0.003203s, 312.207/s)
[2021-08-06 16:05:41.47306] [7] [debug] GET "/systems.json" (6bb2a64c)
[2021-08-06 16:05:41.47376] [7] [debug] Routing to a callback
[2021-08-06 16:05:41.47532] [7] [debug] 200 OK (0.002105s, 475.059/s)
[2021-08-06 16:05:41.48488] [7] [debug] GET "/status.json" (2e4e7eff)
[2021-08-06 16:05:41.48672] [7] [debug] Routing to a callback
[2021-08-06 16:05:41.48837] [7] [debug] 200 OK (0.003326s, 300.661/s)
[2021-08-06 16:05:41.49406] [7] [debug] GET "/energy.json" (bfcea4f3)
[2021-08-06 16:05:41.49642] [7] [debug] Routing to a callback
[2021-08-06 16:05:41.49762] [7] [debug] 200 OK (0.0035s, 285.714/s)
[2021-08-06 16:05:41.50465] [7] [debug] GET "/notifications.json" (fb7366e4)
[2021-08-06 16:05:41.50536] [7] [debug] Routing to a callback
[2021-08-06 16:05:41.50661] [7] [debug] 200 OK (0.001904s, 525.210/s)
[2021-08-06 16:05:41.51077] [7] [debug] GET "/status.json" (b9508e39)
[2021-08-06 16:05:41.51293] [7] [debug] Routing to a callback
[2021-08-06 16:05:41.51407] [7] [debug] 200 OK (0.00325s, 307.692/s)
[2021-08-06 16:05:41.52229] [7] [debug] GET "/energy.json" (16786f31)
[2021-08-06 16:05:41.52291] [7] [debug] Routing to a callback
[2021-08-06 16:05:41.52417] [7] [debug] 200 OK (0.001815s, 550.964/s)
[2021-08-06 16:05:41.52720] [7] [debug] GET "/systems.json" (885846fe)
[2021-08-06 16:05:41.52794] [7] [debug] Routing to a callback
[2021-08-06 16:05:41.52932] [7] [debug] 200 OK (0.00207s, 483.092/s)
[2021-08-06 16:05:41.53389] [7] [debug] GET "/notifications.json" (7df7529d)
[2021-08-06 16:05:41.53459] [7] [debug] Routing to a callback
[2021-08-06 16:05:41.53587] [7] [debug] 200 OK (0.001921s, 520.562/s)
[2021-08-06 16:05:48.59131] [7] [info] systems-11111111111-status cached or passthru disabled
[2021-08-06 16:05:48.59177] [7] [info] /systems/11111111111/status
[2021-08-06 16:05:48.59879] [7] [info] <status version="1.41"><localTime>2021-08-06T11:04:51-06:01</localTime><oat>82</oat><mode>off</mode><cfgem>F</cfgem><cfgtype>heatcool</cfgtype><vacatrunning>off</vacatrunning><filtrlvl>0</filtrlvl><uvlvl>1</uvlvl><humlvl>1</humlvl><ventlvl>10</ventlvl><humid>off</humid><oprstsmsg/><idu><type>furnacemodulating</type><opstat>off</opstat><cfm>0</cfm></idu><odu><type>gs3ngipac</type><opstat>off</opstat><opmode>off</opmode></odu><zones><zone id="1"><name>ZONE 1</name><enabled>on</enabled><currentActivity>home</currentActivity><rt>74.0</rt><rh>50</rh><fan>off</fan><htsp>68.0</htsp><clsp>74.0</clsp><hold>off</hold><otmr/><zoneconditioning>idle</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="2"><name>Zone 2</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>50</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>idle</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="3"><name>Zone 3</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>50</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>idle</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="4"><name>Zone 4</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>50</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>idle</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="5"><name>Zone 5</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>50</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>idle</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="6"><name>Zone 6</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>50</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>idle</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="7"><name>Zone 7</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>50</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>idle</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="8"><name>Zone 8</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>50</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>idle</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone></zones></status>
[2021-08-06 16:05:48.59918] [7] [info] Saving status
[2021-08-06 16:05:48.61815] [7] [debug] POST "/systems/11111111111/status" (b7e3769c)
[2021-08-06 16:05:48.61890] [7] [debug] Routing to a callback
[2021-08-06 16:05:48.62094] [7] [debug] 200 OK (0.00276s, 362.319/s)

Desktop container looks similar, but as mentioned, I do see the lines about the 'change flags' here that I don't on the Pi

[2021-08-06 16:52:28.28901] [7] [info] No cache for systems-11111111111-status. Make Carrier request
[2021-08-06 16:52:28.68223] [7] [info] /systems/11111111111/status
[2021-08-06 16:52:28.68315] [7] [info] <status version="1.41"><localTime>2021-08-06T11:51:28-06:01</localTime><oat>84</oat><mode>cool</mode><cfgem>F</cfgem><cfgtype>heatcool</cfgtype><vacatrunning>off</vacatrunning><filtrlvl>0</filtrlvl><uvlvl>1</uvlvl><humlvl>1</humlvl><ventlvl>10</ventlvl><humid>off</humid><oprstsmsg/><idu><type>furnacemodulating</type><opstat>off</opstat><cfm>320</cfm></idu><odu><type>gs3ngipac</type><opstat>73</opstat><opmode>cooling</opmode></odu><zones><zone id="1"><name>ZONE 1</name><enabled>on</enabled><currentActivity>home</currentActivity><rt>74.0</rt><rh>49</rh><fan>off</fan><htsp>68.0</htsp><clsp>74.0</clsp><hold>off</hold><otmr/><zoneconditioning>active_cool</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="2"><name>Zone 2</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>49</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>active_cool</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="3"><name>Zone 3</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>49</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>active_cool</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="4"><name>Zone 4</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>49</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>active_cool</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="5"><name>Zone 5</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>49</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>active_cool</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="6"><name>Zone 6</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>49</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>active_cool</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="7"><name>Zone 7</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>49</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>active_cool</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="8"><name>Zone 8</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>49</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>active_cool</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone></zones></status>
[2021-08-06 16:52:28.68566] [7] [info] Saving status
[2021-08-06 16:52:28.73038] [7] [debug] POST "/systems/11111111111/status" (972b806c)
[2021-08-06 16:52:28.73065] [7] [debug] Routing to a callback
[2021-08-06 16:52:28.73533] [7] [info] ********** Check Carrier/Bryant change flags ****************
[2021-08-06 16:52:28.73786] [7] [debug] 200 OK (0.007457s, 134.102/s)
[2021-08-06 16:52:40.40643] [7] [info] systems-11111111111-status cached or passthru disabled
[2021-08-06 16:52:40.40667] [7] [info] /systems/11111111111/status
[2021-08-06 16:52:40.40725] [7] [info] <status version="1.41"><localTime>2021-08-06T11:51:38-06:01</localTime><oat>84</oat><mode>cool</mode><cfgem>F</cfgem><cfgtype>heatcool</cfgtype><vacatrunning>off</vacatrunning><filtrlvl>0</filtrlvl><uvlvl>1</uvlvl><humlvl>1</humlvl><ventlvl>10</ventlvl><humid>off</humid><oprstsmsg/><idu><type>furnacemodulating</type><opstat>off</opstat><cfm>320</cfm></idu><odu><type>gs3ngipac</type><opstat>72</opstat><opmode>cooling</opmode></odu><zones><zone id="1"><name>ZONE 1</name><enabled>on</enabled><currentActivity>home</currentActivity><rt>74.0</rt><rh>49</rh><fan>off</fan><htsp>68.0</htsp><clsp>74.0</clsp><hold>off</hold><otmr/><zoneconditioning>active_cool</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="2"><name>Zone 2</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>49</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>active_cool</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="3"><name>Zone 3</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>49</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>active_cool</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="4"><name>Zone 4</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>49</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>active_cool</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="5"><name>Zone 5</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>49</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>active_cool</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="6"><name>Zone 6</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>49</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>active_cool</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="7"><name>Zone 7</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>49</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>active_cool</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="8"><name>Zone 8</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>49</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>active_cool</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone></zones></status>
[2021-08-06 16:52:40.40914] [7] [info] Saving status
[2021-08-06 16:52:40.45038] [7] [debug] POST "/systems/11111111111/status" (936e0655)
[2021-08-06 16:52:40.45066] [7] [debug] Routing to a callback
[2021-08-06 16:52:40.45775] [7] [debug] 200 OK (0.007353s, 135.999/s)
[2021-08-06 16:52:41.86727] [7] [debug] GET "/systems.json" (ef44204d)
[2021-08-06 16:52:41.86751] [7] [debug] Routing to a callback
[2021-08-06 16:52:41.87626] [7] [debug] 200 OK (0.008967s, 111.520/s)
[2021-08-06 16:52:41.89183] [7] [debug] GET "/status.json" (a9591deb)
[2021-08-06 16:52:41.89207] [7] [debug] Routing to a callback
[2021-08-06 16:52:41.90023] [7] [debug] 200 OK (0.008379s, 119.346/s)
[2021-08-06 16:52:41.91125] [7] [debug] GET "/notifications.json" (314e8914)
[2021-08-06 16:52:41.91154] [7] [debug] Routing to a callback
[2021-08-06 16:52:41.91675] [7] [debug] 200 OK (0.00548s, 182.482/s)
[2021-08-06 16:52:41.92205] [7] [debug] GET "/energy.json" (63153ffd)
[2021-08-06 16:52:41.92227] [7] [debug] Routing to a callback
[2021-08-06 16:52:41.92432] [7] [debug] 200 OK (0.002244s, 445.633/s)
[2021-08-06 16:52:41.94002] [7] [debug] GET "/systems.json" (c98503db)
[2021-08-06 16:52:41.94028] [7] [debug] Routing to a callback
[2021-08-06 16:52:41.94677] [7] [debug] 200 OK (0.006729s, 148.610/s)
[2021-08-06 16:52:41.94822] [7] [debug] GET "/status.json" (938b96d3)
[2021-08-06 16:52:41.94849] [7] [debug] Routing to a callback
[2021-08-06 16:52:41.95245] [7] [debug] 200 OK (0.004204s, 237.869/s)
[2021-08-06 16:52:41.96644] [7] [debug] GET "/notifications.json" (3fd7c290)
[2021-08-06 16:52:41.96668] [7] [debug] Routing to a callback
[2021-08-06 16:52:41.96946] [7] [debug] 200 OK (0.002989s, 334.560/s)
[2021-08-06 16:52:41.97869] [7] [debug] GET "/energy.json" (d9791b1c)
[2021-08-06 16:52:41.97893] [7] [debug] Routing to a callback
[2021-08-06 16:52:41.98160] [7] [debug] 200 OK (0.002881s, 347.102/s)
[2021-08-06 16:52:52.66752] [7] [info] systems-11111111111-status cached or passthru disabled
[2021-08-06 16:52:52.66795] [7] [info] /systems/11111111111/status
[2021-08-06 16:52:52.66919] [7] [info] <status version="1.41"><localTime>2021-08-06T11:51:53-06:01</localTime><oat>84</oat><mode>cool</mode><cfgem>F</cfgem><cfgtype>heatcool</cfgtype><vacatrunning>off</vacatrunning><filtrlvl>0</filtrlvl><uvlvl>1</uvlvl><humlvl>1</humlvl><ventlvl>10</ventlvl><humid>off</humid><oprstsmsg/><idu><type>furnacemodulating</type><opstat>off</opstat><cfm>458</cfm></idu><odu><type>gs3ngipac</type><opstat>72</opstat><opmode>cooling</opmode></odu><zones><zone id="1"><name>ZONE 1</name><enabled>on</enabled><currentActivity>home</currentActivity><rt>74.0</rt><rh>49</rh><fan>off</fan><htsp>68.0</htsp><clsp>74.0</clsp><hold>off</hold><otmr/><zoneconditioning>active_cool</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="2"><name>Zone 2</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>49</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>active_cool</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="3"><name>Zone 3</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>49</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>active_cool</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="4"><name>Zone 4</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>49</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>active_cool</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="5"><name>Zone 5</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>49</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>active_cool</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="6"><name>Zone 6</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>49</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>active_cool</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="7"><name>Zone 7</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>49</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>active_cool</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone><zone id="8"><name>Zone 8</name><enabled>off</enabled><currentActivity>away</currentActivity><rt/><rh>49</rh><fan>off</fan><htsp>60.0</htsp><clsp>80.0</clsp><hold>off</hold><otmr/><zoneconditioning>active_cool</zoneconditioning><damperposition>15</damperposition><occupancy>unoccupied</occupancy><occupancyOverride>off</occupancyOverride></zone></zones></status>
[2021-08-06 16:52:52.67306] [7] [info] Saving status
[2021-08-06 16:52:52.72012] [7] [debug] POST "/systems/11111111111/status" (452f81ab)
[2021-08-06 16:52:52.72040] [7] [debug] Routing to a callback
[2021-08-06 16:52:52.72484] [7] [debug] 200 OK (0.004644s, 215.332/s)

In the 'status' folder, I did notice that the res-* files all have similar content in the pi containers

HTTP/1.1 404 Not Found
Content-Length: 0

while the Desktop container has successes

HTTP/1.1 200 OK
Via: 1.1 249aa6f890bf73000af910ed0724d955.cloudfront.net (CloudFront)
Date: Thu, 05 Aug 2021 17:06:30 GMT
Content-Type: application/xml
Content-Length: 1169

I'm trying to piece together a curl command that should work, but when I get some of the data from the req-* files and try to run them, I often just get another command prompt with no data returned from the command, error or success, so not sure if I'm doing it right or not.

Could there be something in the Pi architecture that isn't supporting one of the required commands?

Good work looking at res- and req-. Check the req files for any differences between the Pi and the desktop.

curl -v will tell you more. curl printing nothing and exiting is compatible with a 404 with Content-length 0. You're looking for a situation where the curl on the Desktop returns status 200 with data, and the same curl on the Pi returns 404 with Content-length 0.

I think I'm hot on the trail.

If I enter the container and do

apt update && apt upgrade -y

it prompts for 22 packages to be upgrades

The following packages will be upgraded:
  apt base-files ca-certificates debian-archive-keyring iproute2 iputils-ping libapt-pkg5.0 libdatetime-timezone-perl libgcrypt20 libgnutls30 libhogweed4
  libjs-jquery liblz4-1 libnettle6 libp11-kit0 libssl1.1 libsystemd0 libudev1 libxml2 libzstd1 openssl tzdata
22 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

After updating all of these and committing the container and restarting, everything starts working as expected. My res-* files start returning a 200 OK status.

Going to work through them one at a time to try to determine which package in particular is causing it, but finally feel like I'm on the right track.

Good work. Perhaps an old/insecure openssl?

Looks like the package that needs updated is: ca-certificates

By going back to the original docker image and then only doing

apt update
apt install ca-certificates -y

and then commit the container changes (since I couldn't figure out how to restart infinitude from within the container) and restart with my new image, things start working with res- files showing a success.

During the update, this is what it shows, which looks to be a minor update:

Unpacking ca-certificates (20200601~deb10u2) over (20200601~deb10u1) ...

@scyto, it you maintain the docker hub images, could you test the docker image with this package updated and consider including it in the main public image?

Seems odd to me that it only impacts the Raspberry Pi from my testing

(After I upgraded all of the outdated packages, it seemed like not as many items were created in the state folder after 5 min of running, so there could be something going on with it, but I was running in development mode then and now running with production mode. Going to test more later today)

I was also having this problem and found this thread. This fixed my issue. Thank you!!!

Thanks for debugging that issue, @MallocArray. I forced a new build of the docker containers (which I'm thinking should result in a less stale ca-certificates package) that should be available on dockerhub soon.

Odd, I pulled the latest image and while the ca-certificates is showing as updated, I'm still getting 404 errors in the response file on the Pi.

Q⚌a⚌⚌⚌⚌⚌⚌⚌⚌HTTP/1.1 404 Not Found
Content-Length: 0
Date: Thu, 12 Aug 2021 23:05:53 GMT

root@infinitude:/infinitude# apt list --installed | grep ca-cer

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

ca-certificates/now 20200601~deb10u2 all [installed,local]

edit: I may have some oddness going on. I switched back to my image and was still getting 404 until I stopped and removed the state folder again and then it started working. I'll do further testing with the updated image from docker

Something still isn't right. I verified there are no pending updates in the new image, but I'm still getting 404 on all res-systems* files. Verified the kernel in the new image is the same as the previous: 5.10.17-v7l+

Logs look the same in Production mode:

[2021-08-13 18:16:44.63771] [7] [info] No cache for systems-111111111111-status. Make Carrier request
[2021-08-13 18:16:44.75767] [7] [info] /systems/111111111111/status
[2021-08-13 18:16:44.76455] [7] [info] Saving status
[2021-08-13 18:16:56.83419] [7] [info] systems-111111111111-status cached or passthru disabled
[2021-08-13 18:16:56.83536] [7] [info] /systems/111111111111/status
[2021-08-13 18:16:56.84902] [7] [info] Saving status
[2021-08-13 18:17:08.91351] [7] [info] systems-111111111111-status cached or passthru disabled
[2021-08-13 18:17:08.91463] [7] [info] /systems/111111111111/status
[2021-08-13 18:17:08.92852] [7] [info] Saving status
[2021-08-13 18:17:20.99377] [7] [info] systems-111111111111-status cached or passthru disabled
[2021-08-13 18:17:20.99514] [7] [info] /systems/111111111111/status
[2021-08-13 18:17:21.01137] [7] [info] Saving status
[2021-08-13 18:17:33.10828] [7] [info] systems-111111111111-status cached or passthru disabled
[2021-08-13 18:17:33.10960] [7] [info] /systems/111111111111/status
[2021-08-13 18:17:33.12570] [7] [info] Saving status
[2021-08-13 18:17:45.22523] [7] [info] No cache for systems-111111111111-status. Make Carrier request
[2021-08-13 18:17:45.32914] [7] [info] /systems/111111111111/status
[2021-08-13 18:17:45.33914] [7] [info] Saving status
[2021-08-13 18:17:57.43556] [7] [info] systems-111111111111-status cached or passthru disabled
[2021-08-13 18:17:57.43685] [7] [info] /systems/111111111111/status
[2021-08-13 18:17:57.45655] [7] [info] Saving status
[2021-08-13 18:18:09.51909] [7] [info] systems-111111111111-status cached or passthru disabled
[2021-08-13 18:18:09.52047] [7] [info] /systems/111111111111/status
[2021-08-13 18:18:09.53637] [7] [info] Saving status
[2021-08-13 18:18:21.59626] [7] [info] systems-111111111111-status cached or passthru disabled
[2021-08-13 18:18:21.59771] [7] [info] /systems/111111111111/status
[2021-08-13 18:18:21.61352] [7] [info] Saving status
[2021-08-13 18:18:33.67025] [7] [info] systems-111111111111-status cached or passthru disabled
[2021-08-13 18:18:33.67132] [7] [info] /systems/111111111111/status
[2021-08-13 18:18:33.68551] [7] [info] Saving status
[2021-08-13 18:18:45.83347] [7] [info] No cache for systems-111111111111-status. Make Carrier request
[2021-08-13 18:18:45.96288] [7] [info] /systems/111111111111/status
[2021-08-13 18:18:45.97871] [7] [info] Saving status

Went back to my image based on the previously released docker image but manually did the apt upgrade inside of it and everything is working as expected.

Very curious...

Not sure if this is indicative of the issue or not, but after removing my local state folder and running the latest image, I get 404 errors in the res-systems files as described above. If I leave these files present and stop the container and change image to my custom one and start that image (using docker-compose) the files are not updated and still show the 404 error. The timestamp does not increment. If I keep the image running and then delete .dat from the state folder, new files are generated that are successful. Does this indicate a permission or ownership issue? They look to be the same before or after removing them.

-rw-r--r-- 1 root root 1.6K Aug 13 19:36 res-systems-111111111111-status+2etxt.dat

I may have missed it -- but I have the following questions.

When you initially bring up a container from the released image, and infinitude is failing, does curl -v also fail on the same URLs?

When you do whatever you do to the container so that infinitude starts working, does curl -v also start to succeed on the URLs it had failed on?

Since your problem is subtle it is really important to isolate it to as few moving parts as possible. It is really unclear to me why updating ca-certificates could cause the status of an HTTP response to change. If there was a CA cert issue, you should get no response at all -- the TLS socket should not pass any data, even an error response. So either Mojolicious is doing something distasteful in the event of an insecure cert, or there wasn't an insecure cert and something else caused it to start working.

I've not been able to figure out the proper curl commands to try on either the raspberry pi host or inside of the container, and the container does not have curl installed, so I already have to modify the image just to test that.

With the previous container image, I was able to repeatedly test the default container was giving 404 in the res- files, and after only installing the latest ca-certificates and re-committing the image so I could reboot with the updated package, the res- files started having proper responses.

With the latest image, I still get 404 out of the box, but there are no updates to install and re-committing the image does not resolve it. I moved from one SD card to another one so now I no longer have access to the former image on the new OS install, so I may need to flip back to the original SD card on my side to try to pull it out for further testing.

I agree that a 404 error is unexpected, but it was what I am currently encountering, and the new image with the latest ca-certificates installed is no longer resolving it. I may try forcefully downgrading the ca-certs and then upgrading again to see if anything else was maybe installed or modified during that process.

I'm just about out of ideas, but I do have one. When you updated ca-certificates before, was ca-certificates the only package updated, or were any of its dependencies updated? Like openssl for example? That is something that could cause a 404 with an old version versus a 200 with a newer version, if the server is configured to give a 404 for older TLS versions or cipher suites, for example.

That's not something you or I would ordinarily do, we'd ordinarily just fail the TLS handshake, but iirc Carrier has a CDN in front of their server. If Carrier were going to evaluate the client's TLS parameters on their servers (instead of telling the CDN what to pass and what to drop), they might reject via a 404.

No dependencies that I remember being updated, for sure not openssl. I just tried manually installing openssl for good measure but no luck.

I did get into PostMan on Windows and copied all of the fields found in the req- files, other than oauth_signature that I couldn't find a field in Postman for, but it seemed to calculate it. I tried both the Dealer POST and the Time GET commands. Closest I could come was an error saying the nonce was already used. I was able to get this same error with Postman in Windows, Raspberry Pi OS, and inside the container once I installed curl. So it seems to be resolving the hostnames enough to know that the nonce was used by infinitude already.

I wanted to make sure it wasn't OpenDNS causing the problem, so I changed the container to use 4.2.2.2 as a DNS server with no luck. I also tried installing openssl to make sure it had the latest.

@ericreich we are stuck solving this problem. You were seeing the same issue and you solved it by updating your ca-certificates, right?

Could you help by trying the latest image and seeing whether that works for you? Keep your old image around because possibly the new image will have the same problem and you won't be able to fix it by upgrading ca-certificates...

I was able to export my custom image from my other SD card and copied it over to the new one. It started right up and is getting proper HTTP/1.1 200 OK responses.

Any way to do a compare between two images?

That's constructive. https://github.com/GoogleContainerTools/container-diff is the thing I've used to diff images.

hi, i was on vacation and only paying half attention to the thread

I have a pi with docker i use for other purposes, if it would help i could pull the latest image and test too? Let me know if thats worth doing.

@dulitz Yes I can later this weekend. I have noticed the updating ca did fix the issue at first, I also have to restart the docker a couple times a day at first to keep it connected. But for the last 2 days or so, I have not had to restart the docker at all.

@scyto It would be helpful to have additional data points.

Using the original image from 10 months ago, I loaded it up using docker-compose and confirmed that I was getting 404 in the res- files. I then got into the container and did an apt-update and then existed and did a docker commit as a new image. I then modified the docker-compose.yml file to use the new image and loaded it up, confirmed I was still getting 404 errors and then got into the container and did

root@infinitude:/infinitude# apt install ca-certificates
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be upgraded:
  ca-certificates
1 upgraded, 0 newly installed, 0 to remove and 21 not upgraded.
Need to get 166 kB of archives.
After this operation, 21.5 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian buster/main armhf ca-certificates all 20200601~deb10u2 [166 kB]
Fetched 166 kB in 0s (889 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
(Reading database ... 15020 files and directories currently installed.)
Preparing to unpack .../ca-certificates_20200601~deb10u2_all.deb ...
Unpacking ca-certificates (20200601~deb10u2) over (20200601~deb10u1) ...
Setting up ca-certificates (20200601~deb10u2) ...
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76.)
debconf: falling back to frontend: Readline
Updating certificates in /etc/ssl/certs...
11 added, 0 removed; done.
Processing triggers for ca-certificates (20200601~deb10u2) ...
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.

Exited the container, did another docker commit and updated docker-compose to use this new image with only ca-certificates updated and started it back up. Confirmed that I started getting 200 OK in the response files. Saved both images to .tar files and used container-diff to compare them. Here are all of the listed files that were changed, but I don't know if I fully trust it, since I get a bunch of errors the first time I run the command and running it a second time with the same options returns this data:

.\container-diff.exe diff infinitude-2020-11-apt-update.tar infinitude-2020-11-ca-cert-upgrade.tar --type=file

-----File-----

These entries have been added to infinitude-2020-11-apt-update.tar:
FILE                                                                                                       SIZE
\etc\ca-certificates.conf.dpkg-old                                                                         5.3K
\usr                                                                                                       256.4K
\usr\local                                                                                                 0
\usr\local\share                                                                                           0
\usr\local\share\ca-certificates                                                                           0
\usr\sbin                                                                                                  5.2K
\usr\sbin\update-ca-certificates                                                                           5.2K
\usr\share                                                                                                 251.2K
\usr\share\ca-certificates                                                                                 208.9K
\usr\share\ca-certificates\mozilla                                                                         208.9K
\usr\share\ca-certificates\mozilla\ACCVRAIZ1.crt                                                           2.7K
\usr\share\ca-certificates\mozilla\AC_RAIZ_FNMT-RCM.crt                                                    1.9K
\usr\share\ca-certificates\mozilla\Actalis_Authentication_Root_CA.crt                                      2K
\usr\share\ca-certificates\mozilla\AffirmTrust_Commercial.crt                                              1.2K
\usr\share\ca-certificates\mozilla\AffirmTrust_Networking.crt                                              1.2K
\usr\share\ca-certificates\mozilla\AffirmTrust_Premium.crt                                                 1.8K
\usr\share\ca-certificates\mozilla\AffirmTrust_Premium_ECC.crt                                             753B
\usr\share\ca-certificates\mozilla\Amazon_Root_CA_1.crt                                                    1.2K
\usr\share\ca-certificates\mozilla\Amazon_Root_CA_2.crt                                                    1.8K
\usr\share\ca-certificates\mozilla\Amazon_Root_CA_3.crt                                                    656B
\usr\share\ca-certificates\mozilla\Amazon_Root_CA_4.crt                                                    737B
\usr\share\ca-certificates\mozilla\Atos_TrustedRoot_2011.crt                                               1.2K
\usr\share\ca-certificates\mozilla\Autoridad_de_Certificacion_Firmaprofesional_CIF_A62634068.crt           2.1K
\usr\share\ca-certificates\mozilla\Baltimore_CyberTrust_Root.crt                                           1.2K
\usr\share\ca-certificates\mozilla\Buypass_Class_2_Root_CA.crt                                             1.9K
\usr\share\ca-certificates\mozilla\Buypass_Class_3_Root_CA.crt                                             1.9K
\usr\share\ca-certificates\mozilla\CA_Disig_Root_R2.crt                                                    1.9K
\usr\share\ca-certificates\mozilla\CFCA_EV_ROOT.crt                                                        1.9K
\usr\share\ca-certificates\mozilla\COMODO_Certification_Authority.crt                                      1.5K
\usr\share\ca-certificates\mozilla\COMODO_ECC_Certification_Authority.crt                                  940B
\usr\share\ca-certificates\mozilla\COMODO_RSA_Certification_Authority.crt                                  2K
\usr\share\ca-certificates\mozilla\Certigna.crt                                                            1.3K
\usr\share\ca-certificates\mozilla\Certigna_Root_CA.crt                                                    2.2K
\usr\share\ca-certificates\mozilla\Certum_Trusted_Network_CA.crt                                           1.3K
\usr\share\ca-certificates\mozilla\Certum_Trusted_Network_CA_2.crt                                         2K
\usr\share\ca-certificates\mozilla\Chambers_of_Commerce_Root_-_2008.crt                                    2.5K
\usr\share\ca-certificates\mozilla\Comodo_AAA_Services_root.crt                                            1.5K
\usr\share\ca-certificates\mozilla\Cybertrust_Global_Root.crt                                              1.3K
\usr\share\ca-certificates\mozilla\D-TRUST_Root_Class_3_CA_2_2009.crt                                      1.5K
\usr\share\ca-certificates\mozilla\D-TRUST_Root_Class_3_CA_2_EV_2009.crt                                   1.5K
\usr\share\ca-certificates\mozilla\DST_Root_CA_X3.crt                                                      1.2K
\usr\share\ca-certificates\mozilla\DigiCert_Assured_ID_Root_CA.crt                                         1.3K
\usr\share\ca-certificates\mozilla\DigiCert_Assured_ID_Root_G2.crt                                         1.3K
\usr\share\ca-certificates\mozilla\DigiCert_Assured_ID_Root_G3.crt                                         851B
\usr\share\ca-certificates\mozilla\DigiCert_Global_Root_CA.crt                                             1.3K
\usr\share\ca-certificates\mozilla\DigiCert_Global_Root_G2.crt                                             1.3K
\usr\share\ca-certificates\mozilla\DigiCert_Global_Root_G3.crt                                             839B
\usr\share\ca-certificates\mozilla\DigiCert_High_Assurance_EV_Root_CA.crt                                  1.3K
\usr\share\ca-certificates\mozilla\DigiCert_Trusted_Root_G4.crt                                            1.9K
\usr\share\ca-certificates\mozilla\E-Tugra_Certification_Authority.crt                                     2.2K
\usr\share\ca-certificates\mozilla\EC-ACC.crt                                                              1.9K
\usr\share\ca-certificates\mozilla\EE_Certification_Centre_Root_CA.crt                                     1.4K
\usr\share\ca-certificates\mozilla\Entrust.net_Premium_2048_Secure_Server_CA.crt                           1.5K
\usr\share\ca-certificates\mozilla\Entrust_Root_Certification_Authority.crt                                1.6K
\usr\share\ca-certificates\mozilla\Entrust_Root_Certification_Authority_-_EC1.crt                          1.1K
\usr\share\ca-certificates\mozilla\Entrust_Root_Certification_Authority_-_G2.crt                           1.5K
\usr\share\ca-certificates\mozilla\Entrust_Root_Certification_Authority_-_G4.crt                           2.2K
\usr\share\ca-certificates\mozilla\GDCA_TrustAUTH_R5_ROOT.crt                                              1.9K
\usr\share\ca-certificates\mozilla\GTS_Root_R1.crt                                                         1.9K
\usr\share\ca-certificates\mozilla\GTS_Root_R2.crt                                                         1.9K
\usr\share\ca-certificates\mozilla\GTS_Root_R3.crt                                                         769B
\usr\share\ca-certificates\mozilla\GTS_Root_R4.crt                                                         769B
\usr\share\ca-certificates\mozilla\GeoTrust_Global_CA.crt                                                  1.2K
\usr\share\ca-certificates\mozilla\GeoTrust_Primary_Certification_Authority.crt                            1.2K
\usr\share\ca-certificates\mozilla\GeoTrust_Primary_Certification_Authority_-_G2.crt                       989B
\usr\share\ca-certificates\mozilla\GeoTrust_Primary_Certification_Authority_-_G3.crt                       1.4K
\usr\share\ca-certificates\mozilla\GeoTrust_Universal_CA.crt                                               1.9K
\usr\share\ca-certificates\mozilla\GeoTrust_Universal_CA_2.crt                                             1.9K
\usr\share\ca-certificates\mozilla\GlobalSign_ECC_Root_CA_-_R4.crt                                         713B
\usr\share\ca-certificates\mozilla\GlobalSign_ECC_Root_CA_-_R5.crt                                         794B
\usr\share\ca-certificates\mozilla\GlobalSign_Root_CA.crt                                                  1.2K
\usr\share\ca-certificates\mozilla\GlobalSign_Root_CA_-_R2.crt                                             1.3K
\usr\share\ca-certificates\mozilla\GlobalSign_Root_CA_-_R3.crt                                             1.2K
\usr\share\ca-certificates\mozilla\GlobalSign_Root_CA_-_R6.crt                                             1.9K
\usr\share\ca-certificates\mozilla\Global_Chambersign_Root_-_2008.crt                                      2.5K
\usr\share\ca-certificates\mozilla\Go_Daddy_Class_2_CA.crt                                                 1.4K
\usr\share\ca-certificates\mozilla\Go_Daddy_Root_Certificate_Authority_-_G2.crt                            1.3K
\usr\share\ca-certificates\mozilla\Hellenic_Academic_and_Research_Institutions_ECC_RootCA_2015.crt         1017B
\usr\share\ca-certificates\mozilla\Hellenic_Academic_and_Research_Institutions_RootCA_2011.crt             1.5K
\usr\share\ca-certificates\mozilla\Hellenic_Academic_and_Research_Institutions_RootCA_2015.crt             2.1K
\usr\share\ca-certificates\mozilla\Hongkong_Post_Root_CA_1.crt                                             1.1K
\usr\share\ca-certificates\mozilla\Hongkong_Post_Root_CA_3.crt                                             2K
\usr\share\ca-certificates\mozilla\ISRG_Root_X1.crt                                                        1.9K
\usr\share\ca-certificates\mozilla\IdenTrust_Commercial_Root_CA_1.crt                                      1.9K
\usr\share\ca-certificates\mozilla\IdenTrust_Public_Sector_Root_CA_1.crt                                   1.9K
\usr\share\ca-certificates\mozilla\Izenpe.com.crt                                                          2.1K
\usr\share\ca-certificates\mozilla\LuxTrust_Global_Root_2.crt                                              2K
\usr\share\ca-certificates\mozilla\Microsec_e-Szigno_Root_CA_2009.crt                                      1.4K
\usr\share\ca-certificates\mozilla\NetLock_Arany_=Class_Gold=_Főtanúsítvány.crt                            1.4K
\usr\share\ca-certificates\mozilla\Network_Solutions_Certificate_Authority.crt                             1.4K
\usr\share\ca-certificates\mozilla\OISTE_WISeKey_Global_Root_GA_CA.crt                                     1.4K
\usr\share\ca-certificates\mozilla\OISTE_WISeKey_Global_Root_GB_CA.crt                                     1.3K
\usr\share\ca-certificates\mozilla\OISTE_WISeKey_Global_Root_GC_CA.crt                                     895B
\usr\share\ca-certificates\mozilla\QuoVadis_Root_CA.crt                                                    2K
\usr\share\ca-certificates\mozilla\QuoVadis_Root_CA_1_G3.crt                                               1.9K
\usr\share\ca-certificates\mozilla\QuoVadis_Root_CA_2.crt                                                  2K
\usr\share\ca-certificates\mozilla\QuoVadis_Root_CA_2_G3.crt                                               1.9K
\usr\share\ca-certificates\mozilla\QuoVadis_Root_CA_3.crt                                                  2.3K
\usr\share\ca-certificates\mozilla\QuoVadis_Root_CA_3_G3.crt                                               1.9K
\usr\share\ca-certificates\mozilla\SSL.com_EV_Root_Certification_Authority_ECC.crt                         956B
\usr\share\ca-certificates\mozilla\SSL.com_EV_Root_Certification_Authority_RSA_R2.crt                      2.1K
\usr\share\ca-certificates\mozilla\SSL.com_Root_Certification_Authority_ECC.crt                            944B
\usr\share\ca-certificates\mozilla\SSL.com_Root_Certification_Authority_RSA.crt                            2K
\usr\share\ca-certificates\mozilla\SZAFIR_ROOT_CA2.crt                                                     1.2K
\usr\share\ca-certificates\mozilla\SecureSign_RootCA11.crt                                                 1.2K
\usr\share\ca-certificates\mozilla\SecureTrust_CA.crt                                                      1.3K
\usr\share\ca-certificates\mozilla\Secure_Global_CA.crt                                                    1.3K
\usr\share\ca-certificates\mozilla\Security_Communication_RootCA2.crt                                      1.2K
\usr\share\ca-certificates\mozilla\Security_Communication_Root_CA.crt                                      1.2K
\usr\share\ca-certificates\mozilla\Sonera_Class_2_Root_CA.crt                                              1.1K
\usr\share\ca-certificates\mozilla\Staat_der_Nederlanden_EV_Root_CA.crt                                    1.9K
\usr\share\ca-certificates\mozilla\Staat_der_Nederlanden_Root_CA_-_G2.crt                                  2K
\usr\share\ca-certificates\mozilla\Staat_der_Nederlanden_Root_CA_-_G3.crt                                  1.9K
\usr\share\ca-certificates\mozilla\Starfield_Class_2_CA.crt                                                1.4K
\usr\share\ca-certificates\mozilla\Starfield_Root_Certificate_Authority_-_G2.crt                           1.4K
\usr\share\ca-certificates\mozilla\Starfield_Services_Root_Certificate_Authority_-_G2.crt                  1.4K
\usr\share\ca-certificates\mozilla\SwissSign_Gold_CA_-_G2.crt                                              2K
\usr\share\ca-certificates\mozilla\SwissSign_Silver_CA_-_G2.crt                                            2K
\usr\share\ca-certificates\mozilla\T-TeleSec_GlobalRoot_Class_2.crt                                        1.3K
\usr\share\ca-certificates\mozilla\T-TeleSec_GlobalRoot_Class_3.crt                                        1.3K
\usr\share\ca-certificates\mozilla\TUBITAK_Kamu_SM_SSL_Kok_Sertifikasi_-_Surum_1.crt                       1.5K
\usr\share\ca-certificates\mozilla\TWCA_Global_Root_CA.crt                                                 1.8K
\usr\share\ca-certificates\mozilla\TWCA_Root_Certification_Authority.crt                                   1.2K
\usr\share\ca-certificates\mozilla\Taiwan_GRCA.crt                                                         1.9K
\usr\share\ca-certificates\mozilla\TeliaSonera_Root_CA_v1.crt                                              1.8K
\usr\share\ca-certificates\mozilla\TrustCor_ECA-1.crt                                                      1.5K
\usr\share\ca-certificates\mozilla\TrustCor_RootCert_CA-1.crt                                              1.5K
\usr\share\ca-certificates\mozilla\TrustCor_RootCert_CA-2.crt                                              2.2K
\usr\share\ca-certificates\mozilla\Trustis_FPS_Root_CA.crt                                                 1.2K
\usr\share\ca-certificates\mozilla\UCA_Extended_Validation_Root.crt                                        1.9K
\usr\share\ca-certificates\mozilla\UCA_Global_G2_Root.crt                                                  1.8K
\usr\share\ca-certificates\mozilla\USERTrust_ECC_Certification_Authority.crt                               948B
\usr\share\ca-certificates\mozilla\USERTrust_RSA_Certification_Authority.crt                               2K
\usr\share\ca-certificates\mozilla\VeriSign_Class_3_Public_Primary_Certification_Authority_-_G4.crt        1.3K
\usr\share\ca-certificates\mozilla\VeriSign_Class_3_Public_Primary_Certification_Authority_-_G5.crt        1.7K
\usr\share\ca-certificates\mozilla\VeriSign_Universal_Root_Certification_Authority.crt                     1.7K
\usr\share\ca-certificates\mozilla\Verisign_Class_3_Public_Primary_Certification_Authority_-_G3.crt        1.4K
\usr\share\ca-certificates\mozilla\XRamp_Global_CA_Root.crt                                                1.5K
\usr\share\ca-certificates\mozilla\certSIGN_ROOT_CA.crt                                                    1.1K
\usr\share\ca-certificates\mozilla\ePKI_Root_Certification_Authority.crt                                   2K
\usr\share\ca-certificates\mozilla\emSign_ECC_Root_CA_-_C3.crt                                             814B
\usr\share\ca-certificates\mozilla\emSign_ECC_Root_CA_-_G3.crt                                             859B
\usr\share\ca-certificates\mozilla\emSign_Root_CA_-_C1.crt                                                 1.2K
\usr\share\ca-certificates\mozilla\emSign_Root_CA_-_G1.crt                                                 1.3K
\usr\share\ca-certificates\mozilla\thawte_Primary_Root_CA.crt                                              1.5K
\usr\share\ca-certificates\mozilla\thawte_Primary_Root_CA_-_G2.crt                                         940B
\usr\share\ca-certificates\mozilla\thawte_Primary_Root_CA_-_G3.crt                                         1.5K
\usr\share\doc                                                                                             41.1K
\usr\share\doc\ca-certificates                                                                             41.1K
\usr\share\doc\ca-certificates\README.Debian                                                               2.8K
\usr\share\doc\ca-certificates\changelog.gz                                                                14.4K
\usr\share\doc\ca-certificates\copyright                                                                   18.5K
\usr\share\doc\ca-certificates\examples                                                                    5.5K
\usr\share\doc\ca-certificates\examples\ca-certificates-local                                              5.5K
\usr\share\doc\ca-certificates\examples\ca-certificates-local\Makefile                                     199B
\usr\share\doc\ca-certificates\examples\ca-certificates-local\README                                       3K
\usr\share\doc\ca-certificates\examples\ca-certificates-local\debian                                       2K
\usr\share\doc\ca-certificates\examples\ca-certificates-local\debian\ca-certificates-local.triggers        38B
\usr\share\doc\ca-certificates\examples\ca-certificates-local\debian\changelog                             160B
\usr\share\doc\ca-certificates\examples\ca-certificates-local\debian\compat                                2B
\usr\share\doc\ca-certificates\examples\ca-certificates-local\debian\control                               824B
\usr\share\doc\ca-certificates\examples\ca-certificates-local\debian\copyright                             164B
\usr\share\doc\ca-certificates\examples\ca-certificates-local\debian\postrm                                741B
\usr\share\doc\ca-certificates\examples\ca-certificates-local\debian\rules                                 155B
\usr\share\doc\ca-certificates\examples\ca-certificates-local\debian\source                                13B
\usr\share\doc\ca-certificates\examples\ca-certificates-local\debian\source\format                         13B
\usr\share\doc\ca-certificates\examples\ca-certificates-local\local                                        237B
\usr\share\doc\ca-certificates\examples\ca-certificates-local\local\Local_Root_CA.crt                      123B
\usr\share\doc\ca-certificates\examples\ca-certificates-local\local\Makefile                               114B
\usr\share\man                                                                                             1.2K
\usr\share\man\man8                                                                                        1.2K
\usr\share\man\man8\update-ca-certificates.8.gz                                                            1.2K
\var\cache                                                                                                 1.7M
\var\cache\apt                                                                                             0
\var\cache\apt\archives                                                                                    0
\var\cache\apt\archives\lock                                                                               0
\var\cache\apt\archives\partial                                                                            0
\var\cache\debconf                                                                                         1.7M
\var\cache\debconf\config.dat                                                                              25K
\var\cache\debconf\config.dat-old                                                                          23.9K
\var\cache\debconf\passwords.dat                                                                           0
\var\cache\debconf\templates.dat                                                                           851.4K
\var\cache\debconf\templates.dat-old                                                                       851.4K
\var\lib\apt\extended_states                                                                               13.9K
\var\lib\dpkg                                                                                              511.9K
\var\lib\dpkg\info                                                                                         67K
\var\lib\dpkg\info\ca-certificates.config                                                                  9.5K
\var\lib\dpkg\info\ca-certificates.list                                                                    10.7K
\var\lib\dpkg\info\ca-certificates.md5sums                                                                 15.1K
\var\lib\dpkg\info\ca-certificates.postinst                                                                5.7K
\var\lib\dpkg\info\ca-certificates.postrm                                                                  1.6K
\var\lib\dpkg\info\ca-certificates.templates                                                               24.4K
\var\lib\dpkg\info\ca-certificates.triggers                                                                70B
\var\lib\dpkg\lock                                                                                         0
\var\lib\dpkg\lock-frontend                                                                                0
\var\lib\dpkg\status                                                                                       222.4K
\var\lib\dpkg\status-old                                                                                   222.4K
\var\lib\dpkg\triggers                                                                                     32B
\var\lib\dpkg\triggers\Lock                                                                                0
\var\lib\dpkg\triggers\Unincorp                                                                            0
\var\lib\dpkg\triggers\update-ca-certificates                                                              16B
\var\lib\dpkg\triggers\update-ca-certificates-fresh                                                        16B
\var\lib\dpkg\updates                                                                                      0
\var\log                                                                                                   191.5K
\var\log\apt                                                                                               73.7K
\var\log\apt\eipp.log.xz                                                                                   10.3K
\var\log\apt\history.log                                                                                   11.7K
\var\log\apt\term.log                                                                                      51.7K
\var\log\dpkg.log                                                                                          117.8K

These entries have been deleted from infinitude-2020-11-apt-update.tar: None

These entries have been changed between infinitude-2020-11-apt-update.tar and infinitude-2020-11-ca-cert-upgrade.tar:
FILE                                      SIZE1         SIZE2
\etc\ssl\certs\ca-certificates.crt        193.8K        208.9K
\etc\ca-certificates.conf                 5.3K          5.8K
\root\.bash_history                       16B           49B

Using the latest image of infinitude, this package already shows it has been updated (as do all installed packages) but I still get 404 errors. I tried to reinstall ca-certificates but things really went sideways and I gave up on that attempt.

Note: I do fully delete the state folder from the host machine each time, as I've had mixed luck with it updating the files even after starting a container that I have proven will work. @ericreich would that possibly be related to your issues of needing to restart the container? Once I've got it showing 200 OK messages, I've not had a problem with it failing while the same container was running.

ok, i will updated my running container (not on pi) to baseline everything still works there (i know from your tests it should!) then i will replicate the pull on pi and see what happens, will do it at weekend most likely

@dulitz So I attempted to load a new container but failed. I am running Homeassistant with Portainer. I was able to easily deploy the original image with no problems. It even messed up my original. All I can see is that the container is created then immediately removed. I do see some errors something about a empty Body.. But cannot get it to show up again.

So I am basically in a broken state now. I tried to delete all Infinitude containers, but nothing will load now.

Here is the message I am getting now trying to deploy this docker image.. "failure starting container with non-empty request body was deprecated since API v1.22 and removed in v1.24"

Ohnoes! I don't know anything about Portainer or why loading a different version of a container would break you. I hope someone else does...

@ericreich I've also not used Portainer before, but it looks like it can import and export standard Docker images.

I've shared two container images to the following share: https://1drv.ms/u/s!Ap_rFXZF9o-fh54t6TjXk1GDYa3oQA?e=mhqPSw

Download one or both of these and try to import them with Portainer. You may need to change the name of the image you use when you deploy it, depending on how it imports in.

infinitude-2020-11.tar - The original image from last year with no modifications. Was giving 404 errors on Raspberry Pi, but a good reference infinitude-2021-08-07.tar - The image from last year after I updated all packages including ca-certificates that is working for me on a Raspberry Pi.

Got the container running again. But now Wifi on t-stat shows everything is connected, but the app is old stale data and shows wifi errors. This is the "latest" version of this container.. I tried to update certs, but looks like this version is now running the latest version. It did not upgrade. With the previous container I had installed, I only had to update the certs and it started working, with just restarts of the container ever so often.

If you are using the nebulous/infinitude:latest image, then that matches what I was experiencing. If you want to try one of the images that I added a link to, you may be able to get back to the operating state with the Carrier App.

I forgot I was running my docker fork and that i had a vacation booked for the last weekend.

Here is what i squeezed in during a boring meeting over the last hour:

updated my docker on synology to use scyto/infinitude:latest (which is a year old) and verify the pull still works
put the same container on my pi4 - works just dandy locally, just waiting for it to update carriers backend and be in the app...

next step, update my container image to latest infinitude (i am running version, hmm just looked no idea); will post back when I have more, sorry for slow progress.

@scyto, it you maintain the docker hub images, could you test the docker image with this package updated and consider including it in the main public image?

can do, what i don't understand is why this only affects pi and not my 1 year + container! but ¯(°_o)/¯ i can report i am having simillar flaky issues on my pi - whats weird is at one point the thermostat said it was connected quite happily and the web page populated and the app saw it.... but then it didn't - i have always found carriers backend to be flaky

hmm my brain has fade on how we made the changes and propogated, pulling the main nebulous images now - i need to go look at the build actions as the compose file has stuff in it that isn't needed..... i think.... (aka the compose file didn't work without edit) - need to get my head backup to speed.

@MallocArray I will try the older versions and see what happens..

Something else I noticed. Since mine was not working, I turned the proxy off on the t-stat.. After about 30 minutes, I am not getting alerts about software updates at the t-stat. Apparently having the proxy configured was not allowing hardware / firmware updates to the t-stat, furnace, or condensing unit. (I have a new 26 SEER inverter driven system).

@ericreich my older version on pi didn't work :-(

this is like the issue i had getting the thermostat working initially - its flaky as shit

i can't tell if this is an infinitude issue or a carrier backend issue or some new interplay (i have my existing container on my synology still running we can look at if needed!) because sometimes the t-stat says its connected to the endpoint API but infinitude insists its not - its all very weird.

i know from previous experience i have sometime seen it take hours for the t-stat / app to sort themselves out irrespective of whether infinitude is proxy-ing or not. I have now just spun up the new ebnulous/infinitude image from 13 days ago to see if I can coax it to work or not....

@ericreich I suspected that having the proxy set would not allow software updates, since going to the Service menu only shows updating from a MicroSD card, while screenshots show it should have the option for Web updates. I hadn't tried disabling the proxy, but had a feeling that could be preventing it.

@scyto I wasn't sure how the official Docker Hub images work, but I remember seeing your name associated with the multi-arch builds, so wasn't sure if you were in charge of that area. Nebulous had made a change to the source project and I think that kicked off the automated builds for the nebulous/infinitude:latest image, but we could use some assistance in figuring out why it isn't working. You don't necessarily need to update your own published Docker image.

I have had issues with leaving the tab for the main page open and the Global gauges go blank until a manual refresh, but that doesn't seem to impact the actual API. I'm primarily using Infinitude so my Home Assistant configuration can make REST calls to pull various metrics from the system to log in the database and graph over time. I have the variable speed system, so I like having a graph of the actual speeds. I'm not seeing gaps in that, so for me, the Infinitude API seems to be stable. The only issue is that some of the images don't seem to pass this along to Carrier properly, while the manual update to ca-certificates on the previous image does.

I don't understand how the build process works for various architectures, or how Docker knows to pull down an image that works for Arm. Is there any chance it is getting some component that is actually for x86 and isn't working with the Pi, or is that all handled by the build software and wouldn't be a concern?

@MallocArray thanks, I was struggling with the thread

I think you have everything working when you install the certs right?

A few thoughts:

the build process is quite simple, we could add ca-certificates to the list on line 7 of the dockerfile and that should fix (github will automatically re-build after that's committed to master)
i am still mystified why this would affect pi and not other systems, but we could spend ages worrying about that when in reality it sounds like we have a fix.

to your question and #2 its possible there is some fundamental difference in the base image its using from https://hub.docker.com/_/ubuntu/ (this is what the dockerfile uses as the base, buildx selects from the :latest tags based on architecture, its possible there is a bug in :latest of arm image, i have seen this before, and since building the infinitude docker buildx logic have found its sometimes better to use a fixed tag number (downside you don't get updates for free/easy) for example one latest tag of alpine was utterly borked on pi!

@nebulous i can make the change, would you be willing to commit and then tell us when the action has built the image (or we can all wait until I have time to properly fork and remake the actions for my fork to do full test before commit)

ok i think we should scratch my suggestion and PR above @nebulous @MallocArray when i am in latest container from 13 days ago it says it has latest certs?


apt install ca-certificates
Reading package lists... Done
Building dependency tree        
Reading state information... Done
ca-certificates is already the newest version (20200601~deb10u2).
ca-certificates set to manually installed.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.```

@scyto Here is a current summary: Symptoms: When Infinity System Control thermostat is configured to use a proxy address hosted by Infinitude on a Raspberry Pi 3/4 Docker container (2020-11 version), the Carrier App (confirmed on Android) no longer shows the thermostat being connected to Wifi. Verifying the contents in the "state" folder on the Raspberry Pi, the res-* files show 404 at the top of the file and no response contents. Only seems to affect the container on Raspberry Pi. Using the same image on a Windows Docker container works as expected.

Steps to resolve: Access the shell of the container, running 'apt update' and 'apt install ca-certificates' to perform an update of ca-certificates from 20200601\~deb10u1 to 20200601\~deb10u2. Exit the shell and do a 'docker commit infinitude infinitude-updated' to save the updated image with upgraded package. (Alternatively, update all out of date packages in the container) Stop the current container and restart using the new image. Observe that the contents of the res-* files in the state folder now have '200 OK' at the top of the file with proper response files

Complication: Nebulous issued a new commit to trigger a new build on 2021-08-10. This image replaced the Docker Hub image with the 'latest' tag. This container also has the issue of all res- files having 404 responses, but all packages are already running current versions, including ca-certificates 20200601~deb10u2. Since all packages are already current, the previous fix of updating ca-certificates cannot be used to resume proper operation. Doing a forced reinstall of ca-certificates does not appear to resolve the issue either.

Summary: The previous fix of updating ca-certificates with the older container image does not resolve the issue with the newly built image that already contains the updated package. Unsure how to test from here.

Just getting back from vacation and trying to catch up. I don’t have a pi to test but I’m definitely fine with merging a pr or making a change as needed to test. When dealing with the random result generator that is carriers web service it can be frustrating, but I’m happy to help as my limited time allows.

On Aug 23, 2021, at 7:25 PM, MallocArray @.***> wrote:

@scyto Here is a current summary: Symptoms: When Infinity System Control thermostat is configured to use a proxy address hosted by Infinitude on a Raspberry Pi 3/4 Docker container (2020-11 version), the Carrier App (confirmed on Android) no longer shows the thermostat being connected to Wifi. Verifying the contents in the "state" folder on the Raspberry Pi, the res-* files show 404 at the top of the file and no response contents. Only seems to affect the container on Raspberry Pi. Using the same image on a Windows Docker container works as expected.

Steps to resolve: Access the shell of the container, running 'apt update' and 'apt install ca-certificates' to perform an update of ca-certificates from 20200601deb10u1 to 20200601deb10u2. Exit the shell and do a 'docker commit infinitude infinitude-updated' to save the updated image with upgraded package. (Alternatively, update all out of date packages in the container) Stop the current container and restart using the new image. Observe that the contents of the res-* files in the state folder now have '200 OK' at the top of the file with proper response files

Complication: Nebulous issued a new commit to trigger a new build on 2021-08-10. This image replaced the Docker Hub image with the 'latest' tag. This container also has the issue of all res- files having 404 responses, but all packages are already running current versions, including ca-certificates 20200601~deb10u2. Since all packages are already current, the previous fix of updating ca-certificates cannot be used to resume proper operation. Doing a forced reinstall of ca-certificates does not appear to resolve the issue either.

Summary: The previous fix of updating ca-certificates with the older container image does not resolve the issue with the newly built image that already contains the updated package. Unsure how to test from here.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

@nebulous thanks, tldr as i was being too fucking random above

@MallocArray found a fix for his scenario to update certs in the image he had
i repro the issue with pi and t-start and having 404 errors in the dat so assumed his fix would be it
but on further inspection the docker image from 13 days ago has the latest certs and i still see the issue as per mallocarray

as such i think there is good chance something else is going on, don't know what, reforking and starting from base nebulous repo so i can play

50/50 this is some backend change or issue at carrier IMO, but will work from ground up checking image, build process, etc

Ok this issue is driving me nuts. its exactly like @MallocArray explains.

If one uses the nebulous/infinitude image one has the issue where all the res. files are 96k each and have 404 error in them.
Then if one execs into the running container do apt remove ca-certificates (this also removes liblwp-protocol-https-perl libwww-perl libxml-parser-perl libxml-sax-expat-perl) and then re-installs all of the above and stops and starts the container then it all seems to working (at least the res files have the right data in them) despite the versions not changing.

I have done about 10 build variations using github buildx to try and fix during build and nothing does.
This means there is either an issue with: a) the underlying images provided by debian b) the buildx build process in github

When i have time (maybe this weekend) i will pull back to local build process and see if i have the same issue or not. Will have to pause for the night, my head is exploding.

@scyto is working on this and I'm just kibitzing. I chatted with a friend who has a great deal of experience with Docker, including on Raspbian, and he confirmed that this is all very weird and he'd never seen anything similar.

@MallocArray used container-diff, which was good. But my friend suggested that we look at the output of ls -laR on the containers that work versus the ones that don't. (Such as before and after upgrading ca-certs.) He suspects that buildx might be using a deterministic system time in order to achieve deterministic builds. While that's generally a good thing, he wonders whether it's leading to the certificate files having modification times that are incompatible with the certs themselves. (E.g. file modification time older than the cert's Not Valid Before time.)

@MallocArray if you have time to send those, I'll take a look fwiw.

@dulitz I got the output of the command you requested, but the command may need to be tweaked, as it is only showing the high level folders and not the contents of the folders with certificates in them.

The Latest container that does not work: https://pastebin.com/a8Q47JEA

The container from the 2020-11 date that I manually installed the latest packages and should have the same versions as the one above: https://pastebin.com/9cWEh230

I did see that there are two packages that have been updated that I could install, but I haven't tried it yet libssl1.1 openssl

Ah, sorry, what I meant was ls -laR /

@dulitz Oh, of course. Thanks.

https://www.diffchecker.com/Mt4FXFXZ Above is the diff between the two, but there is a lot to go through. My manually updated one on the left side, and the latest container on the right

Thanks @MallocArray . diffchecker won't give me the files directly and I've just waited 10 minutes for a copy paste to complete and it hasn't. Can you do pastebin again?

PasteBin has a 512k limit and these are 2.5 MB each. I put them here: https://1drv.ms/u/s!Ap_rFXZF9o-fh54t6TjXk1GDYa3oQA?e=OgcBw8

Thanks so much, I have downloaded latest and manually-updated and will take a look this afternoon.

nebulous / infinitude

Unable to use Carrier Home app with Infinitude on Raspberry Pi Docker container #121