gcobb321 / icloud3_v3

iCloud3 v3, Development Version - This Repo/Version is used for development and testing of new and updated features. It is not the official release of iCloud3 v3, .
MIT License
169 stars 13 forks source link

Notifications...: WAITING FOR NOTIFY SVC TO START #338

Closed nzrutman closed 2 weeks ago

nzrutman commented 1 month ago

Logs have this for all devices with Mobile App enabled:

10-01 15:31:55 STAGE5   Tracked Device > icloud3_nzr (iphone NZR/iPhone)
                            • iCloud Tracking Parameters:
                               ◦ FamShr Device: Blintz
                            • Mobile App Tracking Parameters:
                               ◦ Device Tracker: device_tracker.blintz
                               ◦ Action Trigger.: blintz_last_update_trigger
                               ◦ Battery Sensor: blintz_battery_level
                               ◦ Notifications...: WAITING FOR NOTIFY SVC TO START"
                            • Other Parameters:
                               ◦ inZone Interval: 1 hr
                               ◦ Fixed Interval: 30 mins

This is true on every iC3 restart, while HA remains running; i.e. not a HA startup race condition (noticed the recheck in the code). If I'm reading it right, the lack of Device.mobapp[NOTIFY] means MobApp tracking requests will be ignored, which seems to be the case.

action: notify.mobile_app_blintz exists and works correctly.

iCloud3 3.0.5.9 HA 2024.9.3

smarthomefamilyverrips commented 1 month ago

Maybe the above is also causing the integration to say Mobile App is not available in configuration???

372783825-1320c7be-5581-49bf-9128-1e3c7824dde4

gcobb321 commented 1 month ago

When iCloud3 loads, it reads the HA entity registry file and extracts the entities for the mobile App platform. It looks for the device tracker, battery and last update trigger entities. When HA is finished loading, it gets the mobile app notify action service call info directly from HA. It gets it from HA after HA finishes starting since the mobile app can be loaded before or after iCloud3.

If the mobile app info it’s not returned from HA, it will not use t the mobile app as a data source. So, yes, the mobile app integration is necessary to use the mobile app as a data source. It also must not be disabled on the integrations screen

smarthomefamilyverrips commented 1 month ago

When iCloud3 loads, it reads the HA entity registry file and extracts the entities for the mobile App platform. It looks for the device tracker, battery and last update trigger entities. When HA is finished loading, it gets the mobile app notify action service call info directly from HA. It gets it from HA after HA finishes starting since the mobile app can be loaded before or after iCloud3.

If the mobile app info it’s not returned from HA, it will not use t the mobile app as a data source. So, yes, the mobile app integration is necessary to use the mobile app as a data source. It also must not be disabled on the integrations screen

@gcobb321 at me the Mobile App is active on the iPhone and the integration is also up and running (see screenshots), nothing is disabled but still I get the message that Mobile App is not installed

Screenshot_20241003_093801_Home Assistant Screenshot_20241003_093815_Home Assistant

gcobb321 commented 1 month ago

A list of devices is built from the mobile apps returned from HA when iCloud3 starts, as I mentioned. The mobile app selected on the Update Devices screen is checked for all of the devices. If a devices config entry is not found, it will create that error message. The error message is confusing if only 1 device has an issue and the rest are ok. However, if it does find an error for a device, only that device will not use the mobile app. The rest will still use it. Check all of your devices in the Update Devices screen a see if there are any that need to be rejected. I will look at the message on the next update.

smarthomefamilyverrips commented 1 month ago

But if the device is available in the mobile app why it still not get used then in the iCloudV3 integration, if I check event log it says "none" under Mobile App location. But at startup it does see the iPhone mobile app entity and the very first update seconds after start up of the iCloudV3 integration it does show the mobile app location in the events log, but as soon comes the first periodic update (5 minutes as iphone is home) it start show "none" at mobile app location in events log

JtwoA commented 1 month ago

Is the mobile app set to share location always?

John Alan

Sent with Proton Mail secure email.

www.johnalan.org

01000011 01110101 01110010 01101001 01101111 01110011 01101001

01110100 01111001 00100000 01101001 01110011 00100000 01110100 01101000 01100101 00100000 01101101 01101111 01110011 01110100 00100000 01110110 01100001 01101100 01110101 01100001 01100010 01101100 01100101 00100000 01100110 01100101 01100001 01110100 01110101 01110010 01100101 00100000 01101111 01100110 00100000 01101001 01101110 01110100 01100101 01101100 01101100 01101001 01100111 01100101 01101110 01110100 00100000 01101100 01101001 01100110 01100101 00101110

On Thursday, October 3rd, 2024 at 04:28, sesame_madness @.***> wrote:

But if the device is available in the mobile app why it still not get used then in the iCloudV3 integration, if I check event log it says "none" under Mobile App location. But at startup it does see the iPhone mobile app entity and the very first update seconds after start up of the iCloudV3 integration it does show the mobile app location in the events log, but as soon comes the first periodic update (5 minutes as iphone is home) it start show "none" at mobile app location in events log

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: @.***>

gcobb321 commented 1 month ago

@smarthomefamilyverrips Maybe i can answer the Why question if i see some Log files that show what is really going on with your system and see how it is set up and running.

  1. Set iCloud3 Log Level to Debug in the Configure Settings > Page 2 > Format Parameters screen
  2. Delete the Home Assistant log file so I’m not looking at a lot of old entries (config/home-assistant.log)
  3. Restart HA
  4. Wait for 10-minutes or so until it does the locate and the Mobile App state goes to None or Not Used or when it stops tracking the Mobile App devices
  5. Zip the home-assistant.log and icloud3-0.log files and email them to geekstergary@gmail.com
nzrutman commented 1 month ago

I do not have verrips' problem with his config screen, so maybe these are unrelated issues. My enabled Mobile App Devices are correctly found:

10-01 15:31:55 STAGE4   Mobile App Devices > 2 of 4 used by iCloud3
                            ✓Parmesan1, parmesan1 (iPhone12,8) >
                                ◦icloud3_zar, iphone ZAR (Monitored)
                            ✓Blintz, blintz (iPhone15,2) >
                                ◦icloud3_nzr, iphone NZR
                            × PopTart, familys_ipad (iPad6,11) >
                                ◦Not Assigned to an iCloud3 Device
...
10-01 15:31:55 STAGE5 ⡇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━�
��━━━━━━━━━━━━━━━━━
                                          ⡇    STAGE 5 > DEVICE CONFIGURATION SUMMARY
                                          ⡇━━━━━━━━━━━━━━━━━━━━━━━━━�
�━━━━━━━━━━━━━━━━━━━━━━━━
10-01 15:31:55 STAGE5   Monitored Device > icloud3_zar (iphone ZAR/iPhone)
                            • iCloud Tracking Parameters:
                               ◦ FamShr Device: Parmesan
                            • Mobile App Tracking Parameters:
                               ◦ Device Tracker: device_tracker.parmesan1
                               ◦ Action Trigger.: parmesan1_last_update_trigger
                               ◦ Battery Sensor: parmesan1_battery_level
                               ◦ Notifications...: WAITING FOR NOTIFY SVC TO START"
                            • Other Parameters:
                               ◦ inZone Interval: 1 hr
                               ◦ Fixed Interval: 30 mins
10-01 15:31:55 STAGE5   Tracked Device > icloud3_nzr (iphone NZR/iPhone)
                            • iCloud Tracking Parameters:
                               ◦ FamShr Device: Blintz
                            • Mobile App Tracking Parameters:
                               ◦ Device Tracker: device_tracker.blintz
                               ◦ Action Trigger.: blintz_last_update_trigger
                               ◦ Battery Sensor: blintz_battery_level
                               ◦ Notifications...: WAITING FOR NOTIFY SVC TO START"
                            • Other Parameters:
                               ◦ inZone Interval: 1 hr
                               ◦ Fixed Interval: 30 mins

But the "WAITING" message is always there, whenever i restart the iCloud3 integration, either alone or with full HA restart. True even if HA has been running for days, with the Mobile App Notify service working (which I use for many other automations), i.e. it's not a startup timing race with the Notify service.

smarthomefamilyverrips commented 1 month ago

@smarthomefamilyverrips Maybe i can answer the Why question if i see some Log files that show what is really going on with your system and see how it is set up and running.

  1. Set iCloud3 Log Level to Debug in the Configure Settings > Page 2 > Format Parameters screen
  2. Delete the Home Assistant log file so I’m not looking at a lot of old entries (config/home-assistant.log)
  3. Restart HA
  4. Wait for 10-minutes or so until it does the locate and the Mobile App state goes to None or Not Used or when it stops tracking the Mobile App devices
  5. Zip the home-assistant.log and icloud3-0.log files and email them to geekstergary@gmail.com

@gcobb321 will do this ASAP thank you!

smarthomefamilyverrips commented 1 month ago

@smarthomefamilyverrips Maybe i can answer the Why question if i see some Log files that show what is really going on with your system and see how it is set up and running.

  1. Set iCloud3 Log Level to Debug in the Configure Settings > Page 2 > Format Parameters screen
  2. Delete the Home Assistant log file so I’m not looking at a lot of old entries (config/home-assistant.log)
  3. Restart HA
  4. Wait for 10-minutes or so until it does the locate and the Mobile App state goes to None or Not Used or when it stops tracking the Mobile App devices
  5. Zip the home-assistant.log and icloud3-0.log files and email them to geekstergary@gmail.com

@gcobb321 what debug level should I choose as there are several options?

gcobb321 commented 1 month ago

Debug auto reset

smarthomefamilyverrips commented 1 month ago

@gcobb321 I did send you the log files on your email.

gcobb321 commented 1 month ago

Hi Marco, One think I noticed in your files that may be leading to your issue is the MobApp data is not restored when iCloud3 restarts. There is also a bug I found a few days ago where the sensor state values being displayed on the are restored but they are not loaded into the internal Device data fields when the Device is being setup when iCloud3 starts or restarts. I don't think you are affected by it but you might be.

Attached is v3.0.5.10 that addresses these issues and a few others (see below). Unzip the file into the icloud3 directory and restart HA.

icloud3 v3.0.5.10.zip

The MobApp items are not restored the first time you start because they hadn't been saved yet. Restarts after that should display them - you will not see the MobApp State=not_set and then see it's value on a second locate. It will immediately display the last MobApp value on the initial locate.

Change Log - v3.0.5.10 (10/6/2024)

  1. STARTING/RESTARTING ICLOUD3:
    • Device Sensors (Fixed) - They were being reinitialized when Restarting iCloud3 when they shouldn't have been.
    • Mobile App entities (Fixed) - They are no longer being reinitialized when iCloud3 is restarted on a configuration change or Event Log restart request. This includes the device_tracker, battery, trigger and notify entities.
    • Mobile App state values (Improved) - The last Mobile App state values used internally by iCloud3 are now being saved and are then restored on a restart.
  2. DEVICE TRACKING (Change) - When the device was close to Home (<2km), the interval was 15-seconds. If you made a slight turn or a jog or the road curved so the distance was slightly greater than the last update, the direction would change to AwayFrom. This caused the interval calculator to be set to 1-min when it should really retain the 15-sec setting. The AwayFrom interval adjustment was removed from the close to Home interval calculator routine.

Let me know how it goes. I have not updated GitHub and HACS

yet.

Gary

smarthomefamilyverrips commented 1 month ago

Hi Marco, One think I noticed in your files that may be leading to your issue is the MobApp data is not restored when iCloud3 restarts. There is also a bug I found a few days ago where the sensor state values being displayed on the are restored but they are not loaded into the internal Device data fields when the Device is being setup when iCloud3 starts or restarts. I don't think you are affected by it but you might be.

Attached is v3.0.5.10 that addresses these issues and a few others (see below). Unzip the file into the icloud3 directory and restart HA.

icloud3 v3.0.5.10.zip

The MobApp items are not restored the first time you start because they hadn't been saved yet. Restarts after that should display them - you will not see the MobApp State=not_set and then see it's value on a second locate. It will immediately display the last MobApp value on the initial locate.

Change Log - v3.0.5.10 (10/6/2024)

  1. STARTING/RESTARTING ICLOUD3:

    • Device Sensors (Fixed) - They were being reinitialized when Restarting iCloud3 when they shouldn't have been.
    • Mobile App entities (Fixed) - They are no longer being reinitialized when iCloud3 is restarted on a configuration change or Event Log restart request. This includes the device_tracker, battery, trigger and notify entities.
    • Mobile App state values (Improved) - The last Mobile App state values used internally by iCloud3 are now being saved and are then restored on a restart.
  2. DEVICE TRACKING (Change) - When the device was close to Home (<2km), the interval was 15-seconds. If you made a slight turn or a jog or the road curved so the distance was slightly greater than the last update, the direction would change to AwayFrom. This caused the interval calculator to be set to 1-min when it should really retain the 15-sec setting. The AwayFrom interval adjustment was removed from the close to Home interval calculator routine.

Let me know how it goes. I have not updated GitHub and HACS

yet.

Gary

@gcobb321 thanks for your effort, I did install your zip file and will look for a few days how is going, for the moment it does keep showing the mobile app location in the events log and I also not have the error message in configuration screen anymore that Mobile App is not available, so "so far so good" ..... I only got the below error message in my HA log after first restart after I installed this new version (but maybe this is because it is first restart?)

This error originated from a custom integration.

Logger: custom_components.icloud3 Source: custom_components/icloud3/device.py:812 integration: iCloud3 v3: iDevice Tracker (documentation, issues) First occurred: 21:20:02 (1 occurrences) Last logged: 21:20:02

'mobapp' Traceback (most recent call last): File "/config/custom_components/icloud3/device.py", line 812, in _restore_mobapp_items_from_restore_state_file mobapp_items = Gb.restore_state_devices[self.devicename]['mobapp']


KeyError: 'mobapp'
smarthomefamilyverrips commented 1 month ago

@gcobb321 today after a restart of HA again i got the "not used" message for mobile app in the event logs and in same time also showed me the error message in config screen that mobile app was not available, after I noticed I did do a other restart of HA but that not did fix the issue and still had same behavior, after that I did us the "restart icloud3" action from drop down menu in event logs and that made everything come alive again and fixed the issue of having the "not used" message in logs and the "mobile app not available error" in configuration screen, so it seems that (sometimes) during a restart of HA the issue still appears unfortunately.

smarthomefamilyverrips commented 1 month ago

@gcobb321 I did get email notification with your message but I see you did delete it from here, I assume you not want me to try your zip file anymore?

I did notice that iCloud3 not fails using mobile app on every HA restart but it does most of the time (maybe 7 or 8 out of 10 times) a reload of iCloud3 in the drop-down menu of event logs fixes the issue every time in the event the iCloud3 integration stops using mobile app location after a HA restart

smarthomefamilyverrips commented 1 month ago

@gcobb321 I now did get some other weird error, the Mobile App did not update location and kept showing " home " at the moble app device tracker. i did restart iPhone, update mobile app location manual but not did change, I did reload the mobile app integration all with no succes, the icloud device tracker did update to correct location but the mobile app tracker kept saying home, then I did try full restart of HA and still the mobile app stayed on home while the geocode location in the mob app integration did show the correct address now. In the event logs of iCloud3 it showed home zone for as well mobile app as iCloud3, after 5 minutes refresh interval as usual after HA restart it did stop using the mobile app location, directly after this happened it also did update the mobile app tracker from home to the correct location.... so it seems like somehow when the iCloud3 integration still used the mobile app it did block the mobile app updating to correct location and looked like it forced the mobile app to stay on home location???

Hope I explained it clearly.

Screenshot_20241010_142354_Home Assistant

Screenshot_20241010_142304_Home Assistant

gcobb321 commented 1 month ago

Sorry for the delay. A hurricane just went through the area (Florida, US). I reviewed everything that was being done when checking the Mobile App Integration and made some major changes. The Mobile App Integration will now only be used for assigning the Mobile App friendly name. It will not be used for determining if a device should be monitored or not monitored. If it is loaded before iCloud3 starts, the Mobile App's Integration' friendly name will be available and used. If not, it will be checked again when HA finishes starting and a temporary name will be displayed with a question mark, like Gary Iphone(?) in Stage 4 of the Event Log.

Unzip the file below into the icloud3 directory and restart ha. icloud3 v3.0.5.10.zip

smarthomefamilyverrips commented 1 month ago

@gcobb321 no problem, hope you are well and the hurricane did spare you. I will try the zip file first thing tomorrow as I am located in Romania and it now is 23:13 (11:13 AM) here.... take care and be safe!

smarthomefamilyverrips commented 1 month ago

@gcobb321 I did install you ZIP file and after restart of HA i got following error message

This error originated from a custom integration.

Logger: custom_components.icloud3 Source: custom_components/icloud3/support/start_ic3.py:614 integration: iCloud3 v3: iDevice Tracker (documentation, issues) First occurred: 11:25:46 (2 occurrences) Last logged: 11:26:09

'NoneType' object is not subscriptable Traceback (most recent call last): File "/config/custom_components/icloud3/support/start_ic3.py", line 614, in check_mobile_app_integration {k[:10]: v for k, v in Gb.MobileApp_fnames_x_mobapp_id.items()} ~^^^^^ TypeError: 'NoneType' object is not subscriptable

for rest all seems to run normal, now the event log did not got to status "None" for mobile app after first 5 minutes update interval, later today will go away from home and will see how it is then. I will keep you up to date

smarthomefamilyverrips commented 1 month ago

@gcobb321 today at restart of HA i got again error message

This error originated from a custom integration.

Logger: custom_components.icloud3 Source: custom_components/icloud3/support/start_ic3.py:614 integration: iCloud3 v3: iDevice Tracker (documentation, issues) First occurred: 13:38:35 (2 occurrences) Last logged: 13:39:02

'NoneType' object is not subscriptable Traceback (most recent call last): File "/config/custom_components/icloud3/support/start_ic3.py", line 614, in check_mobile_app_integration {k[:10]: v for k, v in Gb.MobileApp_fnames_x_mobapp_id.items()} ~^^^^^ TypeError: 'NoneType' object is not subscriptable

gcobb321 commented 1 month ago

That error is in the routine that dumps the values of some tables to the log file for debugging. You can ignore it

smarthomefamilyverrips commented 1 month ago

@gcobb321 today I again got the error of the person entity using the iCloud3 tracker jumping location for a short period when we came home, again did go home/away/home (see screenshot),

I also added event log where you can see that the mobile app location already was home while the icloud location still stayed away for a other 30 seconds, at the same time as the 2nd 15 seconds update, the person entity jumped from home to away and when the next 15 seconds update occured and also the icloud location changed to home location in the event log and exactly the same time then also the location of the person entity changed back to home again.

Opera Snapshot_2024-10-18_181353_homeassistant local

icloud3-event-log_2024.1018-18.15.log

smarthomefamilyverrips commented 1 month ago

@gcobb321 I got a lot of the below error messages in my logs today

This error originated from a custom integration.

Logger: custom_components.icloud3 Source: custom_components/icloud3/support/pyicloud_ic3.py:172 integration: iCloud3 v3: iDevice Tracker (documentation, issues) First occurred: 11:11:40 (30 occurrences) Last logged: 21:07:40

('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 716, in urlopen httplib_response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 468, in _make_request six.raise_from(e, None) File "", line 3, in raise_from File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 463, in _make_request httplib_response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 802, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 552, in increment raise six.reraise(type(error), error, _stacktrace) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/packages/six.py", line 769, in reraise raise value.with_traceback(tb) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 716, in urlopen httplib_response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 468, in _make_request six.raise_from(e, None) File "", line 3, in raise_from File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 463, in _make_request httplib_response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 1428, in getresponse response.begin() File "/usr/local/lib/python3.12/http/client.py", line 331, in begin version, status, reason = self._read_status() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/http/client.py", line 300, in _read_status raise RemoteDisconnected("Remote end closed connection without" urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/config/custom_components/icloud3/support/pyicloud_ic3.py", line 172, in request response = Session.request(self, method, url, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 682, in send raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

gcobb321 commented 2 weeks ago

Closed with the v3.1.1 Release. Create a new issue if you continue to experience this problem.