gcobb321 / icloud3_v3

iCloud3 v3, Development Version - This Repo/Version is used for development and testing of new and updated features. It is not the official release of iCloud3 v3, .
MIT License
169 stars 13 forks source link

Stationary Zone Automation Trigger #188

Closed scotty1395 closed 9 months ago

scotty1395 commented 1 year ago

Hi Gary,

Running v3.0-pr1a at the moment.

Stationary zones seem to be having the same issue that normal zones were having pre beta 14, not triggering enter/exit events for automations.

From memory it was fixed by storing the lat/long figures as numeric rather than strings. Are stationary zones still storing the coordinates as strings?

This is where normal zones got fixed: https://github.com/gcobb321/icloud3_v3/issues/65#issuecomment-1454177416

gcobb321 commented 1 year ago

A Stationary Zone it’s handled internally as all other zones. The difference is they are created when needed by iCloud3 and can change location. The iOS app picks up the start zone after it is created and handles all enter/exit notifications.

The exit triggers seem to be ok for me. When I create a start zone, and then leave, the exit trigger is processed as normal. I did find some issues recently where, if the location was old and the interval was less than old location threshold, the old location was used instead of requesting a new one. That is, if the interval was 15-secs and the data was 1-min old, it would be less than a3-min threshold so that location was used instead of getting a new one. If you were exiting a zone, the inzone location might still be used so the iOS app would still think you are in the zone.

I wonder if that it’s what you are seeing. This was just recently fixed. I would download the latest version (pr1.3) and see what h happens.

The next version (pr1.4) has not been released yet but it’s on an issue by Snuffy2. In it, the Stationary Zone is removed from HA when there are no devices in it instead of moving it back to its’base’ location.

scotty1395 commented 1 year ago

IC3 exit triggers seem to be working fine for IC3 purposes. It is only HA automations triggered by a zone enter/exit for a person entity or device_tracker entity on the Stationary Zones that don't fire.

alias: "Test zone exit - device tracker"
trigger:
  - platform: zone
    entity_id: device_tracker.brads_iphone
    zone: zone.ic3_stationary_1
    event: leave
    enabled: true
condition: []
action:
  - service: notify.mobile_app_brads_iphone_2
    data:
      message: Device Tracker Zone Exit Test
      title: Zone Exit Test
mode: single

I'm wondering whether it is related to IC3 changing the stationary zone from passive: false to passive: true and vice versa. I've noticed when someone is in the stationary zone the state count is zero even though IC3 has changed the zone to passive: false. Perhaps the passive change happens in the same process as moving the zone and HA misses the change? There seems to be quite a bit of discussion around about HA not correctly handling zone enter/exit automation triggers when zones are passive.

In pr1.4 will the stationary zones that are created and removed always be passive: false since they don't need to hide when not in use?

gcobb321 commented 1 year ago

They will be passive: false. But I do not think that your test Automation will ever work in any case.

HA sets the person count in the zone component when the gps coordinates for the device are inside the zone radius for the gps coordinates of the zone. This happens after the device gps values is updated. The zone has to be set up for that to happen. The stat zone may or may not have been set before the devices is updated depending on the sequence HA is doing things which I have no control over. Also, you can not be sure of the StatZone number that will be assigned. There can be several devices in different StatZones at the same time.

I know the above works with normal zones since the zones are initialized when HA starts or the zone is created. Stat zones are added and removed when needed.

I would base not base an automation trigger on HA issuing an update like you have but would would watch the ic3 zone sensor changing values to or from one the starts with ic3_stationary.

scotty1395 commented 1 year ago

That's unfortunate. It worked fine in IC3 v2.

gcobb321 commented 1 year ago

I assume you are not having that issue with normal zones using that type of trigger. And it’s only with stationary zones. I’ll take a look at it and see if I see anything.

I can see keeping track of people in a normal zone but how are you using the number of people in a StatZone or getting notified when you enter or leave one?

scotty1395 commented 1 year ago

Thanks Gary. Yes, normal zones are working fine with that type of trigger.

I'm not using the number of people in zones at all. That was just an observation about the stationary zone, hoping it may point to a potential cause of exit triggers not firing. I'm only using zone exit triggers for stationary zones along the lines of the example automation I posted.

gcobb321 commented 1 year ago

Have you looked at the following When they don’t fire:

gcobb321 commented 1 year ago

I’m away right now on a trip and reviewed my events and found some StatZone that did not process an exit trigger. There were messages in th Event Log that said the “exit trigger was canceled because the detected distance was less than the radius so the StatZone was being moved instead”. The iOSapp now showed Away while iCloud3 showed StatZon2. Zones exit/enter triggers were then being discarded because there were older than the last update. (I showed the detail with Event Log > Actions > Shores Tracking Monitors. )

I’ve commented out this relocate Test and will see what happens.

Are you seeing this message when your StatZone exits are missed?

scotty1395 commented 1 year ago

I haven't noticed them, but I haven't been looking for them. Will have a look when I next test.

I haven't really been concerned with the IC3 exit triggers triggering IC3 updates, only the HA zone exit triggers > if the device_tracker entity exits the zone the HA zone exit trigger should fire. If IC3 keeps the device_tracker entity in the zone I wouldn't expect the HA zone exit trigger to fire. Problem is it is not firing when the device_tracker entity has actually exited the stationary zone. HA wouldn't care about the age of the result or anything IC3 is doing. HA would only care about the transition from in zone to out of zone as it moves in HA.

Is the stationary zone being modified/shrunk as soon as the IC3 notices the exit? If it's being done at the same time as updating the device_tracker entity perhaps HA doesn't see it as a zone exit. Should there be a delay in updating the stationary zone once it's been exited? In pr1.4 is there a delay before the zone is destroyed?

gcobb321 commented 1 year ago

The prices is:

  1. Iosapp issues an exit trigger
  2. iCloud3 see it and counts the devices in it. If the count is zero, it does a hass remove entity call with the zone entity name (zone.ic3stationary#) and stress the set StatZone’s interval table entry to passive =true and entity_exists_in _ha to false so it can be reused the next time it needs to create one.
  3. Determines the zone the device is in which will be away
  4. Updates the device device_tracker location
  5. Hass , a at some point in time, updates the iosapp with the zone info. I don’t know the process it uses or how long after the remove that the delete is done but think the iosapp created an entity Update Watch Event which is triggered on zone entity changes.

My current thinking is:

  1. If the iosapp takes the device out of the zone but iCloud3 does not, that messes thinks up. That’s my reason for the ‘exit trigger canceled and iCloud3 will relocate the zoneI’ question. I had seen those messages in my logs.
  2. If hass sends the zone info update to the iosapp on all devices running the iosapp but the iosapp does not receive it (poor cell area, asleep, etc), that creates a problem where that device thinks the zone still exists when it doesn’t or doesn’t exist when it does. If the latter, then no exit triggers will be issued.
  3. if iCloud3 issues a remove call but it’s in a poor cell area or something happens and it doesn’t happen, iosapp on the devices think it exists when iCloud3 thinks it doesn’t. Then iosapp would issue a zone enter triggers for an unknown zone, iCloud3 would discard it but do an update. That might recreate the StatZone with a different name which might create a problem but I think that would be self healing.

I’m currently Away from home for the next 2.5 weeks (live in Florida, in Europe) so I’m able to do a bunch of testing. I can make code updates and restart HA back home from anywhere.

I’m going to change some tracking monitor messages to regular events and display the StatZone device counts to see if anything strange shows up. Also thinking about issuing hass remove zone calls every 10-mins or so on StatZones that have been created even when it does not exist to make sure.

scotty1395 commented 1 year ago

I've just upgraded to pr1.4 so I'll see how that goes.

scotty1395 commented 1 year ago

This is still an issue with pr1.4.

scotty1395 commented 1 year ago

This is still an issue with pr1.5.

gcobb321 commented 1 year ago

I took another look at the original question/problem and it was dealing with enter/exit triggers not firing for stationary zones in HA automations.

Is that still the problem you are having o or is it something else? If it is, StatZone are now removed from HA when all devices have left them where they were changed to a passive state and continued to exist before.

I was away all of last week and StatZone were used all the time. The only issue I encountered was the iOS app would still see my phone in a StatZone after it had left the zone and the zone had been removed from HA. I made a change to not report this on the Event Log for the iosapp zone value since the zone has been removed and HA had not sent zone update info to all of the devices yet.

If you are still having some kind of problem, because of the changes I made in handling StatZone exits and removal, I will need another explanation to understand what the issue still is and what you are trying to do.

scotty1395 commented 1 year ago

The behaviour hasn’t changed with the StatZone removal change.

I have two automations set up to test.

Automation one triggers on state change from stationary of the person entity linked to the IC3 device tracker. This always fires when leaving a stationary zone.

Automation two triggers on HA zone exit event from stationary of the same person entity linked to the IC3 device tracker. This never fires when leaving a stationary zone. This is the problem.

With pr1.5 I’m not seeing the StatZone being removed once exited, by the way. I was in a StatZone earlier today and it is still in the same location since I left it. Hasn’t been removed or reset to its default location. The only change I have made to the stationary setup from default configuration is changed the Friendly Name Base to Stationary without a wildcard character.

gcobb321 commented 1 year ago

When the last device leaves the StatZone and iCloud3 removes it, there will be an entry in the Event Log that says the “StatZone was exited and removed” for that device. If you go to the Devices & services > Areas $ zones > Zones screen, the StatZone is still listed. The screen needs to be refreshed. Pull down on it to refresh the screen and it is no longer displayed.

After it is removed, it also no longer it’s displayed on the Developer Tools > States screen.

There is no default location anymore. All that code has been removed.

Are these tests automations or are they used for a real task? What happens if you create a zone, move into it and then delete it before moving out of it with the Automation 2 test on that zone you. Does it fire then?

scotty1395 commented 1 year ago

Ahh, got it. The pull down refresh removed the display of the zone as you said.

The automations are for real tasks. I’ve just duplicated the task with the different triggers so I can see which one is actually firing.

I will see if I can test manually creating a zone etc. I don’t think this is related to the zone being deleted though, as it was an issue before they were being removed. Unless it was also a problem when their location was being reset instantly on exit. Perhaps a delay of a few seconds before deleting the zone may make a difference?

gcobb321 commented 1 year ago

Your automation tests_ zone.ic3_stationary1 . iC3 can create more than 1 stationary zone at the same time with _1, _2, _3, etc as an identifier. You would end up with multiple automations or triggers that may have to test all of them. I'm not sure how HA would handle a zone that does not exist (ignore or error).

Are these stationary zones in the same location all the time? Are you trying to keep track of the people in a statzone and know when they leave it? Trying to get a better understanding of what you are actually trying to do.

HA is the one that keeps track of the people in the zone based on the lat/long in the device_tracker entities and if it falls within the zone radius (it does not look at accuracy so that might/would lead to false exit errors). The zone's state is the number of devices in it. So if the zone is deleted, there is no place to store the people count.... And I do not think there is any way to for HA to trigger a 'zone deleted' action.

Delaying deleting the StatZone until the iOS App processed an exit trigger would probably create a lot of problems for iCloud3. And may cause more conflicts between the iOS App and iC3 zone/state values.

scotty1395 commented 1 year ago

I only track two devices so only have to deal with two stationary zones. It is an added complication compared to IC3 v2 where you only needed to worry about the person’s own stat zone, but not impossible (with only two devices).

gcobb321 commented 1 year ago

I am curious as to what you are really trying to do by monitoring when someone enters a StatZone. Especially when it will move from one place to another. Would a regular zone be easier and more practical?

scotty1395 commented 1 year ago

I’m looking to trigger automations when devices are on the move again when exiting stationary.

I just did a test with device 1 staying still to create a StatZone. If device 2 exits device 1’s StatZone the HA zone exit trigger fires fine as the StatZone didn’t get destroyed because it was still needed by device 1.

A delay on destroying the zone may help. It’s not the iOS zone exit event HA is listening for. HA zone exit events work with other platforms that only provide coordinates. HA is only waiting for the device coordinates to move from inside a zone to outside a zone.

gcobb321 commented 1 year ago

I moved the StatZone deletion from before the individual device_tracker entity was updated to after all the devices and sensors were update in the current 5-second update cycle. This updates the device attributes while the zone still exists so HA will fire the device_tracker update listener associated with the zone before the zone is deleted. Your automations, along with the HA zone update blueprint issuing the mobile_app notification to send the message.

I checked the home-assistant.log file and it's full of warning messages (_2023-10-07 16:45:39.510 WARNING (MainThread) [homeassistant.components.zone.trigger] Automation 'Test stat zone 1 exit - device tracker Lillian' is referencing non-existing zone 'zone.ic3_stationary1' in a zone trigger} when the stat zone doesn't exist. I assume you have seen these and have a way to suppress them.

Unzip rc6 into the iCloud directory and restart HA. Let me know if it is OK on your end and I'll publish this as a new release

Change Log:

  1. Bug Fix - Fixed the error causing the "AttributeError: 'iCloud3_Device' object has no attribute 'interval_secs' message at line 680, in determine_interval_after_error" error.
  2. China Users - Added an option to the Configure Settings - iCloud Account & iOS App Data Sources screen to enable the '.cn. Apple Web Server URL address suffix.
  3. Monitored Devices - The iOSApp state value is no longer displayed on the Event Log if it is different than the iCloud3 zone and it is earlier the last FamShr update time since it is old and is probably incorrect.
  4. iCloud3 Version - The version that is currently running is now displayed in the Configure Settings menu screen and more clearly on Event Log messages during startup.
  5. Battery - HA has changed the battery 'state class' internal value from 'battery' to 'measurement'.
  6. Prerelease Version number - This has changed to Release Candidate number to better conform to HA standards. Version pr1.5 was a developer test version and not released.
  7. Configure Setting - Made some minor menu text changes for consistency.

icloud3_v3.0.rc6.zip

scotty1395 commented 1 year ago

Thanks Gary. Will test today and let you know.

scotty1395 commented 1 year ago

Unfortunately testing results have been hit and miss with only one of five StatZone exits triggering the test automation. I haven't been able to identify a pattern for success or failure.

gcobb321 commented 1 year ago

I have 4 automations set up for 2 devices (gary_iphone & lillian_iphone) and 2 StatZones (ic3_stationary_1 & _2). With 2 devices, you will not know which one will be picked up as leaving the zone last since it is triggered by the iOS app. Also, I didn’t know which zone would be created (_1 or _2). You can probably create some HA Automation variables for the zone and device but that would get too complicated for testing.

The notifications would fire correctly for both devices l leaving and a single device leaving.

I’ll run the tests again over the next few days and see if I see anything. Try setting up an automation for entering the StatZone to see the order of the notifications.

I have posted rc6 since it has a bunch of other fixes.

One more thing - Are you getting the error messages in the HA log file about the StatZone used in an automation does note exist after it is deleted?

scotty1395 commented 1 year ago

I've got the same configured - 2 devices, 4 automations, 1 for each device and zones _1 and _2. For the recent tests I was only testing a single device in the StatZone, not two in close proximity in the same StatZone.

Will test entering StatZones.

Yes, I'm getting errors logged for the StatZones that don't exist for normal automations. Surprisingly, nothing is logged for the Zone Notification Blueprint automations when the StatZones don't exist.

gcobb321 commented 1 year ago

I’ll be testing 2 devices together with Automation today when we go out to lunch. I’ll be testing 1 device with blueprint tomorrow with _1 enter/exit followed by another enter/exit at two locations

gcobb321 commented 1 year ago

I looked at the blueprint Automation for the HA leaving Zone blueprint and to it’s based on the trigger.from_state and trigger.to_state values changing, and does not use the zone.ic3stationary# zone names in the triggers.

gcobb321 commented 1 year ago

I.ve been running around today and everything looks good with the new changes.

There are 2 zip files:

Unzip into the icloud3 directory

statzone_exit.zip statzone_exit_test.zip

scotty1395 commented 1 year ago

Thanks Gary. Will test it out and let you know how it goes.

scotty1395 commented 1 year ago

Hi Gary. This seems to be working well with the zone triggers.

It does seem to be getting stuck removing StatZones with the below error. It just keeps creating new StatZones and leaves the old ones where they are. I got up to zone.ic3_stationary_7 while testing. image Is this what "Stationary Zones - Minor changes to the handling of deleting a stationary zone when all devices have exited from it." from rc7 relates to?

Does rc7 include the StatZone changes from the above zip files?

gcobb321 commented 1 year ago

I just uploaded v3.0.rc7.1 to check the password on startup and a few other things. See the announcement here for more information

scotty1395 commented 1 year ago

Are these changes included in rc7, rc7.1 or rc8? I'm not seeing any zone triggers with rc7.1.

gcobb321 commented 1 year ago

The rc8 code is the latest and what I am running so everything should be there. The stuff I run is the master code set. There were a lot of GitHub issues with the rc7.1 and rc7.1.1 regarding GitHub commits and the wrong versions being downloaded. I ran into that building rc8 where the commit number for rc8 was the same as rc7.1 so if you downloaded rc8 using HACS, you would get rc7.1.

Are you on rc8? Verify by looking at the Event Log startup info or click Action in the Event Log. Version no. Is displayed

scotty1395 commented 1 year ago

I was on rc7.1. Just updated to rc8 through HACS and now I'm showing rc6. Will manually update to rc8 now.

scotty1395 commented 1 year ago

I'm not getting any statzone triggers with rc8.

gcobb321 commented 1 year ago

The code has not changed. HACS probably has not refreshed and that it’s why you got rc6 on the HACS install. Verify the version you are running in the Event Log just to make sure you are running rc8. Maybe do a HA restart and try again. Send me logs if you can. Remember, triggers are generated by the iOS app so verify that is set up by looking at the ios app settings zones and Event triggers.

scotty1395 commented 1 year ago

I'm definitely running rc8. I manually downloaded the zip, extracted and restarted HA. image

My question is, does rc8 include the changes you posted here: https://github.com/gcobb321/icloud3_v3/issues/188#issuecomment-1760420790

When you provided those changes I was getting statzone triggers, although with the errors I mentioned here: https://github.com/gcobb321/icloud3_v3/issues/188#issuecomment-1763527725

Since then, running rc7, rc7.1 or rc8 has not produced statzone triggers, hence my questions whether the changes were included in rc7, rc7.1 or rc8.

gcobb321 commented 1 year ago

I just ran out for some test and all worked like it should.

statzone-text

My Automations for reference:

alias: Test StatZone 1 Exit - Gary
description: ""
trigger:
  - platform: zone
    entity_id: device_tracker.gary_iphone
    zone: zone.ic3_stationary_1
    event: leave
    enabled: true
condition: []
action:
  - service: notify.mobile_app_gary_iphone
    data:
      message: Device Tracker StatZone 1 Exited
      title: StatZone 1 Exit Test (Gary)
mode: single
alias: Test StatZone 1 Enter - Gary
description: ""
trigger:
  - platform: zone
    entity_id: device_tracker.gary_iphone
    zone: zone.ic3_stationary_1
    event: enter
    enabled: true
condition: []
action:
  - service: notify.mobile_app_gary_iphone
    data:
      message: Device Tracker StatZone 1 Entered
      title: StatZone 1 Enter Test (Gary)
mode: single

I did find one problem where restarting iCloud3 with StatZones would generate an error reinitializing the StatZone during Stage 2 of the startup process. The StatZone would not be reinitialized and the rest of the Stage 2 process would not be completed. The previous waze, config, and other parameters would be used and all tracking would work as it should. The only issue appeared to be that any existing statzones would be set to passive and not used. The next statzone created would be #2. StatZone 1 (ic3_stationary_1) would still exist in HA and the iOS App but not be used by iC3. A HA restart got everything working properly.

Attached is the fix, Unzip it into the icloud3/support directory.

start_ic3.zip

scotty1395 commented 1 year ago

Does rc8 include the changes you posted here?: https://github.com/gcobb321/icloud3_v3/issues/188#issuecomment-1760420790 Or do I need to add that file again after upgrading to rc8?

I'm still not getting statzone triggers with rc8 even with the start_ic3.py update.

gcobb321 commented 1 year ago

Rc8 does not include that fix. It fixes a bug where StatZones are not retained correctly when one exists and an iCloud3 restart is done. I w would install it after you do the update just to be sure that it’s not causing your problem.

As you can see from my automations and screen shots, it is working for me. I changed my home zone location by changing the latitude and restarting HA. I went into a StatZone, rode my bike down the street until I exited. I got the exit notification. Then ride home and got the enter notification.

scotty1395 commented 1 year ago

So rc8 is working for you without the statzone_exit fix or with the statzone_exit fix?

gcobb321 commented 1 year ago

The version of rc8 on HACS does not include the _startic3.zip patch I sent you that fixes a problem restarting iCloud3 when a StatZone exists. The problem generates an error message when reinitializing an existing StatZone. You will not see the error if you do not restart iCloud3.

I am running rc8. The StatZone exit fix has been running in all versions since the last change when I delayed deleting the StatZone until after updating the tracking results for all devices. The automations above were triggered when they should be triggered when I exit and enter a zone. Remember, the iOS ago is initially issuing the exit trigger. Check the iOS app to make sure it says you are in a StatZone you want to trigger. Look at the iOS app Debug logs to see if it was issued.

If it does not work for you then you have something else going on. As far as I can tell, everything is working correctly.

scotty1395 commented 1 year ago

Ok. I will test with running rc8 with the two patches manually added. I think that’s what you’re saying.

gcobb321 commented 1 year ago

You got it.

gcobb321 commented 9 months ago

Closing this issue with the release of v3.0