synchronous calls hang while subscribed

[MadMellow]

Please answer these questions before submitting your issue. Thanks!

Not sure if this is an issue, feature request, or just the way it works.

I'm attempting to use this code as a "helper" program for a HomeSeer plugin -- exposing the mode, siren, camera and motion to HomeSeer. My experience with Python is somewhat limited, but I do OK.

Everything works as expected (although I'm still baffled by RTSPS), except that I cannot use many of the synchronous calls while subscribed to the event stream. I assume this is because the event stream is already established, yet NotifyAndGetResponse attempts to reestablish the event stream. My guess is that this confuses the link.

Any chance of checking the state of the event stream before Subscribing?

What version of Python are you using (python -V)? 3.6 and 3.7

What operating system and processor architecture are you using (python -c 'import platform; print(platform.uname());')? both windows 10 and macOS Catalina

Which Python packages do you have installed (run the pip freeze or pip3 freeze command and paste output)?

Paste your ouptut here

arlo==1.2.40
certifi==2020.12.5
chardet==4.0.0
idna==2.10
monotonic==1.5
mutagen==1.45.1
PySocks==1.7.1
requests==2.25.1
six==1.15.0
sseclient==0.0.22
urllib3==1.26.3

Which version of ffmpeg are you using (ffmpeg -version)?

Paste your output here Irrelevent

Which Arlo hardware do you have (camera types - [Arlo, Pro, Q, etc.], basestation model, etc.)?

Old Arlo VMB4000r3

What did you do?

IT Professional until retirement. Home automation hobbiest for 10 years.

If possible, provide the steps you took to reproduce the issue. A complete runnable program is good. (don't include your user/password or any sensitive info)

from arlo import Arlo
from subprocess import call
import sys
import time
import json
import os
import threading
from sys import stdin
from datetime import datetime

def log(type,reason,message):
    if logging == True and logfilename != "":
        with open(logfilename, 'a') as logfile:
            logfile.write(datetime.now().strftime('%m/%d/%y %H:%M:%S') + ' ' + str(type) + ' ' + str(reason) + ' '  + str(message) + '\n')

def repeve(reason,message):
    jrep = {}
    jrep['type'] = 'event'
    jrep['event'] = reason
    jrep['id'] = id
    jrep['description'] = message
    print(json.dumps(jrep) + '\n', file = sys.stdout) 
    sys.stdout.flush()
    log('event',reason,message)

def reply(reason,message):
    jrep={}
    jrep['type'] = 'reply'
    jrep['command'] = reason
    jrep['id'] = id
    jrep['base'] = base
    jrep['camera'] = cam
    jrep['response'] = message
    print(json.dumps(jrep) + '\n', file = sys.stdout) 
    sys.stdout.flush()
    log('reply',reason,message)

def checkEventQueue():
    subscribed = True
    while subscribed == True:
        if arlo.event_stream.queue.empty() == False:
            event = arlo.event_stream.queue.get()
            repeve('event',str(event))
    else:
            time.sleep(.5)

if len(sys.argv) > 1: 
    logfilename=str(sys.argv[1])
    logging = True
else:
    logfilename = ''
    logging = False

log('start','start','starting')

connected = False

while True:

    try:

        base = 0
        cam = 0
        id = 0
        command = ''
        option = ''
        logging = False

        # Get command line in form: command[=value] [id] [base] [camera] [Logging]

        line = stdin.readline().rstrip()

        if line == '':
            repeve('exit','partner exit')
            sys.exit(0)

        parts = line.split(' ')

        pieces = parts[0].split("=")
        command = pieces[0].lower()
        if len(pieces) > 1:option = pieces[1]

        if len(parts) > 1:id = int(parts[1])
        if len(parts) > 2:base = int(parts[2])
        if len(parts) > 3:cam = int(parts[3])
        if len(parts) > 4:logging = bool(parts[4])

        log('received','command',line)

        if command == 'user':
            USER = option
            reply(command,'ok')
        elif command == 'pass':
            PASS = option
            reply(command,'ok')
        elif command == 'connect':
            if USER == '' or PASS == '':
                reply(command,'No Credentials')
            else:
                try:
                    arlo = Arlo(USER,PASS)
                    connected = True
                    bases = arlo.GetDevices('basestations')
                    cams = arlo.GetDevices('camera')
                    sirens = arlo.GetDevices('siren')
                    reply(command,'ok')
                except Exception as e:  
                    connected = False
                    reply(command,'Error ' + str(e))

        elif command == 'logout':
            stopsubscribe()
            arlo.Logout()
            connected = False
            subscribe = False
            reply(command,'ok')

        elif command == 'exit':
            reply(command,'exiting')
            sys.exit(1)

        elif connected == True:

            if command == 'ping':
                reply(command,arlo.Ping(bases[base]))
            elif command == 'record':
                reply(command,arlo.StartRecording(bases[base],cams[cam]))
            elif command == 'stop':
                reply(command,str(cam)+str(arlo.StopRecording(cams[cam])))
            elif command == "snap":
                reply(command,arlo.TriggerFullFrameSnapshot(bases[base],cams[cam]))
            elif command == 'alarm':
                reply(command,arlo.SirenOn(bases[base]))
            elif command == 'sensor':
                reply(command,arlo.GetSensorConfig(bases[base]))
            elif command == 'devices':
                reply(command,arlo.GetDevices())
            elif command == 'silent':
                reply(command,arlo.SirenOff(bases[base]))
            elif command == 'arm':
                reply(command,arlo.Arm(bases[base]))
            elif command == 'disarm':
                reply(command,arlo.Disarm(bases[base]))
            elif command == 'stay':
                reply(command,arlo.CustomMode(bases[base],'mode2'))
            elif command == 'camstate':
                reply(command,arlo.GetCameraState(bases[base]))
            elif command == 'basestate':
                reply(command,arlo.GetBaseStationState(bases[base]))
            elif command == 'subscribe':
                arlo.Subscribe(bases[base])
                subThread = threading.Thread(target=checkEventQueue)
                subThread.daemon = True
                subThread.start()
                reply(command,str(subThread.is_alive()))
            elif command == 'unsubscribe':
                subscribed = False
                subThread.join()
                reply(command,'ok')
            elif command == 'modes':
                reply(command,arlo.GetModesV2())
            elif command == "base":
                reply(command,bases)
            elif command == "camera":
                reply(command,cams)
            elif command == 'siren':
                reply(command,sirens)
            elif command == 'alarming':
                reply(command,arlo.NotifyAndGetResponse(bases[base], {"action":"get","resource":"siren","publishResponse":True}))
            elif command == 'sensors':
                reply(command,arlo.GetSensorConfig(bases[base]))
            elif command == 'camdevice':
                reply(command,arlo.GetDevice(cams[cam]['deviceName']))
            else:
                reply(command,'Unknown Command')
        else:
            reply(command,'Not Connected')

    except KeyboardInterrupt:
        arlo.Logout()
        repeve('exit','keyboard interrupt')
        sys.exit(0)
    except SystemExit:
        arlo.Logout()
        repeve('exit','sys exit')
        sys.exit(0)
    except Exception as e:   
        repeve('error',str(e))

Paste your output here Synchronous calls (such as getCameraState) hang forever.

What did you expect to see?

Paste your ouptut here

What did you see instead?

Paste your output here

Does this issue reproduce with the latest release?

[MadMellow]

On further review, it appears that there is not a subsequent call to establish the event stream. So, I have not idea what's going on.

[jeffreydwalter]

Your script is a little too convoluted for me. Have you tried the example scripts in the Wiki? Can you craft a simple script based on one of the example scripts that reproduces your issue?

[jeffreydwalter]

You should NOT be reading from the underlying event queue of my Arlo library.

def checkEventQueue():
    subscribed = True
    while subscribed == True:
        if arlo.event_stream.queue.empty() == False:
            event = arlo.event_stream.queue.get()
            repeve('event',str(event))
    else:
            time.sleep(.5)

You are basically "stealing" the events from the queue, which is why all the library calls are hanging... The Arlo API is async by nature (since it uses EventStream). I wrapped it up in a synchronous API to allow people to interact with it in a more familiar way. When you call one of the methods in my library, a message is POST'd to the /notify endpoint. That message includes an id which is included in the response message, which comes back via the EventStream. My library reads from the EventStream and queues all messages it receives. At the same time, in another thread, my code polls the queue looking for the response message with the same id. When it finds that message, it returns it to the caller of the synchronous method.

Hope that helps...

[jeffreydwalter]

If you want to "subscribe" to some series of events, take a look at: https://github.com/jeffreydwalter/arlo/blob/master/examples/arlo-motiondetect.py

[MadMellow]

Thanks Jeff,

Yes, I am stealing the replies. It took me some digging to figure that out. I do want to see the replies, so I will disable "stealing" while processing other commands.

I had looked at motion detect as inspiration/example.

Thanks for your help.

[MadMellow]

Jeff,

I have stopped stealing replies, but I think I've found the root of my problems. In my testing I'm making calls which use NotifyAndGetResponse (NAGR) fairly often. If/when my calls overlap with the Heartbeat Ping it can set up a reentrance condition in NAGR.

I've added debug print statements and have seen NAGR get stuck spinning with the same tid -- which presents as a reentrance problem. I'm still learning Python but I'll see if I can get RLock to help.

[jeffreydwalter]

You don't need to ping. The library does that for you already.

On Sat, Mar 6, 2021, 7:52 AM MadMellow notifications@github.com wrote:

Jeff,

I have stopped stealing replies, but I think I've found the root of my problems. In my testing I'm making calls which use NotifyAndGetResponse (NAGR) fairly often. If/when my calls overlap with the Heartbeat Ping it can set up a reentrance condition in NAGR.

I've added debug print statements and have seen NAGR get stuck spinning with the same tid -- which presents as a reentrance problem. I'm still learning Python but I'll see if I can get RLock to help.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jeffreydwalter/arlo/issues/165#issuecomment-791949139, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEKOBYNELSRCSCCSKQWGOLTCIXR7ANCNFSM4YRYOBHQ .

[MadMellow]

I realize that it pings all by itself, but it doesn't query the state of the camera(s), mode, or siren. I realize that events "should" pick that up, but only if all is stable.

[jeffreydwalter]

You don't query for state change. Those events are pushed to the eventstream as they occur. If you want to trigger actions when those occur, then you need to follow the example in the script I referenced in my previous comment.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.