Improve connection management

[MarkRivers]

Connection management needs improvement. There is currently no indication in the GUI if the camera is removed from the network.

[MarkRivers]

I have created branch improve_management to address this issue.

In https://github.com/areaDetector/ADVimba/commit/1a7610f622c066b4564e037c8ff86a61996398f5 I did the following:

• Added a call to pasynManager->autoConnect(0) in the constructor, so that connection management is handled via callbacks in the driver, and the port is no longer autoConnect=Yes.
• Used the Vimba ICameraListObserver class to get callbacks whenever any camera connects or disconnects.
• When the connection callback is for our camera it calls the new ADVimba::connectionCallback() method.
• If the event is a disconnect it calls asynPortDriver::disconnect(), which calls pasynManager->disconnect(). This will flag the port as disconnected, and will disable all writes to the driver. The port is showns as Disconnected in major alarm color in the OPI screen as shown below.
• If the event is a connect it calls asynPortDriver::disconnect(), which calls pasynManager->connect(). This will flag the port as connected, and will re-enable writes to the driver. The port is showns as Connected in no alarm color in the OPI screen.

I have tried unplugging the cable, and after several seconds it gets the callback and disconnects the port. When I plug it back in it reconnects within a few seconds, and I can then operate the camera normally.

I have not tested leaving the camera disconnected for an extended period to see if it will still reconnect OK.

[MarkRivers]

Steve Hunt said in an e-mail:

Yes, thanks, we now get the asyn disconnect :) which is the main issue as we should be able to warn the operators if the camera is not working. At a first look, we are not able to reconnect and get images without ioc reboot - but I need to check more closely the exact behavior, I may be missing something.

I find that if ReadStatus.SCAN (labeled "Poll features" in the OPI screen) is Passive then I can always restart acquisition when the camera reconnects. I sometimes have to toggle Start and Stop a few times to get it to acquire the first time.

This is what I see at the IOC prompt when I unplug the camera. wait about 35 seconds and plug it back in:

2024/05/24 11:52:53.222 ADVimba::connectionCallback calling disconnect()
2024/05/24 11:52:53.352 13VMB1:cam1:PoolPollStats devAsynInt32::reportQueueRequestStatus queueRequest error port VMB1 not connected
2024/05/24 11:53:29.226 ADVimba::connectionCallback calling connect()

If ReadStatus.SCAN is not passive then the IOC gets into a start where I cannot start acquisition when I reconnect the camera.

Steve, can you test that? It should be fine to leave it Passive, since the communications statistics update even when it is Passive. Set it to periodically scan just makes it poll all of the other features, which is probably not needed.

[alceli]

Confirmed ReadStatus.SCAN is passive :)

But I get different behavior - a) When selecting 'disconnect' button - the scan does not stop, even though status marked as 'disconnected' b) When disconnecting the Ethernet cable to the camera - disconnect activates as expected, and scan stops, but does not restart on re-connection, even when stop/start multiple times. Reboot necessary

The problem may be I have the new Vimba Driver, but have an old ADCcore (3.11.0)

[MarkRivers]

a) When selecting 'disconnect' button - the scan does not stop, even though status marked as 'disconnected'

That is expected. When you press Disconnect that marks the ADVimba as disconnected by asynManager. asynManager will no longer queue requests to write to PVs, and the OPI should signal an error if you try to change a PV. However, it does not disconnect ADVimba from the camera, so the camera continues to collect, and even does callbacks to EPICS. We could implement the ADVimba::disconnect() method, and then pressing Disconnnect would call that method, and that could close the connection between ADVimba and the camera.

b) When disconnecting the Ethernet cable to the camera - disconnect activates as expected, and scan stops, but does not restart on re-connection, even when stop/start multiple times. Reboot necessary

That is not what I see, and I don't think it should be related to the version of ADCore. I do notice that you get 2 connectionCallback messages when you disconnect the cable. I only get 1.

When you plug the cable back in do you see a message like this after a few seconds? You should.

2024/05/24 11:53:29.226 ADVimba::connectionCallback calling connect()

I have been testing with an old Prosilica GC1380CH camera. I will try testing with a Manta.

[MarkRivers]

I just tested with a Manta 507C camera, and it also worked fine for me. I did the following 3 times:

• Start camera acquiring
• Disconnect the Ethernet cable from the camera
• Wait 10-30 seconds
• Plug the Ethernet cable back in
• Wait for the connection to be reestablished, as seen in the IOC message and the Connected text in the OPI display
• Press Acquire Stop
• Press Acquire Start
• The camera began acquiring normally

These are the IOC messages:

2024/05/24 16:51:45.888 ADVimba::connectionCallback calling disconnect()
2024/05/24 16:51:46.405 13VMB1:cam1:PoolPollStats devAsynInt32::reportQueueRequestStatus queueRequest error port VMB1 not connected
2024/05/24 16:52:11.890 ADVimba::connectionCallback calling connect()

2024/05/24 16:52:54.418 ADVimba::processFrame error GetReceiveStatus returned 0 frameStatus=-1
2024/05/24 16:53:01.888 ADVimba::connectionCallback calling disconnect()
2024/05/24 16:53:15.890 ADVimba::connectionCallback calling connect()

2024/05/24 16:53:43.888 ADVimba::connectionCallback calling disconnect()
2024/05/24 16:53:59.890 ADVimba::connectionCallback calling connect()

@alceli is the camera you are testing POE, so that when you disconnect it you also power-cycle the camera? My GC1380CH is not POE. My Manta 507C is POE, but I am using an external power supply, so that when I disconnect the Ethernet it does not lose power.