areaDetector / ADPylon

An EPICS areaDetector driver for cameras from Basler using their Pylon SDK.
https://areadetector.github.io/areaDetector/ADPylon/ADPylon.html
1 stars 3 forks source link

IOC stops collecting images from the camera #2

Open AbdallaDalleh opened 10 months ago

AbdallaDalleh commented 10 months ago

Hi

OS: Rocky Linux 8.6
EPICS base: 3.15.6
Support modules: master branches for all.
Pylon SDK: 7.3

Out of no where, the driver stops collecting images from the camera even if it is connected and "Acquire" on, the IOC shell shows the error message in ADPylon.cpp:485. To resolve it, I have to restart the IOC.

xiaoqiangwang commented 10 months ago

It can only mean the camera stops sending image data. Will the Stop/Start cycle be able to recover, instead of IOC restart?

AbdallaDalleh commented 10 months ago

That's the problem, stop/start does not recover, only an IOC restart. What's more problematic is that it is not reproducible, sometimes it happens on 3M+ frames collected sometimes on few 100Ks. Even across cameras the behavior is different.

AbdallaDalleh commented 10 months ago

I just found out that MTU size (GevSCPSPacketSize) was set to 576 bytes, this caused us similar problem in the past, I set it to 1500 and the camera is working fine for 3 days. Are their certain circumstances that need MTU to be 9000?

xiaoqiangwang commented 10 months ago

From https://docs.baslerweb.com/network-configuration-(gige-cameras)

Using jumbo frames is important for reducing the overhead and the CPU load. The bigger the frame size, the less CPU interrupts are generated and thus, the lower the CPU load.

And

Basler 5GigE or GigE cameras and the recommended network adapters can both handle jumbo frames. For maximum network efficiency, your camera should be configured to use the largest packet size that your network can handle.