search

areaDetector / ADSpinnaker

An EPICS areaDetector driver for cameras from FLIR (formerly Point Grey) using their Spinnaker SDK.
https://areadetector.github.io/areaDetector/ADSpinnaker/ADSpinnaker.html
5 stars 6 forks source link

FLIR camera showing "Collecting" in Single mode permanently #6

Closed LeeYangLBLBCS closed 4 years ago

LeeYangLBLBCS commented 4 years ago

My FLIR camera has been working well for a few week in self trigger mode "i.e. Trigger Mode=Off". Today I tried the external trigger mode (line0) with a 50Hz TTL trigger pulses x 100 and file output turned on. When I noticed error from IOC: 2020/08/22 19:07:01.924 ADSpinnaker::grabImage error GetImageStatus returned 9 I thought the file write must not be able to keep up with the external trigger rate. I reverted back to internal trigger mode. The camera "Acquire Start" shows "Collecting" forever. Screen shot attached. I restarted the computer, power cycled the camera. Problem persists. I attached the camera to a different computer (Windows 10), using Spinview software from FLIR. It can reach good frame rate (~70 Hz Mono16). So I am confident the camera itself is still working fine. Have you ever seen this problem before? Is there a way to clear up the error? image 2020/08/22 19:07:01.924 ADSpinnaker::grabImage error GetImageStatus returned 9

MarkRivers commented 4 years ago

What happens if you change to Continuous mode, not Single mode?

I think I have seen a problem with being stuck in Acquiring. It is sufficient to restart the IOC, no need to power cycle the camera or reboot.

I thought I created an issue when I saw the problem, but I guess not. It happens in Internal trigger mode as well, I did not use External trigger mode when I saw the problem.

Please describe exactly what you do to create the problem. I don’t understand why you are in Single mode if you are using a burst of external triggers?

Mark

LeeYangLBLBCS commented 4 years ago

I havn't had any problem for many weeks using internal trigger, with any mode (single/multiple/continuous). I've also used the external mode prior to that occasionally and didn't have any problems either.

It happened when I used the external trigger at 50 Hz frame rate, 10 msec on, 10 msec off TTL pulse, a burst of 100 pulses. I think the image mode was in "Continuous" but I don't remember. Though I did see error messages in IOC scrolling by quickly, similar to this: 2020/08/22 19:09:57.700 ADSpinnaker::grabImage error GetImageStatus returned 9.

Then I tried to reverted back to the internal trigger to verify the single mode still works as before, but couldn't. I tried these: IOC restart, camera power cycle ethernet port disable/enable computer (ubuntu 18.04) reboot. I can still ping the camera normally though. I also checked the muex, epics> epicsMutexShowAll 1 ellCount(&mutexList) 37584 ellCount(&freeList) 0

So it doesn't appear to have mutex lock as you've seen earlier with PixelFormat=mono12Packed, though the symptom is the same.

I also tried to run spinview on a different windows PC. Everything worked normally.

On Sat, Aug 22, 2020 at 8:09 PM Mark Rivers notifications@github.com wrote:

What happens if you change to Continuous mode, not Single mode?

I think I have seen a problem with being stuck in Acquiring. It is sufficient to restart the IOC, no need to power cycle the camera or reboot.

I thought I created an issue when I saw the problem, but I guess not. It happens in Internal trigger mode as well, I did not use External trigger mode when I saw the problem.

Please describe exactly what you do to create the problem. I don’t understand why you are in Single mode if you are using a burst of external triggers?

Mark

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/areaDetector/ADSpinnaker/issues/6#issuecomment-678723055, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADSYGNGS3IKS46KK7IR5DOTSCCB6PANCNFSM4QIND25A .

-- Lee Yang Lawrence Berkeley National Lab 1 Cyclotron Road, MS 7H210 Berkeley, California 97320 office:(510)486-7320 fax:(510) 486-4633

LeeYangLBLBCS commented 4 years ago

I've always been getting some error messages all along while everything has been working, so I havn't paid attention to them. I am wondering if EPICS remembers the last bad state and is stuck in it. I attached the IOC start up messages in the file, where the particular errors are shown at the end of it: create_monitor_set("auto_settings.req", 30,"P=13SP1:") save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting.

Wait for enum callbacks to complete

epicsThreadSleep(1.0) save_restore: Can't connect to all status PV(s) save_restore: connect failed for channel '13SP1:cam1:GC_BlackLevelAuto' save_restore: connect failed for channel '13SP1:cam1:GC_SaturationEnabled' save_restore: connect failed for channel '13SP1:cam1:GC_GammaEnabled' save_restore: connect failed for channel '13SP1:cam1:GC_Sharpness' save_restore: connect failed for channel '13SP1:cam1:GC_SharpnessAuto' save_restore: connect failed for channel '13SP1:cam1:GC_SharpnessEnabled' save_restore: connect failed for channel '13SP1:cam1:GC_TriDelayEnabled' auto_settings.sav: 1290 of 1297 PV's connected

Wait for callbacks on the property limits (DRVL, DRVH) to complete

epicsThreadSleep(1.0) save_restore: Can't connect to status PV(s) for list 'auto_settings.sav' 2020/08/22 20:26:48.642 Param[ACQ_PERIOD] GenICamFeature::write: node AcquisitionFrameRate is not writable

On Sat, Aug 22, 2020 at 8:25 PM Lee L. Yang llyang@lbl.gov wrote:

I havn't had any problem for many weeks using internal trigger, with any mode (single/multiple/continuous). I've also used the external mode prior to that occasionally and didn't have any problems either.

It happened when I used the external trigger at 50 Hz frame rate, 10 msec on, 10 msec off TTL pulse, a burst of 100 pulses. I think the image mode was in "Continuous" but I don't remember. Though I did see error messages in IOC scrolling by quickly, similar to this: 2020/08/22 19:09:57.700 ADSpinnaker::grabImage error GetImageStatus returned 9.

Then I tried to reverted back to the internal trigger to verify the single mode still works as before, but couldn't. I tried these: IOC restart, camera power cycle ethernet port disable/enable computer (ubuntu 18.04) reboot. I can still ping the camera normally though. I also checked the muex, epics> epicsMutexShowAll 1 ellCount(&mutexList) 37584 ellCount(&freeList) 0

So it doesn't appear to have mutex lock as you've seen earlier with PixelFormat=mono12Packed, though the symptom is the same.

I also tried to run spinview on a different windows PC. Everything worked normally.

On Sat, Aug 22, 2020 at 8:09 PM Mark Rivers notifications@github.com wrote:

What happens if you change to Continuous mode, not Single mode?

I think I have seen a problem with being stuck in Acquiring. It is sufficient to restart the IOC, no need to power cycle the camera or reboot.

I thought I created an issue when I saw the problem, but I guess not. It happens in Internal trigger mode as well, I did not use External trigger mode when I saw the problem.

Please describe exactly what you do to create the problem. I don’t understand why you are in Single mode if you are using a burst of external triggers?

Mark

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/areaDetector/ADSpinnaker/issues/6#issuecomment-678723055, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADSYGNGS3IKS46KK7IR5DOTSCCB6PANCNFSM4QIND25A .

-- Lee Yang Lawrence Berkeley National Lab 1 Cyclotron Road, MS 7H210 Berkeley, California 97320 office:(510)486-7320 fax:(510) 486-4633

-- Lee Yang Lawrence Berkeley National Lab 1 Cyclotron Road, MS 7H210 Berkeley, California 97320 office:(510)486-7320 fax:(510) 486-4633

MEDM Version 3.1.15: Loading aliased fonts................. < envPaths epicsEnvSet("ARCH","linux-x86_64") epicsEnvSet("IOC","iocSpinnaker") epicsEnvSet("TOP","/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADSpinnaker/iocs/spinnakerIOC") epicsEnvSet("SUPPORT","/home/bl832user/epics/modules/synApps_6_1_epics7/support") epicsEnvSet("MOTOR","/home/bl832user/epics/modules/synApps_6_1_epics7/support/motor-R7-2-1") epicsEnvSet("ASYN","/home/bl832user/epics/modules/synApps_6_1_epics7/support/asyn-R4-38") epicsEnvSet("CALC","/home/bl832user/epics/modules/synApps_6_1_epics7/support/calc-R3-7-4") epicsEnvSet("EPICS_BASE","/home/bl832user/epics/base-7.0.4") epicsEnvSet("MOTOR_PV","/home/bl832user/epics/modules/motorPV/iocs/pvIOC/../..") epicsEnvSet("ADCORE","/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADCore") epicsEnvSet("ADGENICAM","/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADGenICam") epicsEnvSet("ADSPINNAKER","/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADSpinnaker") epicsEnvSet("HDF5_PLUGIN_PATH","/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADSupport/lib/linux-x86_64") epicsEnvSet("EPICS_DISPLAY_PATH","/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADCore/ADApp/op/adl/") errlogInit(20000) dbLoadDatabase("/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADSpinnaker/iocs/spinnakerIOC/dbd/spinnakerApp.dbd") spinnakerApp_registerRecordDeviceDriver(pdbbase)

Use this line for a specific camera by serial number, in this case a BlackFlyS GigE

epicsEnvSet("CAMERA_ID", "19333188")

epicsEnvSet("CAMERA_ID", "0")

epicsEnvSet("GENICAM_DB_FILE", "/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADGenICam/db/FLIR_ORX_10G_51S5M.template") < st.cmd.base

Prefix for all records

epicsEnvSet("PREFIX", "13SP1:")

The port name for the detector

epicsEnvSet("PORT", "SP1")

Really large queue so we can stream to disk at full camera speed

epicsEnvSet("QSIZE", "2000")

The maximim image width; used for row profiles in the NDPluginStats plugin

epicsEnvSet("XSIZE", "2048")

The maximim image height; used for column profiles in the NDPluginStats plugin

epicsEnvSet("YSIZE", "2048")

The maximum number of time series points in the NDPluginStats plugin

epicsEnvSet("NCHANS", "2048")

The maximum number of frames buffered in the NDPluginCircularBuff plugin

epicsEnvSet("CBUFFS", "500")

The search path for database files

This is for Windows

epicsEnvSet("EPICS_DB_INCLUDE_PATH", "$(ADCORE)/db;$(ADGENICAM)/db;$(ADSPINNAKER)/db")

This is for Linux

epicsEnvSet("EPICS_DB_INCLUDE_PATH", "/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADCore/db:/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADGenICam/db:/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADSpinnaker/db")

Define NELEMENTS to be enough for a 2048x2048x3 (color) image

epicsEnvSet("NELEMENTS", "12592912")

ADSpinnakerConfig(const char portName, const char cameraId, int traceMask, int memoryChannel,

size_t maxMemory, int priority, int stackSize)

ADSpinnakerConfig("SP1", 19333188, 0x1, 0) asynSetTraceIOMask(SP1, 0, 2)

Set ASYN_TRACE_WARNING and ASYN_TRACE_ERROR

asynSetTraceMask($(PORT), 0, 0xff)

asynSetTraceFile($(PORT), 0, "asynTrace.out")

asynSetTraceInfoMask($(PORT), 0, 0xf)

Main database. This just loads and modifies ADBase.template

dbLoadRecords("/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADSpinnaker/db/spinnaker.template", "P=13SP1:,R=cam1:,PORT=SP1")

Load the autogenerated file of GenICam features

dbLoadRecords("/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADGenICam/db/FLIR_ORX_10G_51S5M.template", "P=13SP1:,R=cam1:,PORT=SP1")

Transport diagnostics

dbLoadRecords("$(ADSPINNAKER)/db/spinnakerIntReadback.template", "P=$(PREFIX),R=cam1:,PORT=$(PORT),PROP=TransmitFailureCount,PN=SP_TRANSMIT_FAILURE_COUNT")

dbLoadRecords("$(ADSPINNAKER)/db/spinnakerIntReadback.template", "P=$(PREFIX),R=cam1:,PORT=$(PORT),PROP=BufferUnderrunCount, PN=SP_BUFFER_UNDERRUN_COUNT")

dbLoadRecords("$(ADSPINNAKER)/db/spinnakerIntReadback.template", "P=$(PREFIX),R=cam1:,PORT=$(PORT),PROP=FailedBufferCount, PN=SP_FAILED_BUFFER_COUNT")

dbLoadRecords("$(ADSPINNAKER)/db/spinnakerIntReadback.template", "P=$(PREFIX),R=cam1:,PORT=$(PORT),PROP=FailedPacketCount, PN=SP_FAILED_PACKET_COUNT")

Create a standard arrays plugin

NDStdArraysConfigure("Image1", 5, 0, "SP1", 0, 0)

Use this line for 8-bit data only

dbLoadRecords("$(ADCORE)/db/NDStdArrays.template", "P=$(PREFIX),R=image1:,PORT=Image1,ADDR=0,TIMEOUT=1,NDARRAY_PORT=$(PORT),TYPE=Int8,FTVL=CHAR,NELEMENTS=$(NELEMENTS)")

Use this line for 8-bit or 16-bit data

dbLoadRecords("/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADCore/db/NDStdArrays.template", "P=13SP1:,R=image1:,PORT=Image1,ADDR=0,TIMEOUT=1,NDARRAY_PORT=SP1,TYPE=Int16,FTVL=SHORT,NELEMENTS=12592912")

Load all other plugins using commonPlugins.cmd

< /home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADCore/iocBoot/commonPlugins.cmd

This is an example file for creating plugins

It uses the following environment variable macros

Many of the parameters defined in this file are also in commonPlugins_settings.req so if autosave is being

use the autosave value will replace the value passed to this file.

$(PREFIX) Prefix for all records

$(PORT) The port name for the detector. In autosave.

$(QSIZE) The queue size for all plugins. In autosave.

$(XSIZE) The maximum image width; used to set the maximum size for row profiles in the NDPluginStats plugin and 1-D FFT

profiles in NDPluginFFT.

$(YSIZE) The maximum image height; used to set the maximum size for column profiles in the NDPluginStats plugin

$(NCHANS) The maximum number of time series points in the NDPluginStats, NDPluginROIStats, and NDPluginAttribute plugins

$(CBUFFS) The maximum number of frames buffered in the NDPluginCircularBuff plugin

$(MAX_THREADS) The maximum number of threads for plugins which can run in multiple threads. Defaults to 5.

Create a netCDF file saving plugin

NDFileNetCDFConfigure("FileNetCDF1", 2000, 0, "SP1", 0) dbLoadRecords("NDFileNetCDF.template","P=13SP1:,R=netCDF1:,PORT=FileNetCDF1,ADDR=0,TIMEOUT=1,NDARRAY_PORT=SP1")

Create a TIFF file saving plugin

NDFileTIFFConfigure("FileTIFF1", 2000, 0, "SP1", 0) dbLoadRecords("NDFileTIFF.template", "P=13SP1:,R=TIFF1:,PORT=FileTIFF1,ADDR=0,TIMEOUT=1,NDARRAY_PORT=SP1")

Create a JPEG file saving plugin

NDFileJPEGConfigure("FileJPEG1", 2000, 0, "SP1", 0) dbLoadRecords("NDFileJPEG.template", "P=13SP1:,R=JPEG1:,PORT=FileJPEG1,ADDR=0,TIMEOUT=1,NDARRAY_PORT=SP1")

Create a NeXus file saving plugin

NDFileNexusConfigure("FileNexus1", 2000, 0, "SP1", 0) dbLoadRecords("NDFileNexus.template", "P=13SP1:,R=Nexus1:,PORT=FileNexus1,ADDR=0,TIMEOUT=1,NDARRAY_PORT=SP1")

Create an HDF5 file saving plugin

NDFileHDF5Configure("FileHDF1", 2000, 0, "SP1", 0) dbLoadRecords("NDFileHDF5.template", "P=13SP1:,R=HDF1:,PORT=FileHDF1,ADDR=0,TIMEOUT=1,XMLSIZE=2048,NDARRAY_PORT=SP1")

Create a Magick file saving plugin

NDFileMagickConfigure("FileMagick1", $(QSIZE), 0, "$(PORT)", 0)

dbLoadRecords("NDFileMagick.template","P=$(PREFIX),R=Magick1:,PORT=FileMagick1,ADDR=0,TIMEOUT=1,NDARRAY_PORT=$(PORT)")

Create 4 ROI plugins

NDROIConfigure("ROI1", 2000, 0, "SP1", 0, 0, 0, 0, 0, 5) dbLoadRecords("NDROI.template", "P=13SP1:,R=ROI1:, PORT=ROI1,ADDR=0,TIMEOUT=1,NDARRAY_PORT=SP1") NDROIConfigure("ROI2", 2000, 0, "SP1", 0, 0, 0, 0, 0, 5) dbLoadRecords("NDROI.template", "P=13SP1:,R=ROI2:, PORT=ROI2,ADDR=0,TIMEOUT=1,NDARRAY_PORT=SP1") NDROIConfigure("ROI3", 2000, 0, "SP1", 0, 0, 0, 0, 0, 5) dbLoadRecords("NDROI.template", "P=13SP1:,R=ROI3:, PORT=ROI3,ADDR=0,TIMEOUT=1,NDARRAY_PORT=SP1") NDROIConfigure("ROI4", 2000, 0, "SP1", 0, 0, 0, 0, 0, 5) dbLoadRecords("NDROI.template", "P=13SP1:,R=ROI4:, PORT=ROI4,ADDR=0,TIMEOUT=1,NDARRAY_PORT=SP1")

Create 8 ROIStat plugins

NDROIStatConfigure("ROISTAT1", 2000, 0, "SP1", 0, 8, 0, 0, 0, 0, 5) dbLoadRecords("NDROIStat.template", "P=13SP1:,R=ROIStat1: ,PORT=ROISTAT1,ADDR=0,TIMEOUT=1,NDARRAY_PORT=SP1,NCHANS=2048") dbLoadRecords("NDROIStatN.template", "P=13SP1:,R=ROIStat1:1:,PORT=ROISTAT1,ADDR=0,TIMEOUT=1,NCHANS=2048") dbLoadRecords("NDROIStatN.template", "P=13SP1:,R=ROIStat1:2:,PORT=ROISTAT1,ADDR=1,TIMEOUT=1,NCHANS=2048") dbLoadRecords("NDROIStatN.template", "P=13SP1:,R=ROIStat1:3:,PORT=ROISTAT1,ADDR=2,TIMEOUT=1,NCHANS=2048") dbLoadRecords("NDROIStatN.template", "P=13SP1:,R=ROIStat1:4:,PORT=ROISTAT1,ADDR=3,TIMEOUT=1,NCHANS=2048") dbLoadRecords("NDROIStatN.template", "P=13SP1:,R=ROIStat1:5:,PORT=ROISTAT1,ADDR=4,TIMEOUT=1,NCHANS=2048") dbLoadRecords("NDROIStatN.template", "P=13SP1:,R=ROIStat1:6:,PORT=ROISTAT1,ADDR=5,TIMEOUT=1,NCHANS=2048") dbLoadRecords("NDROIStatN.template", "P=13SP1:,R=ROIStat1:7:,PORT=ROISTAT1,ADDR=6,TIMEOUT=1,NCHANS=2048") dbLoadRecords("NDROIStatN.template", "P=13SP1:,R=ROIStat1:8:,PORT=ROISTAT1,ADDR=7,TIMEOUT=1,NCHANS=2048")

Create a processing plugin

NDProcessConfigure("PROC1", 2000, 0, "SP1", 0, 0, 0) dbLoadRecords("NDProcess.template", "P=13SP1:,R=Proc1:, PORT=PROC1,ADDR=0,TIMEOUT=1,NDARRAY_PORT=SP1")

Create a TIFF file plugin to read dark and flatfield images into the processing plugin

NDFileTIFFConfigure("PROC1TIFF", 2000, 0, "SP1", 0) dbLoadRecords("NDFileTIFF.template", "P=13SP1:,R=Proc1:TIFF:,PORT=PROC1TIFF,ADDR=0,TIMEOUT=1,NDARRAY_PORT=SP1")

Create a scatter plugin

NDScatterConfigure("SCATTER1", 2000, 0, "SP1", 0, 0, 0) dbLoadRecords("NDScatter.template", "P=13SP1:,R=Scatter1:, PORT=SCATTER1,ADDR=0,TIMEOUT=1,NDARRAY_PORT=SP1")

Create a gather plugin with 8 ports

NDGatherConfigure("GATHER1", 2000, 0, 8, 0, 0) dbLoadRecords("NDGather.template", "P=13SP1:,R=Gather1:, PORT=GATHER1,ADDR=0,TIMEOUT=1,NDARRAY_PORT=SP1") dbLoadRecords("NDGatherN.template", "P=13SP1:,R=Gather1:, N=1, PORT=GATHER1,ADDR=0,TIMEOUT=1,NDARRAY_PORT=SP1") dbLoadRecords("NDGatherN.template", "P=13SP1:,R=Gather1:, N=2, PORT=GATHER1,ADDR=1,TIMEOUT=1,NDARRAY_PORT=SP1") dbLoadRecords("NDGatherN.template", "P=13SP1:,R=Gather1:, N=3, PORT=GATHER1,ADDR=2,TIMEOUT=1,NDARRAY_PORT=SP1") dbLoadRecords("NDGatherN.template", "P=13SP1:,R=Gather1:, N=4, PORT=GATHER1,ADDR=3,TIMEOUT=1,NDARRAY_PORT=SP1") dbLoadRecords("NDGatherN.template", "P=13SP1:,R=Gather1:, N=5, PORT=GATHER1,ADDR=4,TIMEOUT=1,NDARRAY_PORT=SP1") dbLoadRecords("NDGatherN.template", "P=13SP1:,R=Gather1:, N=6, PORT=GATHER1,ADDR=5,TIMEOUT=1,NDARRAY_PORT=SP1") dbLoadRecords("NDGatherN.template", "P=13SP1:,R=Gather1:, N=7, PORT=GATHER1,ADDR=6,TIMEOUT=1,NDARRAY_PORT=SP1") dbLoadRecords("NDGatherN.template", "P=13SP1:,R=Gather1:, N=8, PORT=GATHER1,ADDR=7,TIMEOUT=1,NDARRAY_PORT=SP1")

Create 5 statistics plugins

NDStatsConfigure("STATS1", 2000, 0, "SP1", 0, 0, 0, 0, 0, 5) dbLoadRecords("NDStats.template", "P=13SP1:,R=Stats1:, PORT=STATS1,ADDR=0,TIMEOUT=1,HIST_SIZE=256,XSIZE=2048,YSIZE=2048,NCHANS=2048,NDARRAY_PORT=SP1") NDTimeSeriesConfigure("STATS1_TS", 2000, 0, "STATS1", 1, 23) dbLoadRecords("/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADCore/db/NDTimeSeries.template", "P=13SP1:,R=Stats1:TS:, PORT=STATS1_TS,ADDR=0,TIMEOUT=1,NDARRAY_PORT=STATS1,NDARRAY_ADDR=1,NCHANS=2048,ENABLED=1")

NDStatsConfigure("STATS2", 2000, 0, "ROI1", 0, 0, 0, 0, 0, 5) dbLoadRecords("NDStats.template", "P=13SP1:,R=Stats2:, PORT=STATS2,ADDR=0,TIMEOUT=1,HIST_SIZE=256,XSIZE=2048,YSIZE=2048,NCHANS=2048,NDARRAY_PORT=SP1") NDTimeSeriesConfigure("STATS2_TS", 2000, 0, "STATS2", 1, 23) dbLoadRecords("/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADCore/db/NDTimeSeries.template", "P=13SP1:,R=Stats2:TS:, PORT=STATS2_TS,ADDR=0,TIMEOUT=1,NDARRAY_PORT=STATS2,NDARRAY_ADDR=1,NCHANS=2048,ENABLED=1")

NDStatsConfigure("STATS3", 2000, 0, "ROI2", 0, 0, 0, 0, 0, 5) dbLoadRecords("NDStats.template", "P=13SP1:,R=Stats3:, PORT=STATS3,ADDR=0,TIMEOUT=1,HIST_SIZE=256,XSIZE=2048,YSIZE=2048,NCHANS=2048,NDARRAY_PORT=SP1") NDTimeSeriesConfigure("STATS3_TS", 2000, 0, "STATS3", 1, 23) dbLoadRecords("/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADCore/db/NDTimeSeries.template", "P=13SP1:,R=Stats3:TS:, PORT=STATS3_TS,ADDR=0,TIMEOUT=1,NDARRAY_PORT=STATS3,NDARRAY_ADDR=1,NCHANS=2048,ENABLED=1")

NDStatsConfigure("STATS4", 2000, 0, "ROI3", 0, 0, 0, 0, 0, 5) dbLoadRecords("NDStats.template", "P=13SP1:,R=Stats4:, PORT=STATS4,ADDR=0,TIMEOUT=1,HIST_SIZE=256,XSIZE=2048,YSIZE=2048,NCHANS=2048,NDARRAY_PORT=SP1") NDTimeSeriesConfigure("STATS4_TS", 2000, 0, "STATS4", 1, 23) dbLoadRecords("/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADCore/db/NDTimeSeries.template", "P=13SP1:,R=Stats4:TS:, PORT=STATS4_TS,ADDR=0,TIMEOUT=1,NDARRAY_PORT=STATS4,NDARRAY_ADDR=1,NCHANS=2048,ENABLED=1")

NDStatsConfigure("STATS5", 2000, 0, "ROI4", 0, 0, 0, 0, 0, 5) dbLoadRecords("NDStats.template", "P=13SP1:,R=Stats5:, PORT=STATS5,ADDR=0,TIMEOUT=1,HIST_SIZE=256,XSIZE=2048,YSIZE=2048,NCHANS=2048,NDARRAY_PORT=SP1") NDTimeSeriesConfigure("STATS5_TS", 2000, 0, "STATS5", 1, 23) dbLoadRecords("/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADCore/db/NDTimeSeries.template", "P=13SP1:,R=Stats5:TS:, PORT=STATS5_TS,ADDR=0,TIMEOUT=1,NDARRAY_PORT=STATS5,NDARRAY_ADDR=1,NCHANS=2048,ENABLED=1")

Create a transform plugin

NDTransformConfigure("TRANS1", 2000, 0, "SP1", 0, 0, 0, 0, 0, 5) dbLoadRecords("NDTransform.template", "P=13SP1:,R=Trans1:, PORT=TRANS1,ADDR=0,TIMEOUT=1,NDARRAY_PORT=SP1")

Create an overlay plugin with 8 overlays

NDOverlayConfigure("OVER1", 2000, 0, "SP1", 0, 8, 0, 0, 0, 0, 5) dbLoadRecords("NDOverlay.template", "P=13SP1:,R=Over1:, PORT=OVER1,ADDR=0,TIMEOUT=1,NDARRAY_PORT=SP1") dbLoadRecords("NDOverlayN.template","P=13SP1:,R=Over1:1:,NAME=ROI1, SHAPE=1,O=Over1:,XPOS=13SP1:ROI1:MinX_RBV,YPOS=13SP1:ROI1:MinY_RBV,XSIZE=13SP1:ROI1:SizeX_RBV,YSIZE=13SP1:ROI1:SizeY_RBV,PORT=OVER1,ADDR=0,TIMEOUT=1") dbLoadRecords("NDOverlayN.template","P=13SP1:,R=Over1:2:,NAME=ROI2, SHAPE=1,O=Over1:,XPOS=13SP1:ROI2:MinX_RBV,YPOS=13SP1:ROI2:MinY_RBV,XSIZE=13SP1:ROI2:SizeX_RBV,YSIZE=13SP1:ROI2:SizeY_RBV,PORT=OVER1,ADDR=1,TIMEOUT=1") dbLoadRecords("NDOverlayN.template","P=13SP1:,R=Over1:3:,NAME=ROI3, SHAPE=1,O=Over1:,XPOS=13SP1:ROI3:MinX_RBV,YPOS=13SP1:ROI3:MinY_RBV,XSIZE=13SP1:ROI3:SizeX_RBV,YSIZE=13SP1:ROI3:SizeY_RBV,PORT=OVER1,ADDR=2,TIMEOUT=1") dbLoadRecords("NDOverlayN.template","P=13SP1:,R=Over1:4:,NAME=ROI4, SHAPE=1,O=Over1:,XPOS=13SP1:ROI4:MinX_RBV,YPOS=13SP1:ROI4:MinY_RBV,XSIZE=13SP1:ROI4:SizeX_RBV,YSIZE=13SP1:ROI4:SizeY_RBV,PORT=OVER1,ADDR=3,TIMEOUT=1") dbLoadRecords("NDOverlayN.template","P=13SP1:,R=Over1:5:,NAME=Cursor1,SHAPE=1,O=Over1:,XPOS=junk, YPOS=junk, XSIZE=junk, YSIZE=junk, PORT=OVER1,ADDR=4,TIMEOUT=1") dbLoadRecords("NDOverlayN.template","P=13SP1:,R=Over1:6:,NAME=Cursor2,SHAPE=1,O=Over1:,XPOS=junk, YPOS=junk, XSIZE=junk, YSIZE=junk, PORT=OVER1,ADDR=5,TIMEOUT=1") dbLoadRecords("NDOverlayN.template","P=13SP1:,R=Over1:7:,NAME=Box1, SHAPE=1,O=Over1:,XPOS=junk, YPOS=junk, XSIZE=junk, YSIZE=junk, PORT=OVER1,ADDR=6,TIMEOUT=1") dbLoadRecords("NDOverlayN.template","P=13SP1:,R=Over1:8:,NAME=Box2, SHAPE=1,O=Over1:,XPOS=junk, YPOS=junk, XSIZE=junk, YSIZE=junk, PORT=OVER1,ADDR=7,TIMEOUT=1")

Create 2 color conversion plugins

NDColorConvertConfigure("CC1", 2000, 0, "SP1", 0, 0, 0, 0, 0, 5) dbLoadRecords("NDColorConvert.template", "P=13SP1:,R=CC1:, PORT=CC1,ADDR=0,TIMEOUT=1,NDARRAY_PORT=SP1") NDColorConvertConfigure("CC2", 2000, 0, "SP1", 0, 0, 0, 0, 0, 5) dbLoadRecords("NDColorConvert.template", "P=13SP1:,R=CC2:, PORT=CC2,ADDR=0,TIMEOUT=1,NDARRAY_PORT=SP1")

Create a circular buffer plugin

NDCircularBuffConfigure("CB1", 2000, 0, "SP1", 0, 500, 0) dbLoadRecords("NDCircularBuff.template", "P=13SP1:,R=CB1:, PORT=CB1,ADDR=0,TIMEOUT=1,NDARRAY_PORT=SP1")

Create an NDAttribute plugin with 8 attributes

NDAttrConfigure("ATTR1", 2000, 0, "SP1", 0, 8, 0, 0, 0) dbLoadRecords("NDAttribute.template", "P=13SP1:,R=Attr1:, PORT=ATTR1,ADDR=0,TIMEOUT=1,NCHANS=2048,NDARRAY_PORT=SP1") dbLoadRecords("NDAttributeN.template", "P=13SP1:,R=Attr1:1:, PORT=ATTR1,ADDR=0,TIMEOUT=1,NCHANS=2048") dbLoadRecords("NDAttributeN.template", "P=13SP1:,R=Attr1:2:, PORT=ATTR1,ADDR=1,TIMEOUT=1,NCHANS=2048") dbLoadRecords("NDAttributeN.template", "P=13SP1:,R=Attr1:3:, PORT=ATTR1,ADDR=2,TIMEOUT=1,NCHANS=2048") dbLoadRecords("NDAttributeN.template", "P=13SP1:,R=Attr1:4:, PORT=ATTR1,ADDR=3,TIMEOUT=1,NCHANS=2048") dbLoadRecords("NDAttributeN.template", "P=13SP1:,R=Attr1:5:, PORT=ATTR1,ADDR=4,TIMEOUT=1,NCHANS=2048") dbLoadRecords("NDAttributeN.template", "P=13SP1:,R=Attr1:6:, PORT=ATTR1,ADDR=5,TIMEOUT=1,NCHANS=2048") dbLoadRecords("NDAttributeN.template", "P=13SP1:,R=Attr1:7:, PORT=ATTR1,ADDR=6,TIMEOUT=1,NCHANS=2048") dbLoadRecords("NDAttributeN.template", "P=13SP1:,R=Attr1:8:, PORT=ATTR1,ADDR=7,TIMEOUT=1,NCHANS=2048") NDTimeSeriesConfigure("ATTR1_TS", 2000, 0, "ATTR1", 1, 8) dbLoadRecords("/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADCore/db/NDTimeSeries.template", "P=13SP1:,R=Attr1:TS:, PORT=ATTR1_TS,ADDR=0,TIMEOUT=1,NDARRAY_PORT=ATTR1,NDARRAY_ADDR=1,NCHANS=2048,ENABLED=1")

Create an FFT plugin

NDFFTConfigure("FFT1", 2000, 0, "SP1", 0, 0, 0, 0, 0, 5) dbLoadRecords("NDFFT.template", "P=13SP1:, R=FFT1:, PORT=FFT1, ADDR=0, TIMEOUT=1, NDARRAY_PORT=SP1, NAME=FFT1, NCHANS=2048")

Create 2 Codec plugins

NDCodecConfigure("CODEC1", 2000, 0, "SP1", 0, 0, 0, 0, 0, 5) dbLoadRecords("NDCodec.template", "P=13SP1:, R=Codec1:, PORT=CODEC1, ADDR=0, TIMEOUT=1, NDARRAY_PORT=SP1") NDCodecConfigure("CODEC2", 2000, 0, "SP1", 0, 0, 0, 0, 0, 5) dbLoadRecords("NDCodec.template", "P=13SP1:, R=Codec2:, PORT=CODEC2, ADDR=0, TIMEOUT=1, NDARRAY_PORT=SP1")

set_requestfile_path("./") set_requestfile_path("/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADCore/ADApp/Db") set_requestfile_path("/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADCore/iocBoot") set_savefile_path("./autosave") set_pass0_restoreFile("auto_settings.sav") set_pass1_restoreFile("auto_settings.sav") save_restoreSet_status_prefix("13SP1:") )acLib: macro AUTOSAVE is undefined (expanding string dbLoadRecords("$(AUTOSAVE)/asApp/Db/save_restoreStatus.db", "P=$(PREFIX)")

Optional: load NDPluginPva plugin

NDPvaConfigure("PVA1", $(QSIZE), 0, "$(PORT)", 0, $(PREFIX)Pva1:Image, 0, 0, 0)

dbLoadRecords("NDPva.template", "P=$(PREFIX),R=Pva1:, PORT=PVA1,ADDR=0,TIMEOUT=1,NDARRAY_PORT=$(PORT)")

Must start PVA server if this is enabled

startPVAServer

Optional: load ffmpegServer plugin

ffmpegServerConfigure(8081)

ffmpegStreamConfigure("FfmStream1", 2, 0, "$(PORT)", 0, -1, 0)

dbLoadRecords("$(FFMPEGSERVER)/db/ffmpegStream.template", "P=$(PREFIX),R=ffmstream1:,PORT=FfmStream1,NDARRAY_PORT=$(PORT)")

ffmpegFileConfigure("FfmFile1", 16, 0, "$(PORT)", 0, -1, 0)

dbLoadRecords("$(FFMPEGSERVER)/db/ffmpegFile.template", "P=$(PREFIX),R=ffmfile1:,PORT=FfmFile1,NDARRAY_PORT=$(PORT)")

Optional: load NDPluginEdge plugin

NDEdgeConfigure("EDGE1", $(QSIZE), 0, "$(PORT)", 0, 0, 0, 0)

dbLoadRecords("$(ADPLUGINEDGE)/db/NDEdge.template", "P=$(PREFIX),R=Edge1:, PORT=EDGE1,ADDR=0,TIMEOUT=1,NDARRAY_PORT=$(PORT)")

set_requestfile_path("$(ADPLUGINEDGE)/edgeApp/Db")

Optional: load NDPluginCV plugin

NDCVConfigure("CV1", $(QSIZE), 0, "$(PORT)", 0, 0, 0, 0, 0, $(MAX_THREADS=5))

dbLoadRecords("$(ADCOMPVISION)/db/NDCV.template", "P=$(PREFIX),R=CV1:, PORT=CV1,ADDR=0,TIMEOUT=1,NDARRAY_PORT=$(PORT)")

set_requestfile_path("$(ADCOMPVISION)/adcvApp/Db")

Optional: load NDPluginBar plugin

NDBarConfigure("BAR1", $(QSIZE), 0, "$(PORT)", 0, 0, 0, 0, 0, $(MAX_THREADS=5))

dbLoadRecords("$(ADPLUGINBAR)/db/NDBar.template", "P=$(PREFIX),R=Bar1:, PORT=BAR1,ADDR=0,TIMEOUT=1,NDARRAY_PORT=$(PORT)")

set_requestfile_path("$(ADPLUGINBAR)/barApp/Db")

Optional: load scan records

dbLoadRecords("$(SSCAN)/sscanApp/Db/scan.db", "P=$(PREFIX),MAXPTS1=2000,MAXPTS2=200,MAXPTS3=20,MAXPTS4=10,MAXPTSH=10")

set_requestfile_path("$(SSCAN)/sscanApp/Db")

Optional: load sseq record for acquisition sequence

dbLoadRecords("$(CALC)/calcApp/Db/sseqRecord.db", "P=$(PREFIX), S=AcquireSequence")

set_requestfile_path("$(CALC)/calcApp/Db")

Optional: load devIocStats records (requires DEVIOCSTATS module)

dbLoadRecords("$(DEVIOCSTATS)/db/iocAdminSoft.db", "IOC=$(PREFIX)")

Optional: load alive record (requires ALIVE module)

dbLoadRecords("$(ALIVE)/aliveApp/Db/alive.db", "P=$(PREFIX),RHOST=192.168.1.254")

Set the callback queue size to 5000, up from default of 2000 in base.

This can be needed to avoid errors "callbackRequest: cbLow ring buffer full".

callbackSetQueueSize(5000) set_requestfile_path("/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADGenICam/GenICamApp/Db") set_requestfile_path("/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADSpinnaker/spinnakerApp/Db") iocInit() Starting iocInit ############################################################################

EPICS R7.0.4

Rev. 2020-08-08T21:53-0700

############################################################################ reboot_restore: entry for file 'auto_settings.sav' reboot_restore: Found filename 'auto_settings.sav' in restoreFileList. restoring from './autosave/auto_settings.sav' at initHookState 6 (before record/device init) 7 PVs had no saved value. reboot_restore: done with file 'auto_settings.sav'

reboot_restore: entry for file 'auto_settings.sav' reboot_restore: Found filename 'auto_settings.sav' in restoreFileList. restoring from './autosave/auto_settings.sav' at initHookState 7 (after record/device init) 7 PVs had no saved value. reboot_restore: done with file 'auto_settings.sav'

iocRun: All initialization complete

save things every thirty seconds

create_monitor_set("auto_settings.req", 30,"P=13SP1:") save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting.

Wait for enum callbacks to complete

epicsThreadSleep(1.0) save_restore: Can't connect to all status PV(s) save_restore: connect failed for channel '13SP1:cam1:GC_BlackLevelAuto' save_restore: connect failed for channel '13SP1:cam1:GC_SaturationEnabled' save_restore: connect failed for channel '13SP1:cam1:GC_GammaEnabled' save_restore: connect failed for channel '13SP1:cam1:GC_Sharpness' save_restore: connect failed for channel '13SP1:cam1:GC_SharpnessAuto' save_restore: connect failed for channel '13SP1:cam1:GC_SharpnessEnabled' save_restore: connect failed for channel '13SP1:cam1:GC_TriDelayEnabled' auto_settings.sav: 1290 of 1297 PV's connected

Wait for callbacks on the property limits (DRVL, DRVH) to complete

epicsThreadSleep(1.0) save_restore: Can't connect to status PV(s) for list 'auto_settings.sav' 2020/08/22 20:26:48.642 Param[ACQ_PERIOD] GenICamFeature::write: node AcquisitionFrameRate is not writable

epics> 2020/08/22 20:26:59.426 ADSpinnaker::grabImage error GetImageStatus returned 9

LeeYangLBLBCS commented 4 years ago

One more piece of information I forgot to mention:I left TIFF and HDF5 file plugin turned on, with autosave enabled. This could be the reason to cause the problem as the disk is not able to write full size image at 50 Hz, esp. with both files written at the same time.I attached screen shots for both of these file setup screens: image image

LeeYangLBLBCS commented 4 years ago

It turns out the parameter "DeviceLinkThroughputLimit" lost its value for some reason. It defaults to 848175339 (800M?) after any power loss, or maybe some other hiccups. Only if I set it to about 300M (348175339) will it work again.

This parameter is on Feature screen #2.

I can't find a way to save its value, either from EPICS or Spinview (v1.20).

On Sat, Aug 22, 2020 at 8:32 PM Lee L. Yang llyang@lbl.gov wrote:

I've always been getting some error messages all along while everything has been working, so I havn't paid attention to them. I am wondering if EPICS remembers the last bad state and is stuck in it. I attached the IOC start up messages in the file, where the particular errors are shown at the end of it: create_monitor_set("auto_settings.req", 30,"P=13SP1:") save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting. save_restore:readReqFile: unable to open file sseq_settings.req. Exiting.

Wait for enum callbacks to complete

epicsThreadSleep(1.0) save_restore: Can't connect to all status PV(s) save_restore: connect failed for channel '13SP1:cam1:GC_BlackLevelAuto' save_restore: connect failed for channel '13SP1:cam1:GC_SaturationEnabled' save_restore: connect failed for channel '13SP1:cam1:GC_GammaEnabled' save_restore: connect failed for channel '13SP1:cam1:GC_Sharpness' save_restore: connect failed for channel '13SP1:cam1:GC_SharpnessAuto' save_restore: connect failed for channel '13SP1:cam1:GC_SharpnessEnabled' save_restore: connect failed for channel '13SP1:cam1:GC_TriDelayEnabled' auto_settings.sav: 1290 of 1297 PV's connected

Wait for callbacks on the property limits (DRVL, DRVH) to complete

epicsThreadSleep(1.0) save_restore: Can't connect to status PV(s) for list 'auto_settings.sav' 2020/08/22 20:26:48.642 Param[ACQ_PERIOD] GenICamFeature::write: node AcquisitionFrameRate is not writable

On Sat, Aug 22, 2020 at 8:25 PM Lee L. Yang llyang@lbl.gov wrote:

I havn't had any problem for many weeks using internal trigger, with any mode (single/multiple/continuous). I've also used the external mode prior to that occasionally and didn't have any problems either.

It happened when I used the external trigger at 50 Hz frame rate, 10 msec on, 10 msec off TTL pulse, a burst of 100 pulses. I think the image mode was in "Continuous" but I don't remember. Though I did see error messages in IOC scrolling by quickly, similar to this: 2020/08/22 19:09:57.700 ADSpinnaker::grabImage error GetImageStatus returned 9.

Then I tried to reverted back to the internal trigger to verify the single mode still works as before, but couldn't. I tried these: IOC restart, camera power cycle ethernet port disable/enable computer (ubuntu 18.04) reboot. I can still ping the camera normally though. I also checked the muex, epics> epicsMutexShowAll 1 ellCount(&mutexList) 37584 ellCount(&freeList) 0

So it doesn't appear to have mutex lock as you've seen earlier with PixelFormat=mono12Packed, though the symptom is the same.

I also tried to run spinview on a different windows PC. Everything worked normally.

On Sat, Aug 22, 2020 at 8:09 PM Mark Rivers notifications@github.com wrote:

What happens if you change to Continuous mode, not Single mode?

I think I have seen a problem with being stuck in Acquiring. It is sufficient to restart the IOC, no need to power cycle the camera or reboot.

I thought I created an issue when I saw the problem, but I guess not. It happens in Internal trigger mode as well, I did not use External trigger mode when I saw the problem.

Please describe exactly what you do to create the problem. I don’t understand why you are in Single mode if you are using a burst of external triggers?

Mark

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/areaDetector/ADSpinnaker/issues/6#issuecomment-678723055, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADSYGNGS3IKS46KK7IR5DOTSCCB6PANCNFSM4QIND25A .

-- Lee Yang Lawrence Berkeley National Lab 1 Cyclotron Road, MS 7H210 Berkeley, California 97320 office:(510)486-7320 fax:(510) 486-4633

-- Lee Yang Lawrence Berkeley National Lab 1 Cyclotron Road, MS 7H210 Berkeley, California 97320 office:(510)486-7320 fax:(510) 486-4633

-- Lee Yang Lawrence Berkeley National Lab 1 Cyclotron Road, MS 7H210 Berkeley, California 97320 office:(510)486-7320 fax:(510) 486-4633

MarkRivers commented 4 years ago

It is easy to save its value. You just need to add this line to auto_settings.req in the iocSpinnaker directory (or whatever you have called your IOC boot directory).

$(P)$(R)GC_DevLinThrLimit

However, it appears to me that something must be wrong with your system or some other settings. 848175339 should work fine. In fact if you set it to less than this then you will not be able to run the camera at its maximum speed.

I just tested it on my Oryx camera. If I set GC_DevLinThrLimit to 848175339 and then request 164 frames/s for the FrameRate I get the maximum supported frame rate of 163.121 frames/s. However, if I set GC_DevLinThrLimit to 348175339 then I can only get 69.072 frames/s when I request 164 frames/s.

Have you set the system Ethernet buffer settings large enough, and did they persist through a reboot?

Here are the screen shots of my Oryx screens when I am running the camera at its maximum rate of 163.121 frames/s.

This is the main screen. Note that PixelFormat is Mono8, and FrameRate_RBV is 163.121.

image

This is the Features #2 screen. Note that AdcBitDepth is Bit8, and GC_DevLinThrLimit is 1250000000. That is the full bandwidth of 10 GigE. The current throughput being used (GC_DevLinCurThr_RBV) is 822399201.

image

I am not getting any error messages on the IOC console when streaming at that rate (no dropped frames).

The error message you were getting was:

2020/08/22 19:09:57.700 ADSpinnaker::grabImage error GetImageStatus returned 9.

From SpinnakerDefs.h this is what ImageStatus=9 means:

        IMAGE_DATA_INCOMPLETE = 9, /**< Image data is incomplete. */

That means you are dropping packets when you increase your network bandwidth above some value. That should not happen. Something is wrong with your system, it is not happening for me.

LeeYangLBLBCS commented 4 years ago

Indeed, there is something wrong with this particular computer. After reinstalling the same software, with the same 10G NIC, I got the same frame rate, 162 f/s.

On Mon, Aug 24, 2020 at 3:57 PM Mark Rivers notifications@github.com wrote:

It is easy to save its value. You just need to add this line to auto_settings.req in the iocSpinnaker directory (or whatever you have called your IOC boot directory).

$(P)$(R)GC_DevLinThrLimit

However, it appears to me that something must be wrong with your system or some other settings. 848175339 should work fine. In fact if you set it to less than this then you will not be able to run the camera at its maximum speed.

I just tested it on my Oryx camera. If I set GC_DevLinThrLimit to 848175339 and then request 164 frames/s for the FrameRate I get the maximum supported frame rate of 163.121 frames/s. However, if I set GC_DevLinThrLimit to 348175339 then I can only get 69.072 frames/s when I request 164 frames/s.

Have you set the system Ethernet buffer settings large enough, and did they persist through a reboot?

Here are the screen shots of my Oryx screens when I am running the camera at its maximum rate of 163.121 frames/s.

This is the main screen. Note that PixelFormat is Mono8, and FrameRate_RBV is 163.121.

[image: image] https://user-images.githubusercontent.com/6115534/91103572-6d508180-e631-11ea-9b4a-fe4370345545.png

This is the Features#2 screen. Note that AdcBitDepth is Bit8, and GC_DevLinThrLimit is 1250000000. That is the full bandwidth of 10 GigE. The current throughput being used (GC_DevLinCurThr_RBV) is 822399201.

[image: image] https://user-images.githubusercontent.com/6115534/91104221-f320fc80-e632-11ea-9b9b-08a5814c18b1.png

I am not getting any error messages on the IOC console when streaming at that rate (no dropped frames).

The error message you were getting was:

2020/08/22 19:09:57.700 ADSpinnaker::grabImage error GetImageStatus returned 9.

From SpinnakerDefs.h this is what ImageStatus=9 means:

    IMAGE_DATA_INCOMPLETE = 9, /**< Image data is incomplete. */

That means you are dropping packets when you increase your network bandwidth above some value. That should not happen. Something is wrong with your system, it is not happening for me.

If you cannot achieve this performance.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/areaDetector/ADSpinnaker/issues/6#issuecomment-679407331, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADSYGNG73Q7OM7Y2JNC3DC3SCLV5BANCNFSM4QIND25A .

-- Lee Yang Lawrence Berkeley National Lab 1 Cyclotron Road, MS 7H210 Berkeley, California 97320 office:(510)486-7320 fax:(510) 486-4633

MarkRivers commented 4 years ago

What software did you reinstall?

One possible cause for your problems is that some other parameter was set incorrectly and saved in iocSpinnaker/iocBoot/auto_settings.req. Each time you restart the IOC that bad parameter would be reloaded, even after rebooting the computer and power-cycling the camera. To eliminate that possibility you can do the following:

You now have a "clean start" where all parameters will have their default values from the camera or from the EPICS database, with no autosaved values being restored.

MarkRivers commented 4 years ago

@LeeYangLBLBCS here are some comments on other errors you posted above:

)acLib: macro AUTOSAVE is undefined (expanding string dbLoadRecords("$(AUTOSAVE)/asApp/Db/save_restoreStatus.db", "P=$(PREFIX)")

That error is happening because AUTOSAVE is not defined in the envPaths file in your iocSpinnaker directory. Here is what is defined, judging from the output you posted above.

epicsEnvSet("ARCH","linux-x86_64")
epicsEnvSet("IOC","iocSpinnaker")
epicsEnvSet("TOP","/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADSpinnaker/iocs/spinnakerIOC")
epicsEnvSet("SUPPORT","/home/bl832user/epics/modules/synApps_6_1_epics7/support")
epicsEnvSet("MOTOR","/home/bl832user/epics/modules/synApps_6_1_epics7/support/motor-R7-2-1")
epicsEnvSet("ASYN","/home/bl832user/epics/modules/synApps_6_1_epics7/support/asyn-R4-38")
epicsEnvSet("CALC","/home/bl832user/epics/modules/synApps_6_1_epics7/support/calc-R3-7-4")
epicsEnvSet("EPICS_BASE","/home/bl832user/epics/base-7.0.4")
epicsEnvSet("MOTOR_PV","/home/bl832user/epics/modules/motorPV/iocs/pvIOC/../..")
epicsEnvSet("ADCORE","/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADCore")
epicsEnvSet("ADGENICAM","/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADGenICam")
epicsEnvSet("ADSPINNAKER","/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADSpinnaker")

This is what is defined in my iocSpinnaker/envPaths:

epicsEnvSet("IOC","iocSpinnaker")
epicsEnvSet("TOP","/corvette/home/epics/devel/areaDetector-3-9/ADSpinnaker/iocs/spinnakerIOC")
epicsEnvSet("ADSPINNAKER","/corvette/home/epics/devel/areaDetector-3-9/ADSpinnaker/iocs/spinnakerIOC/../..")
epicsEnvSet("SUPPORT","/corvette/home/epics/devel")
epicsEnvSet("ASYN","/corvette/home/epics/devel/asyn-4-40")
epicsEnvSet("AREA_DETECTOR","/corvette/home/epics/devel/areaDetector-3-9")
epicsEnvSet("ADSUPPORT","/corvette/home/epics/devel/areaDetector-3-9/ADSupport")
epicsEnvSet("ADCORE","/corvette/home/epics/devel/areaDetector-3-9/ADCore")
epicsEnvSet("AUTOSAVE","/corvette/home/epics/devel/autosave-5-10")
epicsEnvSet("BUSY","/corvette/home/epics/devel/busy-1-7-2")
epicsEnvSet("CALC","/corvette/home/epics/devel/calc-3-7-3")
epicsEnvSet("SNCSEQ","/corvette/home/epics/devel/seq-2-2-5")
epicsEnvSet("SSCAN","/corvette/home/epics/devel/sscan-2-11-3")
epicsEnvSet("DEVIOCSTATS","/corvette/home/epics/devel/devIocStats-3-1-16")
epicsEnvSet("EPICS_BASE","/corvette/usr/local/epics-devel/base-7.0.4")
epicsEnvSet("ADGENICAM","/corvette/home/epics/devel/areaDetector-3-9/ADGenICam")

Note that the following are defined for me but not for you:

It gets these definitions from the areaDetector/configure/RELEASE_PRODS.local. Did you comment out BUSY and AUTOSAVE in that file? You should not, they are needed. And if they are not commented out they should be in envPaths.

Did you edit envPaths? I see that you have an entry MOTOR_PV that is unique to your site. How did you add that?

save_restore:readReqFile: unable to open file sseq_settings.req. Exiting.

That error is coming because you have left this line in your st.cmd commented out:

#set_requestfile_path("$(CALC)/calcApp/Db")

NDPluginStats uses the sseq record, so I suggest uncommenting that line.

save_restore: connect failed for channel '13SP1:cam1:GC_BlackLevelAuto'
save_restore: connect failed for channel '13SP1:cam1:GC_SaturationEnabled'
save_restore: connect failed for channel '13SP1:cam1:GC_GammaEnabled'
save_restore: connect failed for channel '13SP1:cam1:GC_Sharpness'
save_restore: connect failed for channel '13SP1:cam1:GC_SharpnessAuto'
save_restore: connect failed for channel '13SP1:cam1:GC_SharpnessEnabled'
save_restore: connect failed for channel '13SP1:cam1:GC_TriDelayEnabled'

Those error are normal. They are happening because many FLIR cameras have those features, but the Oryx does not, so they are not defined. You can ignore them.

auto_settings.sav: 1290 of 1297 PV's connected

All 1297 should have connected. It may be because AUTOSAVE is not defined.

2020/08/22 20:26:48.642 Param[ACQ_PERIOD] GenICamFeature::write: node AcquisitionFrameRate is not writable

That error is normal. It tried to restore the FrameRate but that feature is not writable in the current camera mode.

epics> 2020/08/22 20:26:59.426 ADSpinnaker::grabImage error GetImageStatus returned 9

That is the streaming error discussed above. It means something is wrong with your network connection to the camera.

LeeYangLBLBCS commented 4 years ago

envPaths is not auto generated. I created it somehow manually.

I must have made some mistakes in config files. I added ADSpinnaker entry in RELEASE.local under areaDetector/config: ADSIMDETECTOR=$(AREA_DETECTOR)/ADSimDetector ADSUPPORT=$(AREA_DETECTOR)/ADSupport ADGENICAM=$(AREA_DETECTOR)/ADGenICam ADSPINNAKER=$(AREA_DETECTOR)/ADSpinnaker -include $(TOP)/configure/RELEASE_PRODS.local -include $(TOP)/configure/RELEASE.local.$(EPICS_HOST_ARCH)

Should I also add something in RELEASE_PRODS.local for ADSpinnaker?

On Tue, Aug 25, 2020 at 6:50 AM Mark Rivers notifications@github.com wrote:

@LeeYangLBLBCS https://github.com/LeeYangLBLBCS here are some comments on other errors you posted above:

)acLib: macro AUTOSAVE is undefined (expanding string dbLoadRecords("$(AUTOSAVE)/asApp/Db/save_restoreStatus.db", "P=$(PREFIX)")

That error is happening because AUTOSAVE is not defined in the envPaths file in your iocSpinnaker directory. Here is what is defined, judging from the output you posted above.

epicsEnvSet("ARCH","linux-x86_64") epicsEnvSet("IOC","iocSpinnaker") epicsEnvSet("TOP","/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADSpinnaker/iocs/spinnakerIOC") epicsEnvSet("SUPPORT","/home/bl832user/epics/modules/synApps_6_1_epics7/support") epicsEnvSet("MOTOR","/home/bl832user/epics/modules/synApps_6_1_epics7/support/motor-R7-2-1") epicsEnvSet("ASYN","/home/bl832user/epics/modules/synApps_6_1_epics7/support/asyn-R4-38") epicsEnvSet("CALC","/home/bl832user/epics/modules/synApps_6_1_epics7/support/calc-R3-7-4") epicsEnvSet("EPICS_BASE","/home/bl832user/epics/base-7.0.4") epicsEnvSet("MOTOR_PV","/home/bl832user/epics/modules/motorPV/iocs/pvIOC/../..") epicsEnvSet("ADCORE","/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADCore") epicsEnvSet("ADGENICAM","/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADGenICam") epicsEnvSet("ADSPINNAKER","/home/bl832user/epics/modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADSpinnaker")

This is what is defined in my iocSpinnaker/envPaths:

epicsEnvSet("IOC","iocSpinnaker") epicsEnvSet("TOP","/corvette/home/epics/devel/areaDetector-3-9/ADSpinnaker/iocs/spinnakerIOC") epicsEnvSet("ADSPINNAKER","/corvette/home/epics/devel/areaDetector-3-9/ADSpinnaker/iocs/spinnakerIOC/../..") epicsEnvSet("SUPPORT","/corvette/home/epics/devel") epicsEnvSet("ASYN","/corvette/home/epics/devel/asyn-4-40") epicsEnvSet("AREA_DETECTOR","/corvette/home/epics/devel/areaDetector-3-9") epicsEnvSet("ADSUPPORT","/corvette/home/epics/devel/areaDetector-3-9/ADSupport") epicsEnvSet("ADCORE","/corvette/home/epics/devel/areaDetector-3-9/ADCore") epicsEnvSet("AUTOSAVE","/corvette/home/epics/devel/autosave-5-10") epicsEnvSet("BUSY","/corvette/home/epics/devel/busy-1-7-2") epicsEnvSet("CALC","/corvette/home/epics/devel/calc-3-7-3") epicsEnvSet("SNCSEQ","/corvette/home/epics/devel/seq-2-2-5") epicsEnvSet("SSCAN","/corvette/home/epics/devel/sscan-2-11-3") epicsEnvSet("DEVIOCSTATS","/corvette/home/epics/devel/devIocStats-3-1-16") epicsEnvSet("EPICS_BASE","/corvette/usr/local/epics-devel/base-7.0.4") epicsEnvSet("ADGENICAM","/corvette/home/epics/devel/areaDetector-3-9/ADGenICam")

Note that the following are defined for me but not for you:

  • AREA_DETECTOR
  • ADSUPPORT
  • AUTOSAVE
  • BUSY
  • SNCSEQ
  • SSCAN
  • DEVIOCSTATS

It gets these definitions from the areaDetector/configure/RELEASE_PRODS.local. Did you comment out BUSY and AUTOSAVE in that file? You should not, they are needed. And if they are not commented out they should be in envPaths.

Did you edit envPaths? I see that you have an entry MOTOR_PV that is unique to your site. How did you add that?

save_restore:readReqFile: unable to open file sseq_settings.req. Exiting.

That error is coming because you have left this line in your st.cmd commented out:

set_requestfile_path("$(CALC)/calcApp/Db")

NDPluginStats uses the sseq record, so I suggest uncommenting that line.

save_restore: connect failed for channel '13SP1:cam1:GC_BlackLevelAuto' save_restore: connect failed for channel '13SP1:cam1:GC_SaturationEnabled' save_restore: connect failed for channel '13SP1:cam1:GC_GammaEnabled' save_restore: connect failed for channel '13SP1:cam1:GC_Sharpness' save_restore: connect failed for channel '13SP1:cam1:GC_SharpnessAuto' save_restore: connect failed for channel '13SP1:cam1:GC_SharpnessEnabled' save_restore: connect failed for channel '13SP1:cam1:GC_TriDelayEnabled'

Those error are normal. They are happening because many FLIR cameras have those features, but the Oryx does not, so they are not defined. You can ignore them.

auto_settings.sav: 1290 of 1297 PV's connected

All 1297 should have connected. It may be because AUTOSAVE is not defined.

2020/08/22 20:26:48.642 Param[ACQ_PERIOD] GenICamFeature::write: node AcquisitionFrameRate is not writable

That error is normal. It tried to restore the FrameRate but that feature is not writable in the current camera mode.

epics> 2020/08/22 20:26:59.426 ADSpinnaker::grabImage error GetImageStatus returned 9

That is the streaming error discussed above. It means something is wrong with your network connection to the camera.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/areaDetector/ADSpinnaker/issues/6#issuecomment-680037723, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADSYGNFANNCJZV5KYSGGK5TSCO6UBANCNFSM4QIND25A .

-- Lee Yang Lawrence Berkeley National Lab 1 Cyclotron Road, MS 7H210 Berkeley, California 97320 office:(510)486-7320 fax:(510) 486-4633

MarkRivers commented 4 years ago

envPaths is not auto generated. I created it somehow manually.

That is not the right way to do it. If you run "make" at the top-level of ADSpinnaker it should rebuild envPaths for you. You might need to make sure that iocSpinnaker/Makefile has the correct ARCH for your build, e.g. linux-x86_64 or whatever you are using.

Should I also add something in RELEASE_PRODS.local for ADSpinnaker?

No, you just copy EXAMPLE_RELEASE_PRODS.local to RELEASE_PRODS.local and edit it.

LeeYangLBLBCS commented 4 years ago

problem solved. Makefile is left with: ARCH = linux-x86_64-ub18 from the original download. (Probably from Github). No envPaths is generated as a result.

On Tue, Aug 25, 2020 at 7:43 AM Mark Rivers notifications@github.com wrote:

envPaths is not auto generated. I created it somehow manually.

That is not the right way to do it. If you run "make" at the top-level of ADSpinnaker it should rebuild envPaths for you. You might need to make sure that iocSpinnaker/Makefile has the correct ARCH for your build, e.g. linux-x86_64 or whatever you are using.

Should I also add something in RELEASE_PRODS.local for ADSpinnaker?

No, you just copy EXAMPLE_RELEASE_PRODS.local to RELEASE_PRODS.local and edit it.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/areaDetector/ADSpinnaker/issues/6#issuecomment-680068932, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADSYGNF5RQG2UGCFCRFJMOLSCPE2XANCNFSM4QIND25A .

-- Lee Yang Lawrence Berkeley National Lab 1 Cyclotron Road, MS 7H210 Berkeley, California 97320 office:(510)486-7320 fax:(510) 486-4633

MarkRivers commented 4 years ago

ARCH = linux-x86_64-ub18

Yes, that is the setting for me. I build linux-x86_64 on a Centos 7 system, which cannot build ADSpinnaker because gcc is too old. So I have to give my Ubuntu 18 system a different EPICS_HOST_ARCH name.

MarkRivers commented 4 years ago

Is everything working OK so I can close this issue?

LeeYangLBLBCS commented 4 years ago

Yes. Thanks for the help.

On Thu, Aug 27, 2020, 6:41 AM Mark Rivers notifications@github.com wrote:

Is everything working OK so I can close this issue?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/areaDetector/ADSpinnaker/issues/6#issuecomment-681956320, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADSYGNALDQLUZL6VQL64CI3SCZPBRANCNFSM4QIND25A .