Guidance needed to get Jungfrau simulator to 2,000 frames / second

[graeme-winter]

*Distribution:

RHEL8

*Detector type:

Jungfrau (simulator)

*Software Package Version:

Current main

Priority:

Low

*Describe the bug

Configuring the detector simulator I seem unable to go above ~1,000 frames / second on a single module config, even with two network channels configured:

# jcu01
hostname 192.168.200.200+

# gh01
rx_hostname 192.168.200.201+

0:udp_srcip 192.168.200.200
0:udp_dstip 192.168.200.201
0:udp_dstport 30001

0:udp_srcip2 192.168.200.200
0:udp_dstip2 192.168.200.201
0:udp_dstport2 30002

numinterfaces 2

rx_zmqfreq 1
rx_zmqhwm 50000
rx_zmqstream 1

temp_control 1
temp_threshold 55

# exposure time, cycle time to use
exptime 0.0005
period 0.0005
frames 1000

# enable both network channels
speed full_speed

# save binary data (112 byte header / frame)
fformat binary
fwrite 1
fpath /dev/shm/gw

powerchip 1

I would hope the above config allows ~2,000 frames / second as it does for a real instrument, however:

bl24i-sc-jcu01 bin :) $ ./sls_detector_put exptime 0.0000
exptime 0.0000
bl24i-sc-jcu01 bin :) $ ./sls_detector_put period 0.0000
period 0.0000
bl24i-sc-jcu01 bin :) $ time ./sls_detector_acquire
- 06:00:55.494 INFO: Type 'q' and hit enter to stop acquisition
    100.00 %
Acquired [[10000, 10000]]

real    0m10.085s
user    0m0.012s
sys 0m0.048s

It seems to top out around 1,000 - is this expected?

Expected behavior

To generate 10,000 frames in ~5s (or less with the time and period set to 0)

To Reproduce

Take the config above, run 1x jungfrau simulator, 1x frame receiver, go.

Screenshots

Not needed, included text above

Additional context

Trying to get a good understanding of the simulator capabilities: I appreciate that running 2,000 frames / second is hard

[graeme-winter]

This morning I will look at the timestamps in the header to plot the actual rate

[graeme-winter]

OK, I suspect that this is a computational side-effect of what the program is doing: if I remove the block which sets up some gain values etc.

i.e.

ws448 slsDetectorPackage :) [main] $ git diff
diff --git a/slsDetectorServers/jungfrauDetectorServer/slsDetectorFunctionList.c b/slsDetectorServers/jungfrauDetectorServer/slsDetectorFunctionList.c
index 73d7ec3f2..a323ddf5b 100644
--- a/slsDetectorServers/jungfrauDetectorServer/slsDetectorFunctionList.c
+++ b/slsDetectorServers/jungfrauDetectorServer/slsDetectorFunctionList.c
@@ -2770,7 +2770,7 @@ void *start_timer(void *arg) {
             usleep(expUs);

             // change gain and data for every frame
-            {
+            if (0) {
                 const int npixels = (NCHAN * NCHIP);
                 for (int i = 0; i < npixels; ++i) {
                     int gainVal = 0;

I get 10,000 frames in ~5.7s - far closer to the desired time. I suspect it will be a little bit of work to tune this up to get to 0.5ms / frame (particularly after extending to 1M, to saturate the 40Gb/s link) but I can have a look at this.

[graeme-winter]

I am tempted to think that pre-creating the module data & sending the same every time, with the appropriate header tweaks, could be a useful way to go. For clarity: I am trying to figure out a synthetic method for testing a back-end acquisition system.

[thattil]

Hi @graeme-winter, yes this simulator was intended more for API integration rather than stressing the back end. But yes, we could have a look at it soon after the 9.0.0 release. And yes, your suggestion makes sense.

[graeme-winter]

Not recommending the solutions, but with some nasty hacking in

https://github.com/graeme-winter/slsDetectorPackage/tree/jungfrau-simulator-turbo

get to a little over 2kHz so it can be done

I apprecate that stress testing the acquisition system is not what this is for: maybe we should come out with a tool designed for that? Meanwhile I have what I need to continue work with the nasty hacks above

bl24i-sc-jcu01 bin :) $ time ./sls_detector_acquire
- 10:22:07.381 INFO: Type 'q' and hit enter to stop acquisition
    100.00 %%
Acquired [[100000, 100000], [100000, 100000]]

real    0m48.619s
user    0m0.050s
sys 0m0.155s