search

simonsobs / socs

Simons Observatory specific OCS agents.
BSD 2-Clause "Simplified" License
12 stars 12 forks source link

smurf_recorder index step size out of range #122

Closed jlashner closed 3 years ago

jlashner commented 3 years ago

When the smurf-recorder is monitoring a channel and there is a short data file, the record process crashes with this error: 

ocs-smurf-recorder-s2_1   | 2020-11-25T02:26:18+0000 Writing to file /data/timestreams/16062/crate1slot2/1606271178_000.g3
ocs-smurf-recorder-s2_1   | FATAL (G3Timestream): Step index 19 out of range (G3Timestream.cxx:824 in G3TimestreamPtr {anonymous}::G3Timestream_getslice(const G3Timestream&, boost::python::slice))
ocs-smurf-recorder-s2_1   | 2020-11-25T02:26:36+0000 Unable to format event {'log_logger': <Logger 'ocs.ocs_agent.OCSAgent'>, 'log_level': <LogLevel=info>, 'log_namespace': 'ocs.ocs_agent.OCSAgent', 'log_source': None, 'log_format': "record:4 Crash in thread: [Failure instance: Traceback: <class 'RuntimeError'>: Step index 19 out of range (in G3TimestreamPtr {anonymous}::G3Timestream_getslice(const G3Timestream&, boost::python::slice))\n/usr/lib/python3.6/threading.py:916:_bootstrap_inner\n/usr/lib/python3.6/threading.py:864:run\n/usr/local/lib/python3.6/dist-packages/twisted/_threads/_threadworker.py:46:work\n/usr/local/lib/python3.6/dist-packages/twisted/_threads/_team.py:190:doWork\n--- <exception caught here> ---\n/usr/local/lib/python3.6/dist-packages/twisted/python/threadpool.py:250:inContext\n/usr/local/lib/python3.6/dist-packages/twisted/python/threadpool.py:266:<lambda>\n/usr/local/lib/python3.6/dist-packages/twisted/python/context.py:122:callWithContext\n/usr/local/lib/python3.6/dist-packages/twisted/python/context.py:85:callWithContext\nsmurf_recorder.py:142:start_record\n/app/socs/socs/agent/smurf_recorder.py:442:run\n/app/socs/socs/agent/smurf_recorder.py:473:read_stream_data\n/app_lib/spt3g_software/build/spt3g/core/timestreamextensions.py:13:G3Timestream_getitem\n]", 'log_time': 1606271196.0203595}: 'anonymous'.

Digging into spt3g, this points to this segment:

  808     if (step >= a.size() || step <= 0)
  809         log_fatal("Step index %d out of range", step);

Apparently Timestream slicing does not behave exactly the same as python list slicing, since in ordinary python lists you're allowed to slice with step sizes larger than the length of this list.

This should be an easy fix.

mhasself commented 3 years ago

It also forbids v[:n] if n > len(v). It seems odd for slicing to work differently than Python... (though there is some sense in forbidding step<0 here). Shall we raise/fix in spt3g_software?

jlashner commented 3 years ago

I submitted an issue to the spt3g_software repo: https://github.com/CMB-S4/spt3g_software/issues/45, so I'm going to close this one.