create ROACH simulation capabilities in psyllid

[laroque]

This requires three new components:

• [x] An egg_reader, which is configured with an egg file path, and when told to execute will look through the slices and emit time_data objects (which could be sent to a egg streaming time writer)
• [x] an FFT, which takes time_data and computes frequency_data using FFTW3
• [ ] a packetizer, which takes the time_data and frequency_data and populates a memory_block (which could be sent to a tf_roach_receiver and sent through trigger or whatever else)
• [ ] a packet_sender_socket, which takes a memory_block and produces UDP packets on a network (to the remote receiver, should be indistinguishable from an actual roach)

future issue

We're in crunch time trying to push for Neutrinos, so I'm going to leave some outstanding items that I don't think matter yet... I hate this, but I don't have time to clean it up. I'm going to try and double document my sloppiness with 1) instances of //TODO in source/header files, and the following list of items. If the above are done then this list may be moved into another issue with a more clear name

• [x] the egg3_reader sets packet in batch and session both equal to the RecordID of the M3Record. This value is stored in a relatively small integer and wraps every 16 seconds. The packet in session should not, I believe, wrap in that way (if properly implemented).
• [ ] I don't have an easily available egg with multiple acquisitions to confirm that real acquisition breaks show up properly downstream.
• [ ] right now everything assumes that there is just one stream; would be better to be able to support multiple (ideally dynamically from the egg header)
• [ ] the device details are also in the header, would be nice to have the reader set those rather than having to manually transcribe (is this true? if we split and actually ship packets there's no way to do that)
• [ ] take some time to add prose documentation for midge, especially how the stream objects work, generators etc.

[laroque]

Item the first is close but there's some flow control issues. I'm about to leave for the holidays but should leave notes for myself for later... Also these should make it into the actual reST docs before I merge (and therefore before I forget).

The egg reader should start up in a paused state and then receive a "resume" signal later (this happens by dealing with instructions in a similar fashion to what is done in tf_roach_receiver). This means we then need to use psyllid's start_run via dripline request to cause it to start reading the egg file and streaming the data. This should resolve the fact that currently it segfaults because the egg writer isn't ready.

Further, the execute() function of egg3_reader should remain in the execution loop until it receives the pause instruction, not return when the file is done.

Note also: you can issue dripline commands to psyllid.run-daq-cmd.[stream].[node].[command]. The commands are implemented in [node]_binding::do_run_command(), which receives the [command] from the RKS as a string and implements whatever call(s) to the node instance. Also, The other commands available via dripline are in source/control/run_server.cc starting around line 123 (f_request_receiver->register_cmd_handler() calls).

[laroque]

Note to self:

dragonfly cmd -b rabbit_broker psyllid_reader.start-run filename=/tmp/foo2.egg duration=200

Or for several:

dragonfly cmd -b rabbit_broker psyllid_reader.start-run filenames='["/tmp/foo_t.egg", "/tmp/foo_f.egg"]'

(this may be excessive with quotes, I'd have to experiment, but it works)

[laroque]

note from @cclaessens:

The tf_roach_receiver has a mode to only output f-packets. This is used to build the frequency mask without populating the time output. I think that the frequency_transform class needs to support the same OP_CMD and behavior, @nsoblath is welcome to disagree with the approach.

[nsoblath]

Christine has a good point. The basic way that it's setup for tf_roach_receiver should be easy to copy for frequency_transform.

[laroque]

created new issues for outstanding items. Batch mode exists, including reading an egg file, doing an FFT, and passing everything off to the trigger. I'm closing this issue and we can focus on other details in the new one.