This PR implements a critical bug fix in the core code which fixes external downsampling using a bitmask. This PR was tested by Shawn in lab1 on timing test setup. Included in this PR are all of the configuration files used to verify external downsampling by bitmask was working using the lab1 system. This PR finishes the functional implementation of ESCRYODET-472, the SLAC JIRA ticket for adding external trigger to the pysmurf downsampler. The fix required to make this work is described below, and was successfully tested with a full timing system at SLAC.
In addition to the bug fix for downsampling on external timing system trigger bits, this PR includes some other fixes and new functions to help with testing at SAT including:
Fix to logic bug in the set_downsample_mode function.
New shell_command function for running command line programs, like the programs that can query FRU info for the SMuRF boards inside the pysmurf docker.
New get_fru_info function, which uses the new shell_command function to query FRU information for SMuRF boards via the shell cba_rtm_init, cba_amc_init, and cba_fru_init programs.
New check_full_band_resp function which replicates the functionality of Shawn's full_band_response.py script, and adds features like registering plots and data with the SO publisher, and stamping the plots and data with the AMC and carrier serial numbers.
New functions get_amc_sn, get_carrier_sn, and get_rtm_sn which can use either the new get_fru_info command or the ATCA monitor EPICS server to poll SMuRF board serial numbers.
Bug fix for downsampling on external timing system trigger bits
This PR fixes a bug in pysmurf release v7.0.1 in the SmurfProcessor when downsampling in external mode on external timing system trigger bits ; the logic here which determined which samples to keep is incompatible with the implementation of the fixed rate trigger bits in the SLAC timing system:
Here's an example of how this works with the SLAC implementation of the fixed rate downsampling bits ; when streaming data to disk with the timing system, the timing system increments counters in a 32 bit word (the "Averaging reset bits" starting at word 10 in the header, offset byte 80). Each counter (which is implemented) increments at a different rate. Here are the rates, starting from LSB and ending on MSB;
bit 0 : 15000 Hz (=LSB) FIXEDDIV
bit 1 : 12000 Hz FIXEDDIV
bit 2 : 10000 Hz FIXEDDIV
bit 3 : 8000 Hz FIXEDDIV
bit 4 : 6000 Hz FIXEDDIV
bit 5 : 5000 Hz FIXEDDIV
bit 6 : 4000 Hz FIXEDDIV
bit 7 : 3000 Hz FIXEDDIV
bit 8 : 2000 Hz FIXEDDIV
bit 9 : 1000 Hz FIXEDDIV
bit 10 : 2000 Hz (asserts every 2x 4kHz clock cycles)
bit 11 : 1000 Hz (asserts every 4x 4kHz clock cycles)
bit 12 : 500 Hz (asserts every 8x 4kHz clock cycles)
bit 13 : 250 Hz (asserts every 16x 4kHz clock cycles)
bit 14 : 125 Hz (asserts every 32x 4kHz clock cycles)
bit 15 : 62.5 Hz (asserts every 64x 4kHz clock cycles)
bit 16 : 31.25 Hz (asserts every 128x 4kHz clock cycles)
bit 17 : 15.625 Hz (asserts every 256x 4kHz clock cycles)
bit 18 : 1333.333... Hz (asserts every 3x 4kHz clock cycles)
bit 19 : 444.444... Hz (asserts every 9x 4kHz clock cycles)
bit 20 : 148.148... Hz (asserts every 27x 4kHz clock cycles)
bit 21 : 49.382... Hz (asserts every 81x 4kHz clock cycles)
bit 22 : 16.460... Hz (asserts every 243x 4kHz clock cycles)
bit 23 : 800 Hz (asserts every 5x 4kHz clock cycles)
bit 24 : 160 Hz (asserts every 25x 4kHz clock cycles)
bit 25 : 32 Hz (asserts every 125x 4kHz clock cycles)
bit 26 : never (always zero)
bit 27 : never (always zero)
bit 28 : never (always zero)
bit 29 : never (always zero)
bit 30 : never (always zero)
bit 31 : never (always zero) (=MSB)
where ... here just signifies there's more decimals, not necessarily repeating. For the most recent, recommended version of the SMuRF timing pattern generator software (which configures the timing system rates) v5.0.1, bits 26 through 31 are not used (and are always zero). The first 10 bits (bit 0 through 9 above) are the counters used for triggering the flux ramp. The remaining 16 bits, bit 10 through 25 above are the downsampling trigger bits. Note that the above rates for bits 10 through 25 are configured by the SMuRF timing pattern generator software, and these are the rates programmed by version v5.0.1. Future versions may use different rates; refer to SMuRF TPG-IOC Software Versions for specifics.
Downsampling using these triggers is accomplished by bit masking. In principle you can bit mask on any combination of the downsampling trigger bits and the flux ramp trigger bits, but the downsampling bits 10 through 25 were chosen by @mhasself to provide a good amount of coverage at the frequencies Simons Observatory will likely downsample to. I kludged a script from Matthew to compute the bitmask corresponding to the closest desired sample rate (after downsampling) here in this PR - https://github.com/slaclab/pysmurf/blob/23f5b8c74b673c982d6d03cdbda14a1d0463e240/scratch/shawn/dscounters/estimate_bitmask.py. Here's an example of how to use it (from inside a typical pysmurf sesson in ipython);
In [71]: run scratch/shawn/dscounters/estimate_bitmask.py 650
This target frequency cannot be achieved exactly.
Target: 650.0 Achieved: 666.6666666666666 [Hz]
Bit mask: b0000000100000001
Put this in Downsampler>ExternalBitmask: 263168
for this example the closest frequency found to the 650 Hz is 666.666(repeating) Hz, corresponding to a bitmask (only on the 16 downsampling bits) of 0b0000000100000001 = 257. One thing the script assumes is that you are running with a 4kHz flux ramp rate. The last line of the output above tells you what to set the SmurfProcessor:Downsampler:ExternalBitmask to in order to achieve this rate; it's computed simply by shifting 257 to the left by 10 bits to account for the fact that the 32 bit "Averaging reset bits" word in the SMuRF data file headers has the flux ramp counters as the first 10 bits. So 257<<10 = 263168.
For the specific example of flux ramping at 4kHz and downsampling to 666.666(repeating) using the above bitmask, here's how to configure the downsampler in pysmurf:
After taking data in this mode, the "Averaging reset bits" word can be inspected by loading the data like this
t, d, m, h = S.read_stream_data(filename, return_header=True)
and inspecting the h['reset_bits'] field. To look at a particular bit, for instance downsampling bit 15 which asserts at 62.5 Hz, you can e.g. plot (h['reset_bits']>>15)&1. The way the bits work is that they toggle to 1 at their programmed rate, and otherwise are zero. Here's a random capture of h['reset_bits'] at 4kHz showing how all 32 bits evolve over approximately twenty 4kHz samples:
The righthand column is (h['reset_bits']&bitmask)==bitmask, the output of the new conditional logic implemented in this PR in SmurfProcessor here that determines, when True, which sample to keep, accomplishing the desired downsampling. E.g. here the logic evaluates as True every six 4 kHz samples, or at 666.666(repeating) Hz.
I have successfully tested this logic in a dev version of the smurf docker with a full timing system, SMuRF timing pattern generator software version v5.0.1, and timing carrier firmware version 22d2722 for several different downsampling rates.
Issue
This PR implements a critical bug fix in the core code which fixes external downsampling using a bitmask. This PR was tested by Shawn in lab1 on timing test setup. Included in this PR are all of the configuration files used to verify external downsampling by bitmask was working using the lab1 system. This PR finishes the functional implementation of ESCRYODET-472, the SLAC JIRA ticket for adding external trigger to the pysmurf downsampler. The fix required to make this work is described below, and was successfully tested with a full timing system at SLAC.
In addition to the bug fix for downsampling on external timing system trigger bits, this PR includes some other fixes and new functions to help with testing at SAT including:
set_downsample_modefunction.shell_commandfunction for running command line programs, like the programs that can query FRU info for the SMuRF boards inside thepysmurfdocker.get_fru_infofunction, which uses the newshell_commandfunction to query FRU information for SMuRF boards via the shellcba_rtm_init,cba_amc_init, andcba_fru_initprograms.check_full_band_respfunction which replicates the functionality of Shawn's full_band_response.py script, and adds features like registering plots and data with the SO publisher, and stamping the plots and data with the AMC and carrier serial numbers.get_amc_sn,get_carrier_sn, andget_rtm_snwhich can use either the newget_fru_infocommand or the ATCA monitor EPICS server to poll SMuRF board serial numbers.Bug fix for downsampling on external timing system trigger bits
This PR fixes a bug in pysmurf release v7.0.1 in the SmurfProcessor when downsampling in external mode on external timing system trigger bits ; the logic here which determined which samples to keep is incompatible with the implementation of the fixed rate trigger bits in the SLAC timing system:
https://github.com/slaclab/pysmurf/blob/b266b5141ee26ef618029126e1ba2b8ecfd76be6/src/smurf/core/processors/SmurfProcessor.cpp#L741
Here's an example of how this works with the SLAC implementation of the fixed rate downsampling bits ; when streaming data to disk with the timing system, the timing system increments counters in a 32 bit word (the "Averaging reset bits" starting at word 10 in the header, offset byte 80). Each counter (which is implemented) increments at a different rate. Here are the rates, starting from LSB and ending on MSB;
where
...here just signifies there's more decimals, not necessarily repeating. For the most recent, recommended version of the SMuRF timing pattern generator software (which configures the timing system rates) v5.0.1, bits 26 through 31 are not used (and are always zero). The first 10 bits (bit 0 through 9 above) are the counters used for triggering the flux ramp. The remaining 16 bits, bit 10 through 25 above are the downsampling trigger bits. Note that the above rates for bits 10 through 25 are configured by the SMuRF timing pattern generator software, and these are the rates programmed by version v5.0.1. Future versions may use different rates; refer to SMuRF TPG-IOC Software Versions for specifics.Downsampling using these triggers is accomplished by bit masking. In principle you can bit mask on any combination of the downsampling trigger bits and the flux ramp trigger bits, but the downsampling bits 10 through 25 were chosen by @mhasself to provide a good amount of coverage at the frequencies Simons Observatory will likely downsample to. I kludged a script from Matthew to compute the bitmask corresponding to the closest desired sample rate (after downsampling) here in this PR - https://github.com/slaclab/pysmurf/blob/23f5b8c74b673c982d6d03cdbda14a1d0463e240/scratch/shawn/dscounters/estimate_bitmask.py. Here's an example of how to use it (from inside a typical pysmurf sesson in ipython);
for this example the closest frequency found to the 650 Hz is 666.666(repeating) Hz, corresponding to a bitmask (only on the 16 downsampling bits) of 0b0000000100000001 = 257. One thing the script assumes is that you are running with a 4kHz flux ramp rate. The last line of the output above tells you what to set the SmurfProcessor:Downsampler:ExternalBitmask to in order to achieve this rate; it's computed simply by shifting 257 to the left by 10 bits to account for the fact that the 32 bit "Averaging reset bits" word in the SMuRF data file headers has the flux ramp counters as the first 10 bits. So 257<<10 = 263168.
For the specific example of flux ramping at 4kHz and downsampling to 666.666(repeating) using the above bitmask, here's how to configure the downsampler in pysmurf:
After taking data in this mode, the "Averaging reset bits" word can be inspected by loading the data like this
and inspecting the
h['reset_bits']field. To look at a particular bit, for instance downsampling bit 15 which asserts at 62.5 Hz, you can e.g. plot(h['reset_bits']>>15)&1. The way the bits work is that they toggle to 1 at their programmed rate, and otherwise are zero. Here's a random capture ofh['reset_bits']at 4kHz showing how all 32 bits evolve over approximately twenty 4kHz samples:The righthand column is
(h['reset_bits']&bitmask)==bitmask, the output of the new conditional logic implemented in this PR in SmurfProcessor here that determines, when True, which sample to keep, accomplishing the desired downsampling. E.g. here the logic evaluates as True every six 4 kHz samples, or at 666.666(repeating) Hz.I have successfully tested this logic in a dev version of the smurf docker with a full timing system, SMuRF timing pattern generator software version v5.0.1, and timing carrier firmware version 22d2722 for several different downsampling rates.