These changes add a sequence number to the currently-unused top 6 bits of the sample values. The numbers are added immediately after sampling, then checked and removed by DomesdayDuplicator when the data is written to disk, so it can detect samples being dropped at any point in the transfer process.
The length of the test data sequence is also changed so that different transfers/buffers will contain different data in test mode, which makes it possible to detect some faults that couldn't be detected before (e.g. DomesdayDuplicator dropping or reordering buffers).
See below for some new text for the FPGA firmware documentation, with rationale for the numbers chosen. I used this script to experiment with different sequence lengths (and this test program might also be useful in the future as an end-to-end check).
DomesdayDuplicator and dddutil should still work correctly with existing FPGA firmware, although I haven't tested this extensively!
At present, DomesdayDuplicator stops the capture when missing samples are detected. It would be possible to make it insert an appropriate amount of padding and then continue instead - this probably isn't very useful at the moment, but if you were doing a capture that you only had one attempt at (say, a disintegrating videotape) then it would be worth implementing...
Fixes #107. Fixes #115.
Documentation
The data generator module is responsible for generating data either from the ADC or (if in test mode) internally. When in test mode the generator outputs a repeating sequence of 10-bit numbers, 0 to 1020 inclusive.
The data generator also inserts a sequence number into the top 6 bits of each sample, so that the DomesdayDuplicator application can detect missing samples. The sequence numbers count repeatedly from 0 to 62 inclusive, incrementing every 65536 samples.
Data from the ADC is read on the positive edge of the ADC clock and passed to the FIFO buffer.
The lengths of the test sequence (1021) and the sequence number sequence (63) were chosen in order to maximise the length of time before a USB transfer has the same contents as a previous transfer in test mode (about 210s). The number of samples per sequence number was chosen to allow a length of blocks of missing samples up to 0.1s to be detected correctly; experimentation on a machine with an early USB3 controller showed maximum dropouts of about 0.01s under artifically heavy CPU load, so this gives some additional margin.
(In versions of the firmware before June 2022, there were no sequence numbers, and the test sequence ran from 0 to 1023.)
These changes add a sequence number to the currently-unused top 6 bits of the sample values. The numbers are added immediately after sampling, then checked and removed by DomesdayDuplicator when the data is written to disk, so it can detect samples being dropped at any point in the transfer process.
The length of the test data sequence is also changed so that different transfers/buffers will contain different data in test mode, which makes it possible to detect some faults that couldn't be detected before (e.g. DomesdayDuplicator dropping or reordering buffers).
See below for some new text for the FPGA firmware documentation, with rationale for the numbers chosen. I used this script to experiment with different sequence lengths (and this test program might also be useful in the future as an end-to-end check).
DomesdayDuplicator and dddutil should still work correctly with existing FPGA firmware, although I haven't tested this extensively!
At present, DomesdayDuplicator stops the capture when missing samples are detected. It would be possible to make it insert an appropriate amount of padding and then continue instead - this probably isn't very useful at the moment, but if you were doing a capture that you only had one attempt at (say, a disintegrating videotape) then it would be worth implementing...
Fixes #107. Fixes #115.
Documentation
The data generator module is responsible for generating data either from the ADC or (if in test mode) internally. When in test mode the generator outputs a repeating sequence of 10-bit numbers, 0 to 1020 inclusive.
The data generator also inserts a sequence number into the top 6 bits of each sample, so that the DomesdayDuplicator application can detect missing samples. The sequence numbers count repeatedly from 0 to 62 inclusive, incrementing every 65536 samples.
Data from the ADC is read on the positive edge of the ADC clock and passed to the FIFO buffer.
The lengths of the test sequence (1021) and the sequence number sequence (63) were chosen in order to maximise the length of time before a USB transfer has the same contents as a previous transfer in test mode (about 210s). The number of samples per sequence number was chosen to allow a length of blocks of missing samples up to 0.1s to be detected correctly; experimentation on a machine with an early USB3 controller showed maximum dropouts of about 0.01s under artifically heavy CPU load, so this gives some additional margin.
(In versions of the firmware before June 2022, there were no sequence numbers, and the test sequence ran from 0 to 1023.)