position of dropped_samples in stream

[sivantoledo]

The airspy_transfer struct that is passed to the callback function (the one specified in airspy_start_rx) contains a dropped_samples field, which I assume is the number of packets dropped because of USB performance issues or lack of space in buffers etc. The struct also points to an array with samples.

Do the provided samples follow the dropped ones? That is, if the callback is called with dropped_samples=10 and sample_count=17, do the 17 samples that do exist follow (in time) 10 samples that were lost?

I assume that this is the case but it would be good to get confirmation. I am trying to get the samples aligned in time for a ToA application.

Thanks, Sivan Toledo

[bvernoux]

For your use case you shall use this firmware https://github.com/airspy/airspyone_firmware/tree/timestamp (branch timestamp) It is better to know accurately which frames and how many frames are lost/dropped with the MCU timestamp Note1: Such special firmware requires some changes in airspyone_host tools / DLL to manage it correctly and such code shall be written

Contributions are welcome but it will requires also to merge the airspyone_firmware branch timestamp to latest master too

[sivantoledo]

I see that Benjamin marked this as a completed enhancement. Does this mean that this is now part of the official firmware? If so, have the host tools and DLL been updated?

Thanks, Sivan

[bvernoux]

It is answered but you have never provided any feedback The only solution available is to do what is described

For your use case you shall use this firmware https://github.com/airspy/airspyone_firmware/tree/timestamp (branch timestamp)
It is better to know accurately which frames and how many frames are lost/dropped with the MCU timestamp
Note1: Such special firmware requires some changes in airspyone_host tools / DLL to manage it correctly and such code shall be written

Contributions are welcome but it will requires also to merge the airspyone_firmware branch timestamp to latest master too

[sivantoledo]

Thanks a lot Benjamin.

Can you please explain what added features the timestamp branch has? I didn't see any documentation. I am asking because all I need is to know how many samples were dropped before those delivered to the callback of airspy_start_rx. I do not actually need time stamps.

To clarify further: When samples start streaming, my code decides on a time stamp for the first sample, and from then on, the code just counts samples and uses the start time plus sample count divided by sample rate to determine the time of each sample. This has been working for me for years with USRPs. There I use the PPS to set the time stamp on the first sample, but I do not need this level of accuracy. With the Airspy R2, I use an approximate time stamp from a GPS receiver that is part of a GPS-Disciplined oscillator, and this is good enough. Becuase the Airspy R2 uses an external clock that's locked to GPS, counting samples generates very accurate time stamps (that is, all with the same absolute initial error, which is okay for me).

So if the main branch of the firmware generates USB packets that have a sequence number so the host side can determine reliably that one or more packets have been dropped or delivered out of order, that is good enough for me, and I'll stay with the main branch. If the dropped_samples field of the transfer structure delivered to the callback is valid, this is all I need. If the firmware generates sequence numbers but the host side does not translate them into correct dropped_samples field, I can try to add this code.

I hope that the question is now clearer.

Thanks, Sivan

[sivantoledo]

OK, I think that the extra bit of code in the timestamp branch indeed sends a frame number. The code I found in is airspy_m4.c, in the dma_isr,

if(use_packing == AIRSPY_PACKING_TIMESTAMP)
{
    uint32_t ofs = get_usb_buffer_offset_m4();
    uint32_t* hdr = (uint32_t*)(usb_bulk_buffer+ofs);
    uint32_t ads = LPC_ADCHS->STATUS0;
    LPC_ADCHS->CLR_STAT0 = ads;
    hdr[0]=framecounter|(ads<<16);
    set_usb_buffer_offset_m4( inc_mask_usb_buffer_offset_m4(ofs, USB_DATA_TRANSFER_SIZE_BYTE/2));
    framecounter=(framecounter+1)&4095;
} else if(use_packing == AIRSPY_PACKING_ON)
{
    set_usb_buffer_offset_m4( inc_mask_usb_buffer_offset_m4(get_usb_buffer_offset_m4(), USB_DATA_TRANSFER_SIZE_BYTE/2));
} else // default AIRSPY_PACKING_OFF
{
    set_usb_buffer_offset( inc_mask_usb_buffer_offset(get_usb_buffer_offset(), USB_DATA_TRANSFER_SIZE_BYTE) );
    signal_sev();
}

Is this the only difference? It is still hard for me to fully understand the structure of the buffer that is transfered to the host. Is there any documentation for this?

Thansk Sivan

[bvernoux]

Yes it is the main difference with a 32bits data LPC_ADC status0 on 16 bits and a 16bits frame counter which is present on each frame receiver over USB (only for the specific mode AIRSPY_PACKING_TIMESTAMP) so each 4096 bytes (as each full frame contains 4096 bytes)

[sivantoledo]

Thanks Benjamin. Where do these 32-bit words live in the received USB buffer that libairspy processes in consumer_threadproc? Are they interleaved before/after every 4096 bytes? I can't see in the firmware that AIRSPY_PACKING_TIMESTAMP causes the length of the USB buffers to increase.

[bvernoux]

It is the 1st 32bits word received on each packet (4096 bytes) only available with AIRSPY_PACKING_TIMESTAMP and of course the data use packing (12bits stored) to have space for that extra 32bits word per packet.

[sivantoledo]

But does that mean that the 32-bit frame-counter and status word overwrites the first 4 bytes of the ADC's samples? I do not see that the buffer lengths or offsets are adjusted in any way. They seem to be exactctly the same as in the AIRSPY_PACKING_ON setting.

If this is the case, then when the buffer is processed in mainI(), the call to pack() would corrupt the frame number, because it unpacks the entire buffer.

[bvernoux]

Such code has been provided by an anonymous customer and it is provided as this as so far there was no any contributor helping to integrate it in airspy host tools. Anyone is welcome to integrate it with airspy host tools and provide pull request. For information other alternative is to use standard airspy firmware and enable statistics to be sure no any frames are lost (it heavily depends on the PC/OS/USB2 HS controller) see also https://github.com/airspy/airspyone_host/wiki/Troubleshooting

[sivantoledo]

Thanks Benjamin. I ran my experiment on Linux, and I didn't see any dropped packets, so I think we'll try to use the R2 in our system. I will also try to add (and contribute) the missing code to reliably detect any missing packet.

[bvernoux]

Thanks Benjamin. I ran my experiment on Linux, and I didn't see any dropped packets, so I think we'll try to use the R2 in our system. I will also try to add (and contribute) the missing code to reliably detect any missing packet.

Thanks for all contributions are very welcome