search

JvanKatwijk / qt-dab

Qt-DAB, a general software DAB (DAB+) decoder with a (slight) focus on showing the signal
http://www.sdr-j.tk
GNU General Public License v2.0
300 stars 62 forks source link

RAW-recordings XML-Header stuff #329

Closed Drehrumbum closed 1 month ago

Drehrumbum commented 4 months ago

Hello Jan!

I've checked out the latest Qt-DAB version for Windows over the last weekend and currently I take a closer look into the raw-recordings. Something went wrong...

Example:

Filename: Generic RTL2832U OEM-12A-So-Jul-7-2024-10-59-40.uff Filesize: 510861753 Bytes Recorder: Qt-DAB (Qt-DAB-6.X with RTLSDR-V4) End of recording (FILETIME): 2024-07-07 09:01:47.878 (last write access to the file on disk)

The XML-Header of the file looks like this:

<?xml version="1.0" encoding="utf-8"?>
<SDR>
 <Recorder Version="Qt-DAB-6.X with RTLSDR-V4" Name="Qt-DAB"/>
 <Device Model="Generic RTL2832U OEM" Name="rtlsdr"/>
 <Time Unit="UTC" Value="2024-07-07 09:01:47"/>
 <Sample>
  <Samplerate Unit="Hz" Value="2048000"/>
  <Channels Container="uint8" Ordering="LSB" Bits="8">
   <Channel Value="I"/>
   <Channel Value="Q"/>
  </Channels>
 </Sample>
 <Datablocks>
  <Datablock Unit="Channel" Number="1" Count="510859046">
   <Frequency Unit="kHz" Value="223936"/>
   <Modulation Value="DAB"/>
  </Datablock>
 </Datablocks>
</SDR>

Well, the first thing is the recording-time inside of the header. It shows the time when the file was finished at the end. The next point is the "Count" value (510859046). Because the file-offset of the first sample is 5000, the file-size should be 510859046 + 5000 = 510864046 bytes, but we only have a size of 510861753 bytes. The file is too short and has a odd size, too. Last but not least - the "File-Save Dialog" takes time, so there is a time lag between the time inside of the filename and the time the recording really starts when the user selects another directory. I think it's better to start the recording in a fixed directory without further user-action - just one click (like QIRX does).

BFN Heiko

Clipboard01

JvanKatwijk commented 4 months ago

Apology for the late reply, I ve been quite busy lately, but as soon I have some time I'll have a look

Op wo 10 jul 2024 om 22:58 schreef Drehrumbum @.***>:

Hello Jan!

I've checked out the latest Qt-DAB version for Windows over the last weekend and currently I take a closer look into the raw-recordings. Something went wrong...

Example:

Filename: Generic RTL2832U OEM-12A-So-Jul-7-2024-10-59-40.uff Filesize: 510861753 Bytes Recorder: Qt-DAB (Qt-DAB-6.X with RTLSDR-V4) End of recording (FILETIME): 2024-07-07 09:01:47.878 (last write access to the file on disk)

The XML-Header of the file looks like this:

<?xml version="1.0" encoding="utf-8"?>

Well, the first thing is the recording-time inside of the header. It shows the time when the file was finished at the end. The next point is the "Count" value (510859046). Because the file-offset of the first sample is 5000, the file-size should be 510859046 + 5000 = 510864046 bytes, but we only have a size of 510861753 bytes. The file is too short and has a odd size, too. Last but not least - the "File-Save Dialog" takes time, so there is a time lag between the time inside of the filename and the time the recording really starts when the user selects another directory. I think it's better to start the recording in a fixed directory without further user-action - just one click (like QIRX does).

BFN Heiko

Clipboard01.png (view on web) https://github.com/JvanKatwijk/qt-dab/assets/86889245/ff17cc31-0e1a-4baf-932f-ec4e8812b2a0

— Reply to this email directly, view it on GitHub https://github.com/JvanKatwijk/qt-dab/issues/329, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCPHQEYRLZB7ONGUR7UDYLZLWOANAVCNFSM6AAAAABKVVOCUKVHI2DSMVQWIX3LMV43ASLTON2WKOZSGQYDCNZSGAZDSMQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- Jan van Katwijk

JvanKatwijk commented 4 months ago

I had a (very) quick look a. the time is set at the start of creating the ml text, to me it seems rather arbitrarily whether it is the time at the start of the recording of at the end, if there are good arguments to take the time at the start, I'll change it no problem b. the count is definitely not the file size, it is the size (supposed to) tell the number of data elements c. you are absolutely right that the count can be over optimistic, I'll have it corrected

Op di 16 jul 2024 om 14:49 schreef jan van katwijk @.***>:

Apology for the late reply, I ve been quite busy lately, but as soon I have some time I'll have a look

Op wo 10 jul 2024 om 22:58 schreef Drehrumbum @.***>:

Hello Jan!

I've checked out the latest Qt-DAB version for Windows over the last weekend and currently I take a closer look into the raw-recordings. Something went wrong...

Example:

Filename: Generic RTL2832U OEM-12A-So-Jul-7-2024-10-59-40.uff Filesize: 510861753 Bytes Recorder: Qt-DAB (Qt-DAB-6.X with RTLSDR-V4) End of recording (FILETIME): 2024-07-07 09:01:47.878 (last write access to the file on disk)

The XML-Header of the file looks like this:

<?xml version="1.0" encoding="utf-8"?>

Well, the first thing is the recording-time inside of the header. It shows the time when the file was finished at the end. The next point is the "Count" value (510859046). Because the file-offset of the first sample is 5000, the file-size should be 510859046 + 5000 = 510864046 bytes, but we only have a size of 510861753 bytes. The file is too short and has a odd size, too. Last but not least - the "File-Save Dialog" takes time, so there is a time lag between the time inside of the filename and the time the recording really starts when the user selects another directory. I think it's better to start the recording in a fixed directory without further user-action - just one click (like QIRX does).

BFN Heiko

Clipboard01.png (view on web) https://github.com/JvanKatwijk/qt-dab/assets/86889245/ff17cc31-0e1a-4baf-932f-ec4e8812b2a0

— Reply to this email directly, view it on GitHub https://github.com/JvanKatwijk/qt-dab/issues/329, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCPHQEYRLZB7ONGUR7UDYLZLWOANAVCNFSM6AAAAABKVVOCUKVHI2DSMVQWIX3LMV43ASLTON2WKOZSGQYDCNZSGAZDSMQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- Jan van Katwijk

-- Jan van Katwijk

JvanKatwijk commented 4 months ago

I uploaded sources and Windows (32) installers. One remark: note that the count counts the number of elements, i.e. for dabsticks single bytes, but for the SDRplay devices int16 values The recorded time is the time of the start of the recording, and the count is now precisely counting the number of elements

Op di 16 jul 2024 om 15:13 schreef jan van katwijk @.***>:

I had a (very) quick look a. the time is set at the start of creating the ml text, to me it seems rather arbitrarily whether it is the time at the start of the recording of at the end, if there are good arguments to take the time at the start, I'll change it no problem b. the count is definitely not the file size, it is the size (supposed to) tell the number of data elements c. you are absolutely right that the count can be over optimistic, I'll have it corrected

Op di 16 jul 2024 om 14:49 schreef jan van katwijk @.***

:

Apology for the late reply, I ve been quite busy lately, but as soon I have some time I'll have a look

Op wo 10 jul 2024 om 22:58 schreef Drehrumbum @.***>:

Hello Jan!

I've checked out the latest Qt-DAB version for Windows over the last weekend and currently I take a closer look into the raw-recordings. Something went wrong...

Example:

Filename: Generic RTL2832U OEM-12A-So-Jul-7-2024-10-59-40.uff Filesize: 510861753 Bytes Recorder: Qt-DAB (Qt-DAB-6.X with RTLSDR-V4) End of recording (FILETIME): 2024-07-07 09:01:47.878 (last write access to the file on disk)

The XML-Header of the file looks like this:

<?xml version="1.0" encoding="utf-8"?>

Well, the first thing is the recording-time inside of the header. It shows the time when the file was finished at the end. The next point is the "Count" value (510859046). Because the file-offset of the first sample is 5000, the file-size should be 510859046 + 5000 = 510864046 bytes, but we only have a size of 510861753 bytes. The file is too short and has a odd size, too. Last but not least - the "File-Save Dialog" takes time, so there is a time lag between the time inside of the filename and the time the recording really starts when the user selects another directory. I think it's better to start the recording in a fixed directory without further user-action - just one click (like QIRX does).

BFN Heiko

Clipboard01.png (view on web) https://github.com/JvanKatwijk/qt-dab/assets/86889245/ff17cc31-0e1a-4baf-932f-ec4e8812b2a0

— Reply to this email directly, view it on GitHub https://github.com/JvanKatwijk/qt-dab/issues/329, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCPHQEYRLZB7ONGUR7UDYLZLWOANAVCNFSM6AAAAABKVVOCUKVHI2DSMVQWIX3LMV43ASLTON2WKOZSGQYDCNZSGAZDSMQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- Jan van Katwijk

-- Jan van Katwijk

-- Jan van Katwijk

Drehrumbum commented 4 months ago

Hello Jan,

a. the time is set at the start of creating the ml text, to me it seems rather arbitrarily whether it is the time at the start of the recording of at the end, if there are good arguments to take the time at the start, I'll change it no problem

One reason for introducing the XML header was to enable better interchangeability of the recordings of different SDR programs. In this respect, it would be good if all SDR programs agreed on which data is written to the headers.

It seems more logical to me if the time of the start of the recording is stored there. If one renames the file (for whatever reason), the time stamp is still retained. QIRX does it this way, AbracaDABra as well, if the user quickly responses to the file save dialog.

b. the count is definitely not the file size, it is the size (supposed to) tell the number of data elements

You are right, if we follow the XML header draft on Clem's website. I personally prefer the term "sample" and "number of samples", not "number of data elements".

Well, in the case of an RTLSDR dongle (two bytes per sample) and with a single datablock, this gives:

file_size = sample_file_offset + datablock_count

That's why I talked about the file size and wondered about the difference to the "Datablock Count" . In any case, the file size should be even for 8-bit IQ samples. In the example above, it was odd. This is not always the case and should be checked.

Ciao Heiko

Drehrumbum commented 4 months ago

Whoops, U R fast... ;)

THX, I will check the new version.