Unable to suppress checksum creation when importing metadata from SigMF Archive

[bhorsfield]

Hi All,

I have a set of SigMF Archive files that I created with the skip_checksum flag set to True. I have inspected these files using a tar file viewer, and confirmed that there is no core:sha512 field in the Global object. So far, so good.

However, whenever I import the metadata object from one of these files in Python, the core:sha512 field is always populated, despite the fact that I have specified skip_checksum = True in the fromfile() function call.

A typical code example is as follows:

# Step 1: Create new SigMF archive
meta = SigMFFile(...)   # populate metadata object here
meta.set_data_file('logfile.sigmf-data', skip_checksum=True)
meta.archive('logfile.sigmf')

# Step 2: Read metadata back in from SigMF archive and print SHA512 hash key
meta_check = sigmffile.fromfile('logfile.sigmf', skip_checksum=True)
print('Checksum: ', meta_check.get_global_field(SigMFFile.HASH_KEY))

The expected output from the above code snippet is Checksum: None. However, in practice I find the core:sha512 field is always populated. A typical output is:

Checksum: e1c87d27930a6824ad192c62d5536abdc35c49b88d0dedd1c0c21e7418db18b7b52dae2fc64b4dd31f1bf5b640aaa6ee3c009513589eb04b99dab4144270a55c

My suspicion is that there is a bug in the SigMFArchiveReader() class, as this problem only occurs with .sigmf files. It does not occur with .sigmf-meta files.

Details of my system configuration are as follows:

• HP Omen laptop with i7 processor & 32 GB of RAM
• Ubuntu 22.04
• Python 3.10.12
• SigMF version 1.2.1

Can anyone tell me what the root cause of this problem might be?

Thanks, Brendan.

[Teque5]

Someone more familiar with the SigMFArchive code will look into this.

[gmabey]

Hi @bhorsfield --

Unfortunately I'm not able to reproduce your report in 1.2.2. What I did was to tar-extract the ri16_le_1.sigmf file from the SigMF repo, edited the .sigmf-meta file such that I removed the line, and then tar-ed it back up. Then:

In [1]: import sigmf

In [2]: a = sigmf.sigmffile.fromfile('no_checksum/ri16_le_1.sigmf', skip_checksum=True)

In [3]: a
Out[3]: 
SigMFFile({
    "global": {
        "core:author": "Glen M",
        "core:datatype": "ri16_le",
        "core:license": "https://creativecommons.org/licenses/by-sa/4.0/",
        "core:num_channels": 1,
        "core:sample_rate": 48000,
        "core:version": "1.2.0"
    },
    "captures": [
        {
            "core:datetime": "2021-06-18T23:17:51.163959Z",
            "core:sample_start": 0
        }
    ],
    "annotations": []
})

In [4]: a.get_global_field(sigmf.SigMFFile.HASH_KEY)

In [5]: repr(a.get_global_field(sigmf.SigMFFile.HASH_KEY))
Out[5]: 'None'

In [6]: sigmf.__version__
Out[6]: '1.2.2'

Do you have any further hints for me on how to reproduce this issue?

[bhorsfield]

Hi @gmabey,

Thanks for getting back to me.

I have tried the series of commands given in your reply on one of my own SigMF archives (using ver 1.2.1). However, when I import the archive using the sigmf.sigmffile.fromfile() method, I still find that the checksum field is populated:

a.get_global_field(sigmf.SigMFFile.HASH_KEY)
'e631f97f402640388f02db6e0b78fc339b62667c01717174af4dca3c93f0e90103afe24eabb197950987803b998b12cd586cf8386435c9bb6e5070c9872a7fe6'

I have inspected my sigmf archive with a tar file viewer, and confirmed that it contains no checksum field.

Presumably the SigMFArchiveReader class has been changed between version 1.2.1 and 1.2.2. Although 1.2.2 has not been officially released yet (and therefore has no release notes), I see that the last commit message is "archive bugfixes". This sounds promising!

Do you have any idea when 1.2.2 will be officially released?

Cheers, Brendan.

[gmabey]

I don't know when 1.2.2 will be released, but if you have the most basic of git skills you can try out the main branch of sigmf-python by cloning the repo and then starting your python interpreter from within the directory that is created (which will find sigmf/ in that directory). Can you please give that a shot and let us know how it goes?

[Teque5]

I'll spin a new patch release before the GNU Radio Conference next week.