Decimated vs undecimated recordings

[StrandmonYellow]

Joulescope model

JS220

UI version

other

Your question

UI Version 1.1.10.

I want to do some calculations about the signals I recorded. I want to calculate the average power consumption of a signal whenever GPI[0] is High, and do the same for whenever GPI[0] is low. This could be done by using Python and CSV. However, If i have a ~251 second recording with a sample rate of 1Mhz, this export to CSV using the example python script takes a very long time.

So I tried to do it by reading the JLS files directly into python using the pyjls library. Here I can read the average power values without any problems. I can open a file and look at the number of samples in the recording for the power signal and the GPI[0] signal.

path = 'data\\data.jls'
with Reader(path) as r:    
    power = r.signal_lookup('power')
    rfenable = r.signal_lookup('gpi[0]')
    p_length = power.length
    rf_length = rfenable.length
    print(power)
    print(rfenable)
    print(f'length of Power samples is {p_length} samples')
    print(f'length of RF Enable samples is {rf_length} samples')

The output i get is what i expect:

SignalDef(signal_id=3, source_id=1, signal_type=0, data_type=8196, sample_rate=1000000, samples_per_data=8192, sample_decimate_factor=128, entries_per_summary=640, summary_decimate_factor=20, annotation_decimate_factor=100, utc_decimate_factor=100, sample_id_offset=0, name='power', units='W', length=251595035)
SignalDef(signal_id=5, source_id=1, signal_type=0, data_type=259, sample_rate=2000000, samples_per_data=65536, sample_decimate_factor=1024, entries_per_summary=1280, summary_decimate_factor=20, annotation_decimate_factor=100, utc_decimate_factor=100, sample_id_offset=0, name='gpi[0]', units='', length=503189504)
length of Power samples is 251595035 samples
length of RF Enable samples is 503189504 samples

To check if I am doing things correct, I can calculate the overall power of the complete recording to compare with the joulescope UI, and this returns almost the same value as the UI tells me. (I think the UI does some rounding before the calculations are being made vs when i do it with the raw values in python?)

avg_pow = r.fsr(power.signal_id, 0, p_length) 
print(f'The average power calculated with numpy is {round(np.mean(avg_pow) * 1000, 5)} mW')

I know that it is calculated over the complete dataset because with the IDE I am using, the length of the array is correct.

But now there is something I don't understand. When I do this for the GPI[0] signal, the output does not correspond to the number of samples I expect. For example, when I take the first 2000 samples, the output array is just 1/8th the length of what I expect.

rfenable_arr = r.fsr(rfenable.signal_id, 0, 2000)
print(len(rfenable_arr))

With output:

250

I can confirm this using the IDE:

I think this has something to do with the sample_decimate_factor of 128 for the power signal, and 1024 for the GPI[0] signal. However, the signalDef that gets returned states a sample frequency of 2Mhz which would correspond with the duration of the recording (~251 seconds) and the given length of 503189504.

When I open the file in the Joulescope UI software, I can zoom in and see that there are twice as many datapoints for the GPI[0] signal when compared to the power signal, so the data is there inside of the file.

Is there a difference between the decimated saved data and the non-decimated saved data? what am I missing here.

What OS are you using?

Windows 11

[mliberty1]

Hi @StrandmonYellow - I think that the primary issue is that the digital signal bits are returned packed in a byte. So, you get 8 digital samples per byte. You can unpack them using numpy, like this:

y = np.unpackbits(y, bitorder='little')

If you want, you can also use the JLS library to compute average power for you much faster than from python and without having to load all the data. Replace that r.fsr call with:

avg_power = r.fsr_statistics(power.signal_id, 0, p_length, 1)[0, jls.SummaryFSR.MEAN]

The UI uses this fsr_statistics call to compute power. It also truncates the signal to only what is stored for all included signals.

You will need to time align the GPI signal to the power signal. You can use the timestamp_to_sample_id and sample_id_to_timestamp methods for this. See the extract.py entry point for an example.

Does this help?

[StrandmonYellow]

This helps! Thanks. Why is time alignment necessary? I assume that the signals are sampled simultaneously? Only that the GPI signals are sampled with a twice as high frequency compared to the power signal.

[mliberty1]

The signals are sampled simultaneously, but the JLS file and recording makes no guarantee that the first samples are aligned across the signals. For a single Joulescope, you can use the signal's sample_id_offset property to align them. However, this does not work for multiple Joulescopes. In general, it's better just to time align, which is not difficult.