Set default "decimal places"

[Mayerch1]

When generating signals from a log file, all output signals will display with Decimal Places: 0 within i2 by default. I can manually edit the signals to show more decimal places, however that's quite tedious.

Do you happen to know if it's possible to change the default value?

Signals generated by Motec devices (using pro logging), as well as math channels, are not showing this behavior, so I don't think it's i2 related.

Thanks for the lib though.

[stevendaniluk]

I experimented with this a bit and unfortunately I'm not sure how to resolve this. The ldparser library I'm using doesn't seem to properly encode/decode the decimal places. If I hard code all channels to have some fixed decimal places that doesn't get written and recognized by i2. Similarly if I open a log recorded with a MoTeC device with the ldparser library the decimal field for all channels always has a value of zero.

I pushed a commit to a3bbfc7 to partially handle this. It will automatically detect the number of decimal places from CSV and ACCESSPORT logs, and for CAN it will use a fixed value of 3 for all channels. Unfortunately that can't actually be written to the .ld file.

If myself or someone can debug why the ldparser library isn't properly handling decimal places then this can be added easily by replacing the line in motec_log.py for setting decimal places to use the Channel object's value. But until then we're stuck with zero decimal places for everything.

[Mayerch1]

Thanks for the effort. I might take a look at it, but I don't see myself finding time for it in the near future

[jLynx]

Side note, I have noticed some strange encoding where it will set the max for some values instead of leaving them on auto. So I wonder if we are setting the decimals in the wrong section of the struct

[jLynx]

Seems like it is in the correct place. Also there are some other repos doing the same sort of thing and their struct seems to be basically the same https://github.com/GeekyDeaks/sim-to-motec/blob/119cb4aa999d5333bb09eaa279742ecebb4ef010/util/ams2dump-to-motec.py#L107C1-L107C1

"id": 10003,
"datatype": 7,
"datasize": 4,
"freq": 20,
"shift": 0,
"multiplier": 1,
"scale": 1,
"decplaces": 0,
"name": "GPS Longitude",
"shortname": "GPSLong",
"units": "deg"

But then we have other projects like this rust one https://github.com/afonso360/motec-i2/blob/master/examples/write.rs#L71C23-L71C23 where it sets it to 7 but same struct again

 let gps_lat_meta = ChannelMetadata {
        prev_addr: 0,
        next_addr: 0,
        data_addr: 0,
        data_count: 0,
        datatype: Datatype::I32,
        sample_rate: 2,
        offset: 0,
        mul: 1,
        scale: 1,
        dec_places: 7,
        name: "GPS Latitude".to_string(),
        short_name: "GPS Lat".to_string(),
        unit: "deg".to_string(),
    };

[stevendaniluk]

So I tried comparing your log file @jLynx in #2 to another log I have from a MoTeC device that had GNSS data recorded at 20 Hz. When looking at the fields datatype, datasize, freq, shift, multiplier, scale, decplaces in MoTeC device log file I see:

5 0 4 0 20 0 0 0 1 0 1 0 7 0

If I uncomment this line to set the decimal values appropriately and run the tool against your log file I see:

7 0 4 0 20 0 0 0 1 0 1 0 7 0

The datatype value is 7 compared to 5 in the log from the MoTeC device. That gets written by the ldparser here. In both cases though i2 see's the data type as real.

I experimented a bit with modifying that but no change. I also tried modifying the struct data type for decimal places, but that also had no effect. i2 still doesn't recognize the decimal places.

So I do think something is incorrect about how the channel data is being packed, but I'm not confident it's the decimal place entry.

[jLynx]

Take a look at here @stevendaniluk https://github.com/GeekyDeaks/sim-to-motec/blob/119cb4aa999d5333bb09eaa279742ecebb4ef010/stm/motec/ld.py#L220

The struct for the entire thing is slightly different. So maybe this could be it? For example theirs starts with I4s and ours is I4x

Theirs:

header = MotecStruct([
        (    "I", "id"),
        (   "4s", None),
        (    "I", "firstchannelpos"),
        (    "I", "firstchanneldatapos"),
        (  "20s", None),
        (    "I", "eventpos" ),
        (  "26s", None),
        (    "I", "sig1"),
        (    "I", "serial"),
        (   "8s", "type"),
        (    "H", "version"),
        (    "H", "sig2"),
        (    "I", "numchannels"),
        (   "4s", None),
        (  "16s", "date"),
        (  "16s", None),
        (  "16s", "time"),
        (  "16s", None),
        (  "64s", "driver"),
        (  "64s", "vehicle"),
        (  "64s", None),
        (  "64s", "venue"),
        ("1088s", None),
        (   "4s", None),
        (  "66s", None),
        (  "64s", "comment"),
        ( "126s", None)
    ])

Ours:

fmt = '<' + (
        "I4x"     # ldmarker
        "II"      # chann_meta_ptr chann_data_ptr
        "20x"     # ??
        "I"       # event_ptr
        "24x"     # ??
        "HHH"     # unknown static (?) numbers
        "I"       # device serial
        "8s"      # device type
        "H"       # device version
        "H"       # unknown static (?) number
        "I"       # num_channs
        "4x"      # ??
        "16s"     # date
        "16x"     # ??
        "16s"     # time
        "16x"     # ??
        "64s"     # driver
        "64s"     # vehicleid
        "64x"     # ??
        "64s"     # venue
        "64x"     # ??
        "1024x"   # ??
        "I"       # enable "pro logging" (some magic number?)
        "66x"     # ??
        "64s"     # short comment
        "126x"    # ??
        "64s"     # event
        "64s"     # session
    )

[stevendaniluk]

@jLynx So that struct is for the main file, I would think the problem is more likely to be within the format of individual channels.

sim-to-motec's channel format is:

    header = MotecStruct([
        (  "I", "prevpos"),
        (  "I", "nextpos"),
        (  "I", "datapos"),
        (  "I", "numsamples"),
        (  "H", "id"),
        (  "H", "datatype"),
        (  "H", "datasize"),
        (  "H", "freq"),
        (  "h", "shift"),
        (  "h", "multiplier"),
        (  "h", "scale"),
        (  "h", "decplaces"),
        ("32s", "name"),
        ( "8s", "shortname"),
        ("12s", "units"),
        ("40s", None) # should only be 32
    ])

ldparser's channel format is:

    fmt = '<' + (
        "IIII"    # prev_addr next_addr data_ptr n_data
        "H"       # some counter?
        "HHH"     # datatype datatype rec_freq
        "hhhh"    # shift mul scale dec_places
        "32s"     # name
        "8s"      # short name
        "12s"     # unit
        "40x"     # ? (40 bytes for ACC, 32 bytes for acti)
    )

The above is for the most recent commit on master which I noticed is different from what I have submoduled here. This has a few entries as signed shorts while the currently used version has unsigned. I think updating this may be worth a try.

[jLynx]

I have already tried updating the submodule locally and still didn't work sadly. As for that structure, it is the same just presented differently. Only thing actually diffrent is the 40s vs 40x