Signed field not on byte boundaries uses too many bits to calculate sign

[scandey]

For example: the 12 bit signed integer 0x800 returns -30720, or 0x800 - 0x8000. Looks like all integer fields are fit into the nearest numpy size that fits the required bits and sign extension is not working properly?

Example code:

# %%
import ccsdspy
from ccsdspy import PacketField, PacketArray
import io

# %%
pkt = ccsdspy.FixedLength([
    PacketField(name='uintfive', data_type='uint', bit_length=3),
    PacketField(name='negfive', data_type='int', bit_length=5),
    PacketField(name='postwelve', data_type='int', bit_length=12),
    PacketField(name='negtwelve', data_type='int', bit_length=12),
])
# %%
# 0b101, 0b11011, 0b000000001100, 0b111111110100
fakepkt = io.BytesIO(b"\x00\x01\xC0\x00\x00\x03\x5B\x00\xCF\xFA")
print(f'Fake twelve single file has {fakepkt.getbuffer().nbytes} bytes')
with open('fake_off_byte_fields.ccsds', 'wb') as file:
    file.write(fakepkt.getbuffer())

# %%
print(f'Attempting to load just one packet from file')
pkt.load('fake_off_byte_fields.ccsds')
# returns {'uintfive': array([2], dtype=uint8), 'negfive': array([-101], dtype=int8), 
#         'postwelve': array([12], dtype='>i2'), 'negtwelve': array([-28678], dtype='>i2')}

[ddasilva]

Thanks for reporting! I'll take a look at this tomorrow. There was a patch last week made to odd length negative int handling, which is probably related.

[ddasilva]

I think in your test data, your uintfive is actually 2 and negtwelve is actually -6. Can you double check this?

In [1]: from bitstring import BitArray

In [2]: arr = BitArray(bytes=b"\x00\x01\xC0\x00\x00\x03\x5B\x00\xCF\xFA")

In [3]: arr[48:51].uint
Out[3]: 2

In [4]: arr[-12:].int
Out[4]: -6

If that's is indeed the case, then the patch I pushed to gh-issue-78-fix branch should fix the issue.

Fake twelve single file has 10 bytes
Attempting to load just one packet from file
{'negfive': array([-5], dtype=int8),
 'negtwelve': array([-6], dtype=int16),
 'postwelve': array([12], dtype=int16),
 'uintfive': array([2], dtype=uint8)}

Best, Daniel

[scandey]

Totally right on both counts, that'll teach me to do binary conversions by hand late at night! Confirmed that it works on my larger files too. Thank you!

[ddasilva]

Fixed in #81