muccc / iridium-toolkit

A set of tools to parse Iridium frames
477 stars 111 forks source link

error in iridium-parser.py #79

Closed fe8769a closed 2 years ago

fe8769a commented 2 years ago

hello

I have this error with iridium-parser.py :+1:

print("WARN: no sigmf-meta source file. Using (probably-wrong) hardcoded defaults", file=sys.stderr)

I tried Gnuradio 3.7 , gnuradio 3.8 (dragon_os focal 21) I tried previous working outputs.bis I tried pypy pypy3 python3

same problem

Any idea /thanks

Sec42 commented 2 years ago

If you use "-o sigmf" you need to also specify "--sigmf recording.sigmf-meta" to the existing meta-file. Otherwise the extractor doesn't know essential information like the sample rate of your existing recording to produce correct annotations.

fe8769a commented 2 years ago

Thanks for answer but I didn't understand the answer . I m using iridium toolkit from the beginning . Here is my command lines

iridium-extractor --multi-frame -q 80 -D 8 examples/b210.conf > /home/xxxx/Iridium/Data/output.bits

then

pypy /home/fxxxx/Iridium/iridium-toolkit/iridium-parser.py output.bits > output.parsed

That 2 command lines was working for a long time

Sec42 commented 2 years ago

Ah. I misunderstood your error.

You are using "pypy" which is python2. iridium-toolkit requires python3 since last year. Use "python3" or "pypy3" to run it.

If you can't upgrade to python3 for whatever reason, revision a469adc4b2fdd177c3b8ace0166b67618bf03a25 (tagged as python2) is the last one that works with python2.

fe8769a commented 2 years ago

I ll clone this working disk to a new one and upgrade to python3

Thanks

Sec42 commented 2 years ago

Let me know if you get it to work.

fe8769a commented 2 years ago

Hello Python3 is working fine but I have some questions and errors

What is the exact line to get data ?

is this syntax mandatory : | fgrep "A:OK" -->output.bits is empty

What is the extact line ?

iridium-extractor -D 4 -q 60 /home/xxxx/Desktop/Transfert_upgrade/Iridium/gr-iridium/examples/usrp1-b2x0.conf > /home/xxxx/Desktop/Transfert_upgrade/Iridium/data/output.bits

or

iridium-extractor -D 4 -q 60 /home/xxxx/Desktop/Transfert_upgrade/Iridium/gr-iridium/examples/usrp1-b2x0.conf | fgrep "A:OK" > /home/fxxxx/Desktop/Transfert_upgrade/Iridium/data/output.bits

the second line was working fine with previous version


I have an error getting voice in line :+1:

python3 /home/xxxx/Desktop/Transfert_upgrade/Iridium/iridium-toolkit/bits_to_dfs.py /home/xxxx/Desktop/Transfert_upgrade/Iridium/data/voc.bits /home/xxxx/Desktop/Transfert_upgrade/Iridium/data/output.dfs

File "/home/xxxxDesktop/Transfert_upgrade/Iridium/iridium-toolkit/bits_to_dfs.py", line 50, in data = line[10] IndexError: list index out of range

Partial view of voc.bits seems to be good

VOC: p-1647679276-e000 000015386.7453 1625240320 80% -83.1|-129.3|19.4 179 DL LCW(0,T:maint,C:maint[2][lqi:3,power:0,f_dtoa:0,f_dfoa:0],0|0) [40.20.00.9f.40.80.95.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.90.00.00.00.00.00.00] VOC: p-1647679276-e000 000022856.8850 1625239936 73% -81.9|-130.4|18.3 179 DL LCW(0,T:hndof,C:handoff_cand,264,136,010011001000100110110) [40.c4.00.5b.40.89.95.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.18.00.12.00.00.00.00] VOC: p-1647679276-e000 000025106.9251 1625240192 75% -83.1|-130.2|19.5 179 DL LCW(0,T:maint,C:maint[2][lqi:3,power:0,f_dtoa:1,f_dfoa:0],0|0) [40.b4.00.01.40.89.95.00.00.00.90.00.00.00.00.02.10.00.00.00.00.00.01.80.00.00.02.40.00.00.00.00.00.00.00.00.00.00.00] VOC: p-1647679276-e000 000025466.9312 1625240064 82% -81.9|-130.1|20.1 179 DL LCW(0,T:maint,C:maint[2][lqi:3,power:0,f_dtoa:0,f_dfoa:0],0|0) [40.74.00.f3.44.7e.08.62.9e.8e.4b.76.6a.00.00.04.00.d9.00.ab.6b.eb.1b.9b.5b.47.c7.ff.0d.bc.8d.6c.a8.81.f7.9d.61.71.b4]


Other error : google kml files results are opening many windows in firefox , line is :+1:

grep ^IRA /home/xxxx/Desktop/Transfert_upgrade/Iridium/data/output.parsed |perl /home/xxxx/Desktop/Transfert_upgrade/Iridium/iridium-toolkit/mkkml tracks > /home/xxxx/Desktop/Transfert_upgrade/Iridium/data/output.kml

detail of output.parsed

IDA: p-1647679276-e000 000000626.8262 1623434112 91% -75.4|-129.7|20.5 179 DL LCW(2,T:maint,C:maint[2][lqi:0,power:0,f_dtoa:63,f_dfoa:63],0|0) 000 cont=0 0 ctr=000 000 len=00 0:0000 [86.97.09.7b.c0.a4.98.0b.6f.30.4f.c0.05.fc.00.6f.c0.07.fc.00.00.00.00] --- 0000

thanks for help

Sec42 commented 2 years ago

[1]: Please consult the README of the version of gr-iridium that you are running. If you are running from git master, https://github.com/muccc/gr-iridium/blob/master/README.md is correct (and has no reference to A:OK) [2]: At the moment I can not give support for voice decoding [3]: the kml file is intended for google-earth, the desktop app. firefox (or any browser) is not involved.

fe8769a commented 2 years ago

Hello . I give up . All my previous was working fine . I have python 3.5 installed , I migrated it to 3.9 . I still have unknown problem

First with the parser

python3 /home/xxxx/Iridium/iridium-toolkit/iridium-parser.py output.bits > output.parsed

Traceback (most recent call last): File "/home/xxxx/Iridium/iridium-toolkit/iridium-parser.py", line 14, in import bitsparser File "/home/xxxx/Iridium/iridium-toolkit/bitsparser.py", line 18, in import itl File "/home/xxxx/Iridium/iridium-toolkit/itl.py", line 544 BIN_V2= f'{int(PRS_V2,16):0{len(PRS_V2)*4}b}' ^ SyntaxError: invalid syntax


With previous version , voice was working good also with kml (result can be opened in firefox with no trouble)

Sec42 commented 2 years ago

Interesting. I just learned that the f-strings (used in that line) are a language feature introduced in python 3.6. That is why your python 3.5 is complaining about this line.

I had assumed it was a standard python3 feature. I will try to rewrite the three affected lines in a more backwards-compatible way and commit them soon.

fe8769a commented 2 years ago

I made a mixed with 2 versions . ... I acquire signal with my python2 version . I made parser with old version but I saw "--harder" option was working with old version (great) --> voice is much better . I m using the current version only with acars

That is a working mess

fe8769a commented 2 years ago

Some more job - I retrieved git with modify version and with python3, I have the following error

python3 /home/xxxxx/Iridium/iridium-toolkit/iridium-parser.py --harder /home/xxxx/Iridium/Data/output.bits > /home/xxxx/Iridium/Data/output.parsed Traceback (most recent call last): File "/home/xxxxx/Iridium/iridium-toolkit/iridium-parser.py", line 14, in import bitsparser File "/home/xxxx/Iridium/iridium-toolkit/bitsparser.py", line 18, in import itl File "/home/xxxx/Iridium/iridium-toolkit/itl.py", line 569 bseq=f'{int(seq,16):096b}' ^ SyntaxError: invalid syntax

And with pypy3

replace python3 by pypy3

Traceback (most recent call last): File "/home/xxxxx/Iridium/iridium-toolkit/iridium-parser.py", line 14, in import bitsparser File "/home/xxxx/Iridium/iridium-toolkit/bitsparser.py", line 9, in import crcmod ModuleNotFoundError: No module named 'crcmod'

crcmod is installed but not attach to the right python version ?

Sec42 commented 2 years ago

I don't know what to tell you.

This is not ready-for-use software. We called it "toolkit" for a reason. The whole software is written by two people in their free time as a hobby. Using it will require you to invest some time in understanding what it does and how it does it.

The current code is working fine for me (and several other people) on a reasonably modern system with python 3.7 or newer. I have done my best to be backwards compatible where I can. I also don't think it is very constructive to call my code a "mess".

Python2 has been end of live since Jan 2020. I have tagged the last version of the toolkit that runs with python2 which you can continue to use. However if you want new features & improvements at some point you have to bite the bullet and upgrade.

I understand you are frustrated. But maybe also think of the time of people on the other side of your messages.

For example, your initial report says

I tried pypy pypy3 python3

same problem

but we have since established that when running with python3 you get a different error, and not the "same".

Similarly, you said

I have python 3.5 installed , I migrated it to 3.9 .

Yet the error you pasted is specific to python 3.5 and older. - As far as I can tell, you were unsuccessful in your attempt to migrate to 3.9.

It seems you are having trouble with upgrading things in place & have trouble with different versions.

My suggestion would be to stick the version that was working for you.

And once you have calmed down and have some free time, try a second setup from scratch with more current software.

Sec42 commented 2 years ago

a) I have pushed a second patch to itl.py to get rid of the two remaining f-strings b) yes, pypy requires it's own separate installation of python modules. See https://doc.pypy.org/en/latest/faq.html#module-xyz-does-not-work-with-pypy-importerror

fe8769a commented 2 years ago

I ll try with dragon os focal and let you know

fe8769a commented 2 years ago

Thanks for help .

In order to make automatic voice aquisation ,here is the modifications to do 👍

python3 /home/xxxxx/Desktop/Transfert_upgrade/Iridium/iridium-toolkit/iridium-parser.py --harder /home/xxxxx/Desktop/Transfert_upgrade/Iridium/data/output.bits > /home/xxxxx/Desktop/Transfert_upgrade/Iridium/data/output.parsed

python3 /home/xxxxx/Desktop/Transfert_upgrade/Iridium/iridium-toolkit/iridium-parser.py /home/xxxxx/Desktop/Transfert_upgrade/Iridium/data/output.bits > /home/xxxxx/Desktop/Transfert_upgrade/Iridium/data/iridium-packets.txt

grep "VOC" /home/xxxxx/Desktop/Transfert_upgrade/Iridium/data/iridium-packets.txt > /home/xxxxx/Desktop/Transfert_upgrade/Iridium/data/voc.bits


Make the following modification in bits_to_dfs.py .

at line 50 change "data = line[10]" by data = line[9]"

That will avoid the message "list index out of range"

python3 /home/xxxxx/Desktop/Transfert_upgrade/Iridium/iridium-toolkit/bits_to_dfs.py /home/xxxxx/Desktop/Transfert_upgrade/Iridium/data/voc.bits /home/xxxxx/Desktop/Transfert_upgrade/Iridium/data/output.dfs

python3 /home/xxxxx/Desktop/Transfert_upgrade/Iridium/iridium_cor/bits_to_dfs.py /home/xxxxx/Desktop/Transfert_upgrade/Iridium/data/voc.bits /home/xxxxx/Desktop/Transfert_upgrade/Iridium/data/output.dfs

/home/xxxxx/Desktop/src/iridium-toolkit/ir77_ambe_decode /home/xxxxx/Desktop/Transfert_upgrade/Iridium/data/output.dfs /home/xxxxx/Desktop/Transfert_upgrade/Iridium/data/output.wav

Maybe this easiest way by setting python env but I haven't enough knowledge

Working fine with last dragon os focal

CobraXLazer commented 2 years ago

Hello

I also get the error on my fresh 20.04 setup,

Traceback (most recent call last): File "/home/cobra/gr-iridium/iridium-toolkit/iridium-parser.py", line 14, in import bitsparser File "/home/cobra/gr-iridium/iridium-toolkit/bitsparser.py", line 9, in import crcmod ModuleNotFoundError: No module named 'crcmod'

How to get the "crcmod" installed, it should be included in the python pip thing....

Great work anyways, had it working on the older versions . cheers

CobraXLazer commented 2 years ago

Found a fix for the problem!

sudo pip3 install --no-cache-dir -U crcmod

cheers

Sec42 commented 2 years ago

@fe8769a The bits_to_dfs problem is fixed in 38d82e9b120ab71826afbc2e9e82e6bdc34b228e. I believe this finishes up your issues. Feel free to reopen if anything remains.