search

CroatiaControlLtd / asterix

Asterix is utility used to read and parse EUROCONTROL ASTERIX protocol data from stdin, file or network multicast stream and print it to standard output in text, XML or JSON format. Source can be used to generate Wireshark dissector for ASTERIX protocol. All ASTERIX categories are defined through XML definition file.
GNU General Public License v2.0
166 stars 89 forks source link

Decode ASTERIX string message #195

Open marius190 opened 2 years ago

marius190 commented 2 years ago

Hello everyone,

I have a resource problem when decoding an .ast file. Since the file contain 4h of ADS-B data when I use parse the computer run out of ram memory:

file="file directory.ast"
data = file.read()
parsed = asterix.parse(data) # No memory

The .ast file has about 52 MB.

If I read let's say, data[0:500] it works fine and decodes the first messages. So I had an idea and is to use parse on pieces of data: piece=data[k:k+500]

The problem I get when doing this is that parse does give incorrect decoding since (I suspect) I can not break data into equally sized pieces, because then I break the first and/or the last message in two pieces, and parse does not interpret correctly when a message starts and finish, giving 0's CRC check.

Questions:

  1. How can I know when a message ends, before using parse (using just data)?
  2. Also, I have another possibility. I have the data in a .txt file where each row is an asterix message. How can I decode one-by-one the messages. For example, 
    msg="150058F71B7B6BC3A614D70101000D92011D3ACFFE1D0E0E9D6778FF0E871D44A8A238409C3840A716BC51D501B00A0E46057800000007A6887D3840C42811B6C0A820030808C501F78359400303070602020302022A0306"
# asterix.parse( msg ? )
dsalantic commented 2 years ago

Use the following algorithm to read packet by packet from file:

  1. Read 1 byte from the file (this is an ASTERIX category)
  2. Read next 2 bytes from the file and create unsigned short number from it (take care of byte ordering). This number represents the size (N) of the packet (including first 3 bytes).
  3. Read following N-3 bytes from the file and send those data (including first 3 bytes) to the parser
  4. Repeat from step 1. until the end of file

If you want to send string data directly to parser you need to convert it to byte array like in this example: `asterix_packet = bytearray( [0x30, 0x00, 0x30, 0xfd, 0xf7, 0x02, 0x19, 0xc9, 0x35, 0x6d, 0x4d, 0xa0, 0xc5, 0xaf, 0xf1, 0xe0, 0x02, 0x00, 0x05, 0x28, 0x3c, 0x66, 0x0c, 0x10, 0xc2, 0x36, 0xd4, 0x18, 0x20, 0x01, 0xc0, 0x78, 0x00, 0x31, 0xbc, 0x00, 0x00, 0x40, 0x0d, 0xeb, 0x07, 0xb9, 0x58, 0x2e, 0x41, 0x00, 0x20, 0xf5])

parsed = asterix.parse(asterix_packet) `