Found a problem in the Decode function

[Zanoroy]

I have converted your code to C# to use with SDRShape.

I found a problem in your code that results in sections of messages being missed and a false report of 'Invalid Offset'.

In the 'demod_flex.c' you have: `

    int mw2 = mw1+len;
            ...
    if (mw1 > 87 || mw2 > 87){
        verbprintf(3, "FLEX: Invalid Offsets\n");
        continue;               // Invalid offsets
    }

    if (is_alphanumeric_page(flex))
        parse_alphanumeric(flex, phaseptr, PhaseNo, mw1, mw2-1, j);

`

You are getting the message length from the VIW and adding it to the start position, but the start position is the first word, there for the length needs to be reduced by 1, which you are doing when calling the parse_alphanumeric, unfortunately you have the if statement above it checking that mw1 and mw2 fall within the buffer range (which with split messages they do not, your logic always results in mw2 being 88).

Change the code to the following and it works: (I have not tested with numeric or tone only messages) `

    int mw2 = mw1+len - 1;
             ...
    if (mw1 > 87 || mw2 > 87){
        verbprintf(3, "FLEX: Invalid Offsets\n");
        continue;               // Invalid offsets
    }

    if (is_alphanumeric_page(flex))
        parse_alphanumeric(flex, phaseptr, PhaseNo, mw1, mw2, j);

`

I hope that made sense.

[joper112]

I changed it but FLEX: 2017-05-29 10:14:59 1600/2/A 03.105 [001220499] UNK 00000000 00000000 00000000 00000000 00000000 00000000 00000000

[jklnz]

Yeah I did the change as well and none of the fragmented/split messages came through

[Zanoroy]

Really, I made the change I mentioned in my post and since I have been receiving my multi-part messages correctly. I am receiving FLEX1600 2FSK messages off-air.

I can post a couple of WAV files of the recorded audio if that would help you?

[Zanoroy]

You did change BOTH locations right?

int mw2 = mw1+len - 1

and

parse_alphanumeric(flex, phaseptr, PhaseNo, mw1, mw2, j);

[jklnz]

Yeah I did both of the changes in demod_flex.c

[jklnz]

Just recompiled it and re-tested on the pager network, and it still misses them

[Zanoroy]

That's interesting, Do you have any recorded audio I could test with? I'd be interested in seeing why its failing for you. What is the flex network you are testing with? 1600, 3200 or 6400? and which 'type' of message are you expecting (Alpha, Numeric, tone or unknown) ? I noticed in the original code the Numeric,tone and unknown function calls didn't adjust the incorrect mw2 value, so may those functions are expecting the mw2 value to be 1 digit higher? I will go have a look shortly.

Cheers

[jklnz]

Yeah I will try and get a recording from SDR Sharp for you and upload it. The network is Spark's paging network in New Zealand. The messages are FLEX 1600 text messages (usually fire service dispatches).

[jklnz]

Just confirming - does it make a difference that I am using Windows not Linux??

[joper112]

I am using linux and also flex 1600 messages (also fire service only the dutch one) freq 169.650 P2000 FLEX Baudrate 1600 Modulation 2FSK. Some messages are ok but the group messages are unk 0000000000

[Zanoroy]

Oh, your on that Infostream network..

If you could get me a few minutes of RAW Audio or IQ data I can reprocess it with my test bed.

The network I have access to does use Group Messages so I cannot test it with some raw data sorry.

I have the ability to walk through the code Symbol by Symbol so its easy to trace issues.

[jklnz]

Yeah it decodes most of the messages perfectly except for the multipart ones which will not decode at all... Not sure if it is something with our network here or the software.

I have recorded a few minutes off the pager network with some messages that decoded fine in PDW but not in multimon-ng. Let me know if that helps or you need any more info

SDRSharp_20170530_081552Z_157950006Hz_AF.zip

Sorry I had to zip it up as it would not let me upload a WAVE

[jklnz]

What's the infostream network?

[Zanoroy]

Infostream is the company that designed and installed the dutch Base Stations so I guess my Infostream comment was more to Joper112.

I'll let you know how I go with the raw data.

Cheers

[Zanoroy]

Do you have your log information from that raw data please.

[joper112]

@Zanoroy Can you give me the command zo i can get a few minutes of data for you?

[jklnz]

Do you mean the decoded messages?

[Zanoroy]

Sorry, could you run it through your decoder with the verbose set to 3 and shoot me the logs (I don't my Linux VM running at home).

[Zanoroy]

@joper112, what are you using as to control your receiver? GQRX? if so you can use the record button on the bottom right of the GQRX gui.

[joper112]

@Zanoroy no iam using rtl_fm i have a .bin on dropbox when you want a other file i need to install gqrx this evening https://www.dropbox.com/s/dxg0rg2qhakmtb5/demodulated.bin.bz2?dl=0

[Zanoroy]

That's ok, rtl_fm records raw IQ data I can use that (with a little extra playing around)

Cheers..

On Tue, 30 May 2017, 18:26 joper112, notifications@github.com wrote:

@Zanoroy https://github.com/zanoroy no iam using rtl_fm i have a .bin on dropbox when you want a other file i need to install gqrx this evening https://www.dropbox.com/s/dxg0rg2qhakmtb5/demodulated.bin.bz2?dl=0

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/EliasOenal/multimon-ng/issues/63#issuecomment-304816423, or mute the thread https://github.com/notifications/unsubscribe-auth/AH0NJTCsE-FRLpeB0d1qAiSQglaXgPZcks5r-9mmgaJpZM4No1y8 .

[jklnz]

I have uploaded a second file to onedrive: https://1drv.ms/u/s!AroehrZ8a8wxa3Hzw4vAnlRPDCQ This time i included a .log of the multimon-ng -v 3

The file was too big for github apparently!

[jklnz]

This is the relevant info from the log: FLEX: Locked FLEX: SyncInfoWord: sync_code=0x870c baud=1600 levels=2 polarity=POS zero=-0.125367 envelope=0.854760 symrate=1601.641791 FLEX: State: FIW FLEX: FrameInfoWord: cycleno=08 frameno=024 fix3=0x3c time=32:45 FLEX: State: SYNC2 FLEX: State: DATA FLEX: BlockInfoWord: (Phase A) BIW:00000C03 AW:01-03 (2 pages) FLEX: CAPCODE:00000000000f39ba FLEX: Parse Alpha Numeric FLEX: 2017-05-30 09:32:27 1600/2/A 08.024 [000997818] ALN (BULL827, FEIL287, BULL821) PHONE-RESC SH 3 SANSON MANAWATU DISTRICT. (XStr ACOURT ST/MOUNT STEWART HALCOMBE RD) .MVA TRUCK OFF ROAD. (x-338991015 y-603739171) #F2317160 FLEX: CAPCODE:00000000000f416f FLEX: Parse Alpha Numeric FLEX: 2017-05-30 09:32:27 1600/2/A 08.024 [000999791] ALN (BULL827, FEIL287, BULL821) PHONE-RESC SH 3 SANSON MANAWATU DISTRICT. ( FLEX: State: SYNC1 FLEX: SyncInfoWord: sync_code=0x7b18 baud=1600 levels=2 polarity=POS zero=-0.056757 envelope=0.868479 symrate=1600.067300 FLEX: State: FIW FLEX: FrameInfoWord: cycleno=08 frameno=025 fix3=0x3c time=32:46 FLEX: State: SYNC2 FLEX: State: DATA FLEX: Synchronisation Lost FLEX: State: SYNC1 FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: SyncInfoWord: sync_code=0x870c baud=1600 levels=2 polarity=POS zero=-0.123476 envelope=0.883384 symrate=1601.641791 FLEX: State: FIW FLEX: FrameInfoWord: cycleno=08 frameno=032 fix3=0x3c time=33:00 FLEX: State: SYNC2 FLEX: State: DATA FLEX: BlockInfoWord: (Phase A) BIW:0000040B AW:01-01 (0 pages) FLEX: State: SYNC1 FLEX: Synchronisation Lost

[Zanoroy]

Excellent, thanks

On Tue, 30 May 2017, 19:10 jklnz, notifications@github.com wrote:

This is the relevant info from the log: FLEX: Locked FLEX: SyncInfoWord: sync_code=0x870c baud=1600 levels=2 polarity=POS zero=-0.125367 envelope=0.854760 symrate=1601.641791 FLEX: State: FIW FLEX: FrameInfoWord: cycleno=08 frameno=024 fix3=0x3c time=32:45 FLEX: State: SYNC2 FLEX: State: DATA FLEX: BlockInfoWord: (Phase A) BIW:00000C03 AW:01-03 (2 pages) FLEX: CAPCODE:00000000000f39ba FLEX: Parse Alpha Numeric FLEX: 2017-05-30 09:32:27 1600/2/A 08.024 [000997818] ALN (BULL827, FEIL287, BULL821) PHONE-RESC SH 3 SANSON MANAWATU DISTRICT. (XStr ACOURT ST/MOUNT STEWART HALCOMBE RD) .MVA TRUCK OFF ROAD. (x-338991015 y-603739171) #F2317160 FLEX: CAPCODE:00000000000f416f FLEX: Parse Alpha Numeric FLEX: 2017-05-30 09:32:27 1600/2/A 08.024 [000999791] ALN (BULL827, FEIL287, BULL821) PHONE-RESC SH 3 SANSON MANAWATU DISTRICT. ( FLEX: State: SYNC1 FLEX: SyncInfoWord: sync_code=0x7b18 baud=1600 levels=2 polarity=POS zero=-0.056757 envelope=0.868479 symrate=1600.067300 FLEX: State: FIW FLEX: FrameInfoWord: cycleno=08 frameno=025 fix3=0x3c time=32:46 FLEX: State: SYNC2 FLEX: State: DATA FLEX: Synchronisation Lost FLEX: State: SYNC1 FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: Synchronisation Lost FLEX: Locked FLEX: SyncInfoWord: sync_code=0x870c baud=1600 levels=2 polarity=POS zero=-0.123476 envelope=0.883384 symrate=1601.641791 FLEX: State: FIW FLEX: FrameInfoWord: cycleno=08 frameno=032 fix3=0x3c time=33:00 FLEX: State: SYNC2 FLEX: State: DATA FLEX: BlockInfoWord: (Phase A) BIW:0000040B AW:01-01 (0 pages) FLEX: State: SYNC1 FLEX: Synchronisation Lost

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/EliasOenal/multimon-ng/issues/63#issuecomment-304826654, or mute the thread https://github.com/notifications/unsubscribe-auth/AH0NJYfZBNAz_cEsPJfEjR1Hn0He_gq1ks5r--QMgaJpZM4No1y8 .

[Zanoroy]

Sorry guys, I have to call it a night. I'll have some time in the morning tomorrow to work on this and I'll give you an update. One thing I found was if you open the audio file in Autacity and re-export it, the system gets a few more symbols - I had the same trouble with my system when the CPU was busy and the raw data from the USB radio was not getting cleanly demodulated. But let me work on it and see what I can find.

Cheers

[jklnz]

All good, I left multimon-ng running side by side with PDW over night and it did pick up a couple of multipart messages but very few, maybe 1 out of 20.

[Zanoroy]

Did you capture any raw data between the two? Also are you using the same radio for both decoders?

[Zanoroy]

I can decode 90% of the data you supplied in your second file, but the last section has signals that don't last the minimum amount of time to be considered a symbol, it would be good to see what PWD think it can decipher in the same phase. Looking at the PWD code it takes a shorter amount of samples to be considered a valid symbol.

Multimon-ng seems to measure the sample length to be (for 1600bps) ~24 samples, where as PWD seems to accept 20 samples. But I need to check the code a bit more to see how its centered on the symbol and not getting a false reading for consecutive symbols...

[jklnz]

Nah I did not get any raw data as it is an old setup that is running on a thin client. Both PDW and multimon-ng were getting the info from line-in from a uniden scanner. Both PDW and multimon-ng were running on the same computer at the same time.

[jklnz]

The sample thing sort makes sense, guess that could explain the missed decodes.

[Zanoroy]

No problems.. I'm having trouble getting PDW to 'hear' my audio. I'm on windows 10 and the receive indicator in pdw doesnt seem to go above 25% so its a bit hard to use it.. all good.. I'll get it working..

[jklnz]

Yeah PDW is not easy to get working.... it requires alot of volume changes to find the perfect setup. One of my set ups is using sdrsharp to send the audio to VB audio cable then in PDW setting the VB audio cable as input.

Just checking you do have it set to decode FLEX in the options? I think this is disabled by default.

[Zanoroy]

Yeah, i have it set as Flex. Stuffed if I can get it to work at the moment. I assume I don't need the USB hardware device to convert the 2FSK, although I have one at the office. I have the Speaker OUT looped into the Line-In (using a 3.5mm audio cable) .

I'll keep playing. This is a pain in the butt...

[jklnz]

Nah you don't need that device.

Every time i have had to set it up on a new computer it has taken me over a day to get it right, it is very very difficult but once you get it set then you are good.

[Zanoroy]

Ok, so I installed VA Cables and I have it processing data. 👍 Now I'll just have to tune it a little..

Just to confirm the network you recorded the data from was Flex1600?

[jklnz]

Sweet good to hear! Yeah the messages are FLEX 1600. We only have 1600 Flex over here. There is probably a couple of POCSAG messages in there too.

[Zanoroy]

PDW Identifies every one of the missing messages at a buad rate of 3200. PDW seems to ignore the Sync2 baud rate identifier and just deciphers the raw data regardless. Where as Multimon-ng is actually obeying the Sync2 rates and therefore see the symbols as incomplete.
I'm looking into the raw data now to verify

[jklnz]

That's really interesting! two separate computers with completely separate radios and aerials produce the same results... so that is definitely weird.

[Zanoroy]

Ok, so I found the issue.. Alas Multimon-ng was at fault..

Give me a minute and I'll write up the fix (which is quite simple)

[Zanoroy]

So, Multimon-ng is missing a Sync identifier flag from the 'decode_mode' function (currently marks as "Unknown") and therefore never identified the baud rate change, and defaulted to 1600. The easy fix is to change the Array deceleration from:

flex_modes[] = {
        { 0x870C, 1600, 2 },
        { 0xB068, 1600, 4 },
        //  { 0xUNKNOWN,  3200, 2 },
        { 0xDEA0, 3200, 4 },
        { 0x4C7C, 3200, 4 },
        {0,0,0} 

to:

flex_modes[] = {
        { 0x870C, 1600, 2 },
        { 0xB068, 1600, 4 },
        { 0x7B18,  3200, 2 },
        { 0xDEA0, 3200, 4 },
        { 0x4C7C, 3200, 4 },
        {0,0,0} 

I actually modified the function in my code to return a Boolean to indicate if the Sync flag was found. If it wasn't found I still default to 1600, but I added a status message to the output to indicate the missing Sync Flag. As my code is c# I also collect the messages slightly differently because I'm not output to the console for a few seconds so I can stitch my multi-part messages together (its a work in progress but its working well so far).

[Zanoroy]

I'm just trying the RTL_FM bin file from joper112 but having trouble convert it to a standard sample rate to process...

[Zanoroy]

@joper112 could you please tell me the settings you used on RTL_FM to record the data? (I'm assuming you didn't pipe it through SOX before sending to Multimon-ng, if you did I need to know the SOX settings also please)

[jklnz]

Thanks alot for that @Zanoroy - I will make the changes when I get home tonight and test it out. Nice work.

What are you doing C# wise? Are you making an addin for SDRsharp?

[Zanoroy]

Yeah, I'm actually developing a drive test kit for a Flex network and wanted to be able to link Raw IQ, Raw Demodulated FM, Symbols, location Lat/Long, RSSI and Decoded messages at the same time and display it all on openmaps so I ported the code to C# to run on a standard PC.

So I first wrote the C# code as a two part SDRSharp plugin. I have a very beta version that replicates 90% of PDW as an SDRSharp plugin.

Now it time to write a BladeRF (because that's what I had lin the cupboard) interface to pump raw data to the plugin without the need for all the overhead SDRSharp puts on a system (plus I'm a bit picky, the way SDRSharp is coupled together is (in my opinion) messy - but to be fair, its been developed over wide time so of course its going to change from one style to another (as I said, I'm picky).

Let me know how you go.

[Zanoroy]

@joper112 wow you have been dealing with this issue for a long time, I just went though some of the other issue threads and you popup everywhere.

Could you please implement the changes above and let me know how it goes. If it doesn't fix your Group Message problem let me know and I'll work it out and send through the fix.

@EliasOenal I'm happy to pick up the Flex Decoding issues if Chris isn't around for a while. Just point out the ones you want looked at.

[jklnz]

Looking good @Zanoroy ! I just recompiled it and so far two fragmented messages have come through on both PDW and multimon

[Zanoroy]

Awesome :)

[jklnz]

I have had it running for about 55 mins and it is picking up A LOT more messages than before your change which is great, however it is definitely still missing a few messages... I will see if I can get any more info/a log...

[Zanoroy]

No problems, if you can get raw data for me to process I can fix the issues.

Cheers

On Wed, 31 May 2017, 14:22 jklnz, notifications@github.com wrote:

I have had it running for about 55 mins and it is picking up A LOT more messages than before your change which is great, however it is definitely still missing a few messages... I will see if I can get any more info/a log...

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/EliasOenal/multimon-ng/issues/63#issuecomment-305082502, or mute the thread https://github.com/notifications/unsubscribe-auth/AH0NJWTVTm-P619eps95wIESl4H9ZEGCks5r_PIpgaJpZM4No1y8 .

[joper112]

@Zanoroy yes im waiting already a long time :P the command i run is : rtl_fm -f 169.65M -s 22050 -p -19 | multimon-ng -a FLEX -a SCOPE -t raw /dev/stdin >> /var/www/pager/p2000.txt When you have a better one i like to hear!