Parsing special operating activity messages

[VE3NEA]

WsjtxQsoParser is a useful class that should probably be included in the WsjtxUtils library. However, it fails to parse some Special Operating Activity (such as ARRL Field Day) messages, e.g. VE3NEA VE3RYI R 1D GTA . My current work around is to just look for anything that is a valid callsign or grid square:

var wsjtxQso = WsjtxQsoParser.ParseDecode(message);
if (string.IsNullOrEmpty(wsjtxQso.DECallsign)) ParseSpecialMessage(wsjtxQso);

where

        private void ParseSpecialMessage(WsjtxQso wsjtxQso)
        {
            var parts = wsjtxQso.RawMessage
                .Replace("<", "")
                .Replace(">", "")
                .Split(new char[] { ' ' }, StringSplitOptions.RemoveEmptyEntries)
                .Where(part => part != "?")
                .ToArray();

            if (parts.Any() && parts[0] == MyCall)
                wsjtxQso.DXCallsign = parts[0];

            int idx = FindSenderCallsign(parts);
            if (idx == -1) return;
            wsjtxQso.DECallsign = parts[idx];

            if (++idx < parts.Length && parts[idx] == "R") idx++;
            if (idx < parts.Length && Utils.GridSquareRegex.IsMatch(parts[idx]))
                wsjtxQso.GridSquare = parts[idx];
        }

        private int FindSenderCallsign(string[] parts)
        {
            for (int i = parts.Length - 1; i > 0; i--)
                if (Utils.CallsignRegex.IsMatch(parts[i]))
                    return i;

            return -1;
        }

The regular expressions to match the callsign and grid square are as follows:

    internal static class Utils
    {
        public static Regex CallsignRegex = new Regex(
            // portable prefix
            @"^((?:(?:[A-NPR-Z](?:(?:[A-Z](?:\d[A-Z]?)?)|(?:\d[\dA-Z]?))?)|(?:[2-9][A-Z]{1,2}\d?))\/)?" +
            // prefix
            @"((?:(?:[A-NPR-Z][A-Z]?)|(?:[2-9][A-Z]{1,2}))\d)" +
            // suffix
            @"(\d{0,3}[A-Z]{1,6})" +
            // modifier
            @"(\/[\dA-Z]{1,4})?$",
            RegexOptions.Compiled
        );

        public static Regex GridSquareRegex = new Regex(@"^(?!RR73)[A-R]{2}\d{2}$", RegexOptions.Compiled);
}

It would be nice to add support of Special Activities message formats, and perhaps optionally validate the grid square and callsigns.

[KC3PIB]

Thanks, Alex! The intent is to move the WsjtxQsoParser into WsjtxUtils once it is more robust. Searchlight is currently in an alpha state and a testbed for continued progress on WsjtxUtils. The current WsjtxQsoParser implementation provides a basic set of capabilities for FT4/FT8-like messages but, as you noted, doesn't handle all Special Operating Activity messages.

The plan is to start with the FT modes and work from there. I will use the 48 messages output from ft8code -T as test vectors for the parsing classes. These messages include exchanges for field day, fox & hound, nonstandard callsigns, etc.

Adding callsign and grid square validation is an excellent idea, and thank you for your workaround idea and regular expressions.

[KC3PIB]

I have integrated more robust QSO parsing for 77-bit modes (FST4, FT4, FT8, MSK144, Q65) into the development branch of WsjtxUtils with https://github.com/KC3PIB/WsjtxUtils/commit/a978052fb89c519e61470e655f0a6021296402f0. There are unit tests based on the WSJT-X documentation and the output from ft8code -T, although it currently lacks examples of priori data used for decoding that case should be handled. These changes should allow parsing of standard messages, NA VHF Contest, EU VHF Contest, ARRL Field Day, ARRL RTTY Roundup, WW Digi Contest, and nonstandard callsigns. Also included are the validation of callsigns, support for 4 & 6-character grid squares, and better handling of parsing in various scenarios. Once the documentation is updated, I will merge these into the main branch.

[VE3NEA]

Wow! That was fast. Thank you very much, Chris, I will give it a try.