Having trouble decoding FSK signal

[veso266]

Hello there, many thanks for writing this library, it works very well on the wav sample you provide: https://github.com/cytecbg/android-fskmodem/blob/master/examples/FSKModemDecodeWav/assets/fsk.wav

But I tried to decode some other FSK signal (CallerID) which you can find the details here: http://matthieu.benoit.free.fr/cid_dsp.htm and here: https://www.researchgate.net/publication/239926931_Caller_Line_Identity_Delivery_A_Computerized_Call_Alert_Application

The signal has the folowing properties

Transmission Scheme:        Analog, phase-coherent FSK
Logical 1 (mark)            1200 +/- 12 Hz
Logical 0 (space)           2200 +/- 22 Hz
Transmission Rate:          1200 bps

What I did was I added inside FSKConfig.java the signal properties

    public static final int CID_BELLCORE_FSK = 6;
    public static final int CID_BELLCORE_FSK_BAUD_RATE = 1225;
    public static final int CID_BELLCORE_FSK_LOW_FREQ = 2450; //Space
    public static final int CID_BELLCORE_FSK_HIGH_FREQ = 1225; //Mark

case CID_BELLCORE_FSK:

                this.modemBaudRate = CID_BELLCORE_FSK_BAUD_RATE;
                this.modemFreqLow = CID_BELLCORE_FSK_LOW_FREQ;
                this.modemFreqHigh = CID_BELLCORE_FSK_HIGH_FREQ;

                break;

then I inited my config like

 try {
            mConfig = new FSKConfig(FSKConfig.SAMPLE_RATE_44100, FSKConfig.PCM_8BIT, FSKConfig.CHANNELS_MONO, FSKConfig.CID_BELLCORE_FSK, FSKConfig.THRESHOLD_1P);
        } catch (IOException e1) {
            e1.printStackTrace();
        }

What was imidiatly wierd was why does the BAUD_RATE have to be 1225 and not 1200 (if its 1200 I get "Invalid sample rate or baudrate" exeption

also after listing all the frequency in calcFrequencyZerocrossing I discovered that I shold adjust mark from 1200 to 1225 and space from 2200 to 2450 because this 2 frequencies were most common in the file (also they are very close)

I also had to modify determineState function like this because if mark frequency is bigger then space on it did not detect hgih or low state properly

protected STATE determineState(int frequency, double rms) {

        STATE state = STATE.UNKNOWN;

        if (mConfig.modemFreqLow > mConfig.modemFreqHigh) //CID_SIG_V23 and CID_BELLCORE_FSK have Logical 0 (space) bigger frequency then Logical 1 (mark)
        {
            if (rms <= mConfig.rmsSilenceThreshold)
            {
                state = STATE.SILENCE;
            }
            else if (frequency > mConfig.modemFreqHighThresholdHigh && frequency <= mConfig.modemFreqLowThresholdHigh) //If frequency is between bigger and smaller frequency then we have a mark
            {
                state = STATE.LOW;
            }
            else if (frequency <= mConfig.modemFreqHighThresholdHigh)
            {
                state = STATE.HIGH;
            }
        }
        else //Other modem standards
        {
            if (rms <= mConfig.rmsSilenceThreshold)
            {
                state = STATE.SILENCE;
            }
            else if (frequency <= mConfig.modemFreqLowThresholdHigh) //If frequency is between bigger and smaller frequency then we have a mark
            {
                state = STATE.LOW;
            }
            else if (frequency <= mConfig.modemFreqHighThresholdHigh)
            {
                state = STATE.HIGH;
            }
        }

        return state;
    }

then it finaly started demodulating, but sadly the frame is wrong I used this function to dump demodulated bytes to file

public static void WriteByteArrayToFile(byte[] byteArray, String filePath) {
        try {
            // Write the byte array to the file
            FileOutputStream fos = new FileOutputStream(filePath, true);
            fos.write(byteArray);
            fos.close();
        } catch (IOException ex) {
            System.out.println("An error occurred: " + ex.getMessage());
        }
    }

and this is what I got out of my sample:

F6 BE AFF F7 7D FF 7A F7 7B FF FB 7F D7 FE AF FFF

Now I would assume that the first byte should be message type (so 0x80 (MDMF) or 0x40 not 0xF6 the samples are here https://www.mediafire.com/file/1otw9ien0u9iv1u/CID_sample-8.wav/file -- this is whole CallerID sample including Channel Seizure and Mark signal

https://www.mediafire.com/file/m8bvgmwzdqohqeu/fsk_only_no_cs-8.wav/file -- this is only FSK part

I also tried with this very simple sample: https://www.mediafire.com/file/r6nve95k5jduepm/fsk_10010110-8bit.wav/file which only has 10010110 encoded in FSK but I still get only 0xFF 0xFF frame out, so I realy don't know anymore whats happening, is the baudrate I set in config the problem?

Hope you maybe have some idea, why even the simplest sample doesn't want to decode, because I am out of ideas, how to even debug this (I mean I know real world signals need filters, but this simple sample should be fine only using ZeroCrosing to figure out what 1 and what 0 is)

Thanks for Anwsering and Best Regards

[iganev]

Oh gosh... I didn't expect someone to be looking at my ancient spaghetti in 2023 :laughing:
I will try to give you a coherent answer tomorrow.
Thanks!

[veso266]

Thank you so much, I am realy exited for that

This is not a spaghetti, compared to other fskmodem code I read this is the most beautiful and easy to understand code I have ever seen so many thanks for making it

[iganev]

So.. To give you some context.
Almost 15 years ago I was playing around with Arduino and had the idea to write a simple interface for Android to control a stepper motor jig by talking to the Arduino through the 3.5mm jack of the phone.

At the time I found a C library written by someone called SoftModem or something like that. It was essentially mimicking in a very crude way the FSK communication between internet modems from back in the day when that used to be a thing.
It was working OK-ish when you connect two arduinos and have them talk to each other. I did my best to try and understand how the library modulated and demodulated signals, did some wikipedia reading and hacked together this library for Java SE, which was easily portable to older Android versions (no idea how things are nowadays).

To make it work reliably I had to modify both the SoftModem library and my library a little bit by adding extra start bits so I could align the signal start with my lame code.. to this day I am not sure why I had to do the modifications, but essentially I had to place a very specific starting signal so I could align the signal start and end and so encode/decode messages properly.

You can think of it similarly to how UART communication has start and end bits that wrap the transmission bytes... again, very very rudimental.

Looking at the code, what I am referring to is ENCODER_PRE_CARRIER_BITS and ENCODER_POST_CARRIER_BITS.
3 bits of high signal after silence indicate incoming communication. A high bit followed by silence indicates the communication end.

This is super specific to my implementation, so I would suspect your use case has a different signal layout. Looking at the linked documents you should have a single LOW start bit (but in your case, that's the high-frequency signal), followed by 8 bits data and a HIGH end bit (in your case low-frequency signal).

Also, another thing to look into is the fact that my interpretation had the low frequency carrying the LOW bit and the high frequency carrying the HIGH bit, which in your case is the other way around.

The baud rate is also something that you need to be careful about. That would explain why the zero-crossings output you saw isn't matching your expectations.

My code lives in a perfect world where the audio sample rate is perfectly divided into chunks and each chunk holds a valid bit or silence. For example, a sample rate of 22050 (typical 22khz sample rate), a baud rate of 126 bits per second means that there are exactly 175 samples per bit. So I read the signal in frames representing 1 bit and analyze what I've read.

Again, I think you can see how this is very error-prone and probably doesn't fit any real-world scenarios... Low baud rates allow for more samples per bit to be collected and therefore better determine the signal frequency and account for potential errors due to noise or misalignment of the actual signal start. I glanced through the documents you linked and it seems you need to account for different start/stop bit logic compared to how my code works.

I tried to adapt it to that spec, but indeed the results I am getting are not what the spec suggests. I don't know why. Let me know if you need a copy of my sandbox changes to FSKConfig and FSKDecoder, although I couldn't get it to produce the desired results. My best result is something like:

AA F7 AF BD 77 F7 DD BF 75 BB 75 7B AF 5F B7 57 BB FB 77 FD 7F FF F7 F5 FB 7F FF F7 EB FF F7 DB 7F EF DF B7

Which is obviously wrong.

I wrote a basic Java code to debug the signal itself and also opened it in Audacity to look at the waveform.

The file "fsk_only_no_cs-8.wav" looks more promising, but the signal sequence doesn't look like what the spec suggests. I didn't trust my FSKDecoder, so I ran a simple Java code to apply a zero-crossings check on the whole thing and try to pinpoint a signal start and then just list a set of HIGH/LOW bits thereafter for me to compare with the documentation you've provided. It doesn't match... but it does match with what the FSKDecoder came up with, for the most part anyway.

import java.io.BufferedInputStream;
import java.io.ByteArrayOutputStream;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.util.Arrays;

public class Test {

    public static final String WAV_FILE = "fsk_only_no_cs-8.wav"; //"CID_sample-8.wav";

    protected static int calcFrequencyZerocrossing(byte[] data) {
        int numSamples = data.length;
        int numCrossing = 0;

        for (int i = 0; i < numSamples-1; i++) {
            if ((data[i] > 0 && data[i+1] <= 0)
                    || (data[i] < 0 && data[i+1] >= 0)) {

                numCrossing++;
            }
        }

        double numSecondsRecorded = (double) numSamples
                / (double) 44100;
        double numCycles = numCrossing / 2;
        double frequency = numCycles / numSecondsRecorded;

        return (int) Math.round(frequency);
    }

    public static void main(String[] args) {
        ByteArrayOutputStream out = new ByteArrayOutputStream();
        BufferedInputStream in = null;
        try {
            in = new BufferedInputStream(new FileInputStream(WAV_FILE));
        } catch (FileNotFoundException e) {
            e.printStackTrace();
            System.exit(1);
        }

        int read;
        byte[] buff = new byte[1024];

        try {
            while ((read = in.read(buff)) > 0) {
                out.write(buff, 0, read);
            }
            out.flush();
        }
        catch (Exception e) {
            e.printStackTrace();
            System.exit(1);
        }
        byte[] audioBytes = out.toByteArray();

        int samples_per_bit = 44100 / 1225; //sample rate / bitrate

        int freq_high = 1225;
        int freq_low = 2450;

        int signal_offset_start = 0;

        int bits_read = 0;

        for (int i = 0; i < audioBytes.length; i++) {
            byte[] slice = Arrays.copyOfRange(audioBytes, i, i + samples_per_bit);

            int freq = calcFrequencyZerocrossing(slice);

            if (freq == freq_low) {
                //low, possibly a start bit
                System.out.println("First LOW signal found at offset " + i);
                signal_offset_start = i;
                break;
            }
        }

        for (int i = signal_offset_start; i < audioBytes.length - samples_per_bit; i+=samples_per_bit) {
            byte[] slice = Arrays.copyOfRange(audioBytes, i, i + samples_per_bit);

            int freq = calcFrequencyZerocrossing(slice);

            if (freq >= freq_high && freq < (freq_high+freq_low)/2) {
                System.out.println("BIT "+bits_read+" IS HIGH");
            } else if (freq <= freq_low && freq > freq_high) {
                System.out.println("BIT "+bits_read+" IS LOW");
            } else {
                System.out.println("BIT "+bits_read+" IS "+freq);
            }

            bits_read++;
        }

        System.out.println(bits_read);

    }
}

This yields similar results to FSKDecoder.java, comparing the sequence of bits at least... still nowhere near the desired result...

The file "CID_sample-8.wav" has no chance of working imho, open it in Audacity and see how weird the modulation is. At some point in the signal you do get some series of frames with the correct frequency, but it's a mess overall.

Working on "fsk_only_no_cs-8.wav" I noticed that there are gaps in-between bits, which freaks out my decoder. instead of 1 stop bit you get a ton of stop bits before the next start bit arrives eventually. Or we could be reading the signal incorrectly. (most likely)

After looking at "Journal_hamici_jcs_si_72-76_2005.pdf" I think I know what's wrong.

Zero-crossings isn't the right way to calculate signal frequency in your case. You need fast fourier transform (FFT) as this is a multi-frequency signal with a carrier frequency of 1300hz. There's even a carrier detect signal which is longer than the data packet itself...

I poked around QuiFFT and came up with a sample code that almost matches the bits that the documentation suggests we should be getting. (according to this image: http://matthieu.benoit.free.fr/images/dsptab1.jpg )

import org.quifft.QuiFFT;
import org.quifft.output.FFTFrame;
import org.quifft.output.FFTResult;
import org.quifft.output.FrequencyBin;

import javax.sound.sampled.UnsupportedAudioFileException;
import java.io.IOException;
import java.util.Arrays;

public class TestFFT {

    public static final String WAV_FILE = "fsk_only_no_cs-8.wav"; //"CID_sample-8.wav";

    public static void main(String[] args) throws UnsupportedAudioFileException, IOException {
        FFTResult fft = new QuiFFT(WAV_FILE).windowSize(37).windowOverlap(0.0067568).numPoints(64).fullFFT();

        System.out.println(fft.windowDurationMs);
        System.out.println(fft.fftFrames.length);

        for (FFTFrame frame: fft.fftFrames) {
            FrequencyBin highest = Arrays.stream(frame.bins).sorted((o1, o2) -> o1.amplitude > o2.amplitude ? -1 : 1).findFirst().get();

            //System.out.println("Frame "+frame.frameStartMs+" highest bin "+highest.frequency+" "+highest.amplitude);

            if (highest.frequency >= 2000) {
                System.out.println("LOW");
            } else {
                System.out.println("HIGH");
            }
        }
    }
}

You need to add QuiFFT as a dependency for the above to compile. But that produced the closest possible result I managed to get. If you read the sequence of bits from here http://matthieu.benoit.free.fr/images/dsptab1.jpg and compare it to what the FFT black magic outputs it's almost perfect... almost... The stop bit appears to be doubled... meaning two stop bits instead of one... and some bits are flipped.. seemingly at random... I suspect my parameters are not correct and maybe if you poke a little more you could be able to make it work...

Perhaps I didn't init it correctly, or this is not how it is supposed to be used. I don't know. But that's the best I could achieve in few hours poking around.

The way I came up with the numbers in the builder is to calculate the number of samples you are supposed to have per bit (44100 / 1200, round up to the nearest whole integer) and then come up with a black magic overlap like 1 - (36.75 / 37). Probably completely deranged, not enough time to do proper research...

What I would suggest is to look at my code as a starting point idea, proof of concept if you will... and ideally start from scratch by examining your signal closely, even output the samples as numbers in a terminal, then chunk them into frames representing your baud rate derived sample size, feed those frames through some analyzing stage, zero-crossing or something else, ideally FFT, play with the parameters, get a feel of how things look and come up with your own approach eventually. This is how I did it back in the day. The most critical part for me was to determine where in the sea of numbers (samples) the signal actually starts. From there it gets easier, as you can hack together a simple state machine to parse the signal, once you have it.

Sorry I couldn't help more...

[veso266]

Hi there, you are helping more then u think, you give me hope to continue

I played around using your FFT example like this

package clid;

import org.quifft.QuiFFT;
import org.quifft.output.FFTFrame;
import org.quifft.output.FFTResult;
import org.quifft.output.FrequencyBin;

import javax.sound.sampled.UnsupportedAudioFileException;
import java.io.IOException;
import java.util.Arrays;

public class TestFFT {

    //public static final String WAV_FILE = "src/media/fsk_only_no_cs-8.wav";
    public static final String WAV_FILE = "src/media/CID_sample-8.wav"; 
    //public static final String WAV_FILE = "src/media/fsk_10010110-8bit.wav"; 

    public static void main(String[] args) throws UnsupportedAudioFileException, IOException {
        FFTResult fft = new QuiFFT(WAV_FILE).windowSize(37).windowOverlap(0.0067568).numPoints(64).fullFFT();
        //FFTResult fft = new QuiFFT(WAV_FILE).windowSize(37).windowOverlap(0.0067568).numPoints(4096).fullFFT(); //This just takes more time to finish, it doesn't get me any extra frequencies for some reason

        //System.out.println(fft.windowDurationMs);
        //System.out.println(fft.fftFrames.length);

        for (FFTFrame frame: fft.fftFrames) {
            FrequencyBin highest = Arrays.stream(frame.bins).sorted((o1, o2) -> o1.amplitude > o2.amplitude ? -1 : 1).findFirst().get();

            int frequency = (int)highest.frequency;

                //System.out.println(frequency);
            if (frequency == 1378 || frequency == 2067) {
                if (frequency == 1378) {
                    System.out.println("1");
                    DebugTools.WriteBit(true, "src/media/bits.txt");
                }
                if (frequency == 2067) {
                    System.out.println("0");
                    DebugTools.WriteBit(false, "src/media/bits.txt");
                }
            }
        }
    }
}

what was wierd is that no matter what I tried the frequency never within the mark and space parameters

Logical 1 (mark)            1200 +/- 12 Hz
Logical 0 (space)           2200 +/- 22 Hz

So after staring at the frequencies in terminal that FFT gave me for some time, I noticed that 1378 was closest to 1200 and 2067 was closest to 2200 so I decided this will be my mark and space frequencies

and managed to get out this

11110010001101111001011010000010100010010110011001011001100111011110111101011010110101101010010101010101110100110101011110101111101011110101111110111111011111101101110101111101111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111100000000111111100110111110000000110001000100000100110110011000001100110111011001100000110011000111011100001110011110110110110000001101010001100010010111011000011001101000100110010011001101101101110011111011111110110111011111011111111011111100000110101000011011001010110101110110101101101000110101011010000010110101001001111010110011101011001001101101100111011001000011111100011001110010111101011000101101111011000

the brrrr you hear at the beginning is just a ring signal (it shouldn't interfere the decoder since its lower frequency, why its brrr and not 20khz sine wave is a story for itself (when u capture this on a phone line you get 20khz signal 90V peak to peak, which is to high for the sound card to handle (maximum 2V) so a diode cuts everything above 2V so you are left with half cut sinewave as a ring signal)

The way I understand how this callerID works is that at first there is Channel Seizure (1010101010101010...... (300 bits) which wakes the telephone receiver like an alarm clock wakes a human then there is Mark signal (1111111111111111.... (180 bits) which prepares telephone receiver for work like a cup of coffie prepares human for work)

then after mark signal after u receive start bit (0) you imidiatly start decoding and dealing with frames (start bit, data, stop bit)

but looking at what I got I can see Mark signal fine, but channel seizure looks all wierd, instead of looking like 101010... its like 11110010001101 which doesn't look right, but ok said, will try to get the frames by hand

0000000011 -- first frame
11111??????
0011011111 -- second frame
0000000110??
0010001000??
00100110110011000001100110111011001100000110011000111011100001110011110110110110000001101010001100010010111011000011001101000100110010011001101101101110011111011111110110111011111011111111011111100000110101000011011001010110101110110101101101000110101011010000010110101001001111010110011101011001001101101100111011001000011111100011001110010111101011000101101111011000

but even the frames don't look right, some have 2 stop bits, others have 1 stop bit, others have no stop bits or 1 as start bit

so I thought, hmm, what if my sample signal is broken, I don't even know whats inside of it or how it looks like demodulated

So I decided to search for a new sample (since I sadly do not yet have equpment to capture the real phone line)

and I found this: https://forum.arduino.cc/t/esp8266-arduino-telephone-caller-id-system-with-anti-spam-feature/507603 it is a software callerID demodulator, and best of all it has a sample that I know is good and whats inside

MDMF format and contains the number 0800807111 date 2203 and time 0851

and this document: https://forum.arduino.cc/uploads/short-url/yiDh3lvOs56pnACveoCno2APdCP.pdf on Page 13 (9 DATA FORMATS) the sample is even demodulated so it can be compared how demodulation is doing

I even managed to find the FSK sample generator: https://forum.arduino.cc/t/telephone-caller-id-afsk-generator/575844 and some more details on the demodulated frame: https://forum.arduino.cc/uploads/short-url/x6Qq1WITisfLSaYvwUENtvRyb7N.pdf

after a bit more digging I also managed to find this decoder written in c: https://github.com/jeniljain29392/FSK-CallerID-decoding/blob/master/fskmodem.c which also includes a bit more documentation on how it works (sadly its using a lot of DSP magic to detect mark and space frequencies): https://github.com/jeniljain29392/FSK-CallerID-decoding/blob/master/Documentation/CID_documenations.odt

sadly I can not compile it because I am using windows and have no acsess to a linux enviroment right now and compiling this for windows seams a project for itself :)

but I realy liked that it has a similar state machine to your library so I tried to rewrite the demodulator into java

package si.wolf;

public class FilterCoefficients {
    // C coefficient for 3rd order Bandpass Filter
    public static int[] c_coef_3rd_bp = { -1, 0, 3, 0, -3, 0, 1 };

    // C coefficient for 6th order Lowpass Filter
    public static int[] c_coef_6th_lp = { 1, 6, 15, 20, 15, 6, 1 };

    // Zeros for initializing xv and yv array
    public static int[] zeros = { 0, 0, 0, 0, 0, 0, 0 };

    /************************************************************************************************/
    /**************************************** 44100 Hz **********************************************/
    /************************************************************************************************/

    /*********************************  MARK BANDPASS  ********************************/

    //brief Coefficients for Mark filters for 1200Hz and 44.1 Khz sampling
    public static double gain_1200hz = 6.035447139e+03;
    public static double[] d_coef_1200hz = {-0.7960564529, 4.8853503505, -12.5704962310, 17.3572300720, -13.5639614690, 5.6879115366, 1.0};

    //brief Coefficients for Mark filters for 1300Hz and 44.1 Khz sampling
    public static double gain_1300hz = 6.035463052e+03;
    public static double[] d_coef_1300hz = {-0.7960564529, 4.8728234960, -12.5192737550, 17.2778325650, -13.5086884650, 5.6733267812, 1.0};

    /*********************************  SPACE BANDPASS  *******************************/

    /**@brief Coefficients for Space filters for 2100Hz and 44.1 Khz sampling
     *
     * Filter Design parameters
     * @param filtertype Butterworth
     * @param passtype Bandpass
     * @param order 3
     * @param sample_rate 44100
     * @param corner1 1737.756
     * @param corner2 2537.756
     */
    public static double gain_2100hz = 6.035489699e+03;
    public static double[] d_coef_2100hz = {-0.7960564529, 4.7372163175, -11.9731917700, 16.4353856510, -12.9194238570, 5.5154421711, 1.0};

    /**@brief Coefficients for Space filters for 2200Hz and 44.1 Khz sampling
     *
     * Filter Design parameters
     * @param filtertype Butterworth
     * @param passtype Bandpass
     * @param order 3
     * @param sample_rate 44100
     * @param corner1 1737.756
     * @param corner2 2537.756
     */
    public static double gain_2200hz = 6.035490118e+03;
    public static double[] d_coef_2200hz = {-0.7960564529, 4.7158930488, -11.8887264470, 16.3057248700, -12.8282792550, 5.4906159340, 1.0};

    /**********************************  LOW PASS  ************************************/

    /**@brief Coefficients for Low Pass filter for 1700Hz and 44.1 Khz sampling
     *
     * Since 1700 is center of both ranges 1200-2200 and 1300-2100
     *
     * Filter Design parameters
     * @param filtertype Butterworth
     * @param passtype Low Pass
     * @param order 6
     * @param sample_rate 44100
     * @param corner1 1700
     */
    public static double gain_lp_1700hz = 4.921530591e+05;
    public static double[] d_coef_lp_1700hz = {-0.3911214670, 2.7125767203, -7.8710687721, 12.2354456210, -10.7505803000, 5.0646181573, 1.0};

    /************************************************************************************************/
    /**************************************** 48000 Hz **********************************************/
    /************************************************************************************************/

    /*********************************  MARK BANDPASS  ********************************/

    /**@brief Coefficients for Mark filters for 1200Hz and 48 Khz sampling
     *
     * Filter Design parameters
     * @param filtertype Butterworth
     * @param passtype Bandpass
     * @param order 3
     * @param sample_rate 48000
     * @param corner1 864.911
     * @param corner2 1664.911
     */
    public static double gain_1200hz_48 = 7.714531525e+03;
    public static double[] d_coef_1200hz_48 = {-0.8109608719, 4.9734833008, -12.7760535460, 17.5948994070, -13.7006793120, 5.7192975419, 1.0};

    /**@brief Coefficients for Mark filters for 1300Hz and 48 Khz sampling
     *
     * Filter Design parameters
     * @param filtertype Butterworth
     * @param passtype Bandpass
     * @param order 3
     * @param sample_rate 48000
     * @param corner1 960.147
     * @param corner2 1760.147
     */
    public static double gain_1300hz_48 = 7.714552008e+03;
    public static double[] d_coef_1300hz_48 = {-0.8109608719, 4.9627363056, -12.7321344850, 17.5269785470, -13.6535802120, 5.7069389474, 1.0};

    /*********************************  SPACE BANDPASS  *******************************/

    /**@brief Coefficients for Space filters for 2100Hz and 48 Khz sampling
     *
     * Filter Design parameters
     * @param filtertype Butterworth
     * @param passtype Bandpass
     * @param order 3
     * @param sample_rate 48000
     * @param corner1 1737.756
     * @param corner2 2537.756
     */
    public static double gain_2100hz_48 = 7.714586427e+03;
    public static double[] d_coef_2100hz_48 = {-0.8109608719, 4.8463050158, -12.2624122920, 16.8034849110, -13.1498469240, 5.5730478395, 1.0};

    /**@brief Coefficients for Space filters for 2200Hz and 48 Khz sampling
     *
     * Filter Design parameters
     * @param filtertype Butterworth
     * @param passtype Bandpass
     * @param order 3
     * @param sample_rate 48000
     * @param corner1 1737.756
     * @param corner2 2537.756
     */
    public static double gain_2200hz_48 = 7.714586976e+03;
    public static double[] d_coef_2200hz_48 = {-0.8109608719, 4.8279816404, -12.1895052890, 16.6916642700, -13.0716609570, 5.5519767250, 1.0};

    /**********************************  LOW PASS  ************************************/

    /**@brief Coefficients for Low Pass filter for 1700Hz and 48 Khz sampling
     *
     * Since 1700 is center of both ranges 1200-2200 and 1300-2100
     *
     * Filter Design parameters
     * @param filtertype Butterworth
     * @param passtype Low Pass
     * @param order 6
     * @param sample_rate 48000
     * @param corner1 1700
     */
    public static double gain_lp_1700hz_48 = 7.911546151e+05;
    public static double[] d_coef_lp_1700hz_48 = {-0.4222972557, 2.8967245878, -8.3083998149, 12.7576882370, -11.0643521500, 5.1405555009, 1.0};

    /************************************************************************************************/
    /**************************************** 16000 Hz **********************************************/
    /************************************************************************************************/

    /*********************************  MARK BANDPASS  ********************************/

    public static double gain_1200hz_16 = 3.450407729e+02;
    public static double[] d_coef_1200hz_16 = {-0.5320753683, 3.1381363581, -8.1321890445, 11.7763561920, -10.0414792500, 4.7835061525, 1.0};

    public static double gain_1300hz_16 = 7.714552008e+03;
    public static double[] d_coef_1300hz_16 = {-0.8109608719, 4.9627363056, -12.7321344850, 17.5269785470, -13.6535802120, 5.7069389474, 1.0};

    public static double gain_2100hz_16 = 7.714586427e+03;
    public static double[] d_coef_2100hz_16 = {-0.8109608719, 4.8463050158, -12.2624122920, 16.8034849110, -13.1498469240, 5.5730478395, 1.0};

    public static double gain_2200hz_16 = 3.450423755e+02;
    public static double[] d_coef_2200hz_16 = {-0.5320753683, 2.2795472690, -5.2023382825, 7.1423622606, -6.4198613572, 3.4747465189, 1.0};

    /**********************************  LOW PASS  ************************************/

    public static double gain_lp_1700hz_16 = 2.155293674e+03;
    public static double[] d_coef_lp_1700hz_16 = {-0.0712022265, 0.6102683340, -2.2329558363, 4.4846591081, -5.2494746975, 3.4290109889, 1.0};
}

package si.wolf;

class FSKData
{
    public static int NZEROS_POLES = 6;
    class FilterStruct
    {
        public double[] xv;
        public double[] yv;
        public double[] c_coef;
        public double[] d_coef;
        public double Gain;
    }

    public int nbit;                               //< Number of Data Bits (5,7,8)
    public int parity;                             //< Parity 0=none 1=even 2=odd
    public int instop;                             //< Number of Stop Bits
    public int ispb;                               //< Sample per FSK bit (Sample_rate/baud_rate)
    public int fsk_std;                            //< Type of FSK standard

    public int xi0;                                //< current demodulated value
    public int xi1;                                //< previous demodulated value
    public int xi2;                                //< previous to previous demodulated value

    public int state;                              //< Demodulation state

    public int pllispb;                            //< Pll autosense
    public int pllids;                             //< PLL adjustment
    public int pllispb2;                           //< Center of the PLL
    public int icont;                              //< Count for DPLL
    public int pll_round_off;

    public FilterStruct mark_filter;       ///< Structure to store mark filter data
    public FilterStruct space_filter;      ///< Structure to store space filter data
    public FilterStruct demod_filter;      ///< Structure to store demodulator filter data
}

package si.wolf;

class FSKData
{
    public static int NZEROS_POLES = 6;
    class FilterStruct
    {
        public double[] xv;
        public double[] yv;
        public double[] c_coef;
        public double[] d_coef;
        public double Gain;
    }

    public int nbit;                               //< Number of Data Bits (5,7,8)
    public int parity;                             //< Parity 0=none 1=even 2=odd
    public int instop;                             //< Number of Stop Bits
    public int ispb;                               //< Sample per FSK bit (Sample_rate/baud_rate)
    public int fsk_std;                            //< Type of FSK standard

    public int xi0;                                //< current demodulated value
    public int xi1;                                //< previous demodulated value
    public int xi2;                                //< previous to previous demodulated value

    public int state;                              //< Demodulation state

    public int pllispb;                            //< Pll autosense
    public int pllids;                             //< PLL adjustment
    public int pllispb2;                           //< Center of the PLL
    public int icont;                              //< Count for DPLL
    public int pll_round_off;

    public FilterStruct mark_filter;       ///< Structure to store mark filter data
    public FilterStruct space_filter;      ///< Structure to store space filter data
    public FilterStruct demod_filter;      ///< Structure to store demodulator filter data
}

public class FSKModem
{
    private static final boolean VERBOSE = false; //Used for debugging

    private static final int SCALE = 25000000;
    private static final int STATE_SEARCH_STARTBIT = 0;
    private static final int STATE_SEARCH_STARTBIT2 = 1;
    private static final int STATE_CHANNEL_SEIZURE = 2;
    private static final int STATE_MARK_SIGNAL = 3;
    private static final int STATE_GET_DATA_FRAME = 4;

    private static int count = 0;

    private int iget_sample(short[] buffer, int len)
    {
        int retval = (int)buffer[0];
        System.arraycopy(buffer, 1, buffer, 0, len - 1);
        len--;
        return retval;
    }

    private int Filter(FSKData.FilterStruct fs, int input)
    {
        fs.xv[0] = fs.xv[1];
        fs.xv[1] = fs.xv[2];
        fs.xv[2] = fs.xv[3];
        fs.xv[3] = fs.xv[4];
        fs.xv[4] = fs.xv[5];
        fs.xv[5] = fs.xv[6];
        fs.xv[6] = input / fs.Gain;
        fs.yv[0] = fs.yv[1];
        fs.yv[1] = fs.yv[2];
        fs.yv[2] = fs.yv[3];
        fs.yv[3] = fs.yv[4];
        fs.yv[4] = fs.yv[5];
        fs.yv[5] = fs.yv[6];
        fs.yv[6] = (fs.c_coef[0] * fs.xv[0]) + (fs.c_coef[1] * fs.xv[1]) +
                (fs.c_coef[2] * fs.xv[2]) + (fs.c_coef[3] * fs.xv[3]) +
                (fs.c_coef[4] * fs.xv[4]) + (fs.c_coef[5] * fs.xv[5]) +
                (fs.c_coef[6] * fs.xv[6]) +
                (fs.d_coef[0] * fs.yv[0]) + (fs.d_coef[1] * fs.yv[1]) +
                (fs.d_coef[2] * fs.yv[2]) + (fs.d_coef[3] * fs.yv[3]) +
                (fs.d_coef[4] * fs.yv[4]) + (fs.d_coef[5] * fs.yv[5]);
        return (int)fs.yv[6];
    }

    int IDemodulatorReturn = -1;
    private int IDemodulator(FSKData fskd/*, int retval*/, int x)
    {
        int isValue = Filter(fskd.space_filter, x);
        int imValue = Filter(fskd.mark_filter, x);

        double scale = SCALE;
        int ilin2 = (int)((Math.pow(isValue, 2) - Math.pow(imValue, 2)) / scale);
        int idValue = Filter(fskd.demod_filter, ilin2);

    if (VERBOSE) {
        System.out.printf("IGET_SAMPLE: %8d, \tSpace: %8d, \tMark: %8d, \tIlin: %8d, \tID: %8d\n",
                x, isValue, imValue, ilin2, idValue);
    }
        IDemodulatorReturn = idValue; //was retval since java does not pass primitive types by reference (it does pass object by reference)
        return 0;
    }

    public int Init(FSKData fskd)
    {
        for (int i = 0; i <= FSKData.NZEROS_POLES; i++)
        {

            //Init stuff
            fskd.mark_filter.xv = new double[FSKData.NZEROS_POLES + 1];
            fskd.mark_filter.yv = new double[FSKData.NZEROS_POLES + 1];
            fskd.mark_filter.c_coef = new double[FSKData.NZEROS_POLES + 1];
            fskd.mark_filter.d_coef = new double[FSKData.NZEROS_POLES + 1];

            fskd.space_filter.xv = new double[FSKData.NZEROS_POLES + 1];
            fskd.space_filter.yv = new double[FSKData.NZEROS_POLES + 1];
            fskd.space_filter.c_coef = new double[FSKData.NZEROS_POLES + 1];
            fskd.space_filter.d_coef = new double[FSKData.NZEROS_POLES + 1];

            fskd.demod_filter.xv = new double[FSKData.NZEROS_POLES + 1];
            fskd.demod_filter.yv = new double[FSKData.NZEROS_POLES + 1];
            fskd.demod_filter.c_coef = new double[FSKData.NZEROS_POLES + 1];
            fskd.demod_filter.d_coef = new double[FSKData.NZEROS_POLES + 1];

            fskd.mark_filter.xv[i] = 0;
            fskd.mark_filter.yv[i] = 0;
            fskd.mark_filter.c_coef[i] = FilterCoefficients.c_coef_3rd_bp[i];

            fskd.space_filter.xv[i] = 0;
            fskd.space_filter.yv[i] = 0;
            fskd.space_filter.c_coef[i] = FilterCoefficients.c_coef_3rd_bp[i];

            fskd.demod_filter.xv[i] = 0;
            fskd.demod_filter.yv[i] = 0;
            fskd.demod_filter.c_coef[i] = FilterCoefficients.c_coef_6th_lp[i];
            fskd.demod_filter.d_coef[i] = FilterCoefficients.d_coef_lp_1700hz[i];
            fskd.demod_filter.Gain = FilterCoefficients.gain_lp_1700hz;

            if (fskd.fsk_std == 0)
            {
                fskd.mark_filter.d_coef[i] = FilterCoefficients.d_coef_1200hz[i];
                fskd.mark_filter.Gain = FilterCoefficients.gain_1200hz;

                fskd.space_filter.d_coef[i] = FilterCoefficients.d_coef_2200hz[i];
                fskd.space_filter.Gain = FilterCoefficients.gain_2200hz;
            }
            else if (fskd.fsk_std == 1)
            {
                fskd.mark_filter.d_coef[i] = FilterCoefficients.d_coef_1300hz[i];
                fskd.mark_filter.Gain = FilterCoefficients.gain_1300hz;

                fskd.space_filter.d_coef[i] = FilterCoefficients.d_coef_2100hz[i];
                fskd.space_filter.Gain = FilterCoefficients.gain_2100hz;
            }
            else
            {
                System.out.println("Undefined FSK standard.");
                return 1;
            }
        }
        return 0;
    }

    public static int GetBitRaw(FSKData fskd, short[] buffer, int len)
    {
        int f;
        int currentSample;
        int ix = 0;
        int roundoff = 0;

        if (len < (fskd.ispb + 2)) // Minimum samples required
            return -1;

if (VERBOSE) {
    System.out.print("\n\n");
}

        for (f = 0; ;) // DPLL loop
        {
            currentSample = iget_sample(buffer, len);
            //IDemodulator(fskd, out ix, currentSample); // Check current sample

            // Checks for the transition
            if ((ix >= 0 && fskd.xi0 < 0) || (ix < 0 && fskd.xi0 >= 0))
            {
                if (f == 0)
                {
                    if (fskd.icont < (fskd.pllispb2))
                    {
                        fskd.icont += fskd.pllispb2; // Increases DPLL counter
                    }
                    else
                    {
                        fskd.icont -= fskd.pllids; // Decreases DPLL counter
                    }
                    f = 1; // DPLL is adjusted just once
                    roundoff = 0; // no need when there's a transition
                }
            }
            fskd.xi0 = ix;
            fskd.icont += 32;

            if (fskd.icont > fskd.pllispb) // Exit the current DPLL loop
            {
                fskd.icont -= fskd.pllispb; // Reset the counter

                // Sampling of the bit is done with an integer value of ispb.
                // But the result of sample_rate/baud_rate is a float value.
                // So to keep the PLL synced when there is no transition for a long time,
                // we use this roundoff when the ispb is very less.

                //if (f == 0) roundoff++;
                //if (roundoff == fskd.PllRoundOff)
                //{
                //    currentSample = IGetSample(ref buffer, ref len);
                //    IDemodulator(fskd, out ix, currentSample);
                //    roundoff = 0;
                //    Console.WriteLine("here jjjjjjj");
                //}
                break;
            }
        }

if (VERBOSE) {
    System.out.println("\n[" + ((fskd.state == 2) ? "CHANNEL SEIZURE" :
            ((fskd.state == 3) ? "MARK SIGNAL" : "DATA FRAME")) + "], The bit is");
}

        // Presentation
        if (count % 30 == 0)
            System.out.print("\n");

        f = (ix < 0) ? 0x80 : 0; // differentiate Mark and Space
        // based on demodulator value
        System.out.println("\t" + ((f != 0) ? 1 : 0)); // Print 1 or 0
        return f;
    }

    /**@brief Detecting channel seizure signal.
     *
     * The Caller ID message starts with a Channel seizure signal which
     * consists 300 bits containing alternate 1s and 0s.
     * Checking if we detected the channel seizure signal properly.
     *
     * @param fskd Address of pointer to FSK data structure
     * @param buffer Address of pointer to current sample
     * @param len Address of pointer containing pointer of length
     *
     * @return 0 if successful else -1 if error
     */
    public static int GetChannelSeizure(FSKData fskd, short[] buffer, int len)
    {
        int one_zero = 1;
        int olen;
        int res;

        while (len > 0)
        {
            olen = len;
            res = GetBitRaw(fskd, buffer, len); // Getting the current data bit

            if (res == -1)
                return 0; // Number of samples is less than 40
            else if (res != 0)
                one_zero++; // Incrementing for mark signal
            else
                one_zero--; // Incrementing for space

            // Update buffer pointer based on the consumed samples
            buffer = GetUpdatedBuffer(buffer, olen, len);

            if ((count > 295) && (one_zero > 1))
                return count; // We have already detected the starting bit, so count is 298.
                // Sometimes there are more than 300 alternate 1s and 0s.
                // We wait until we get 2 consecutive 1's.

            else if (one_zero < 0 || one_zero > 1)
            {
                System.out.print("\n\nFailed to detect alternate 1s and 0s\n\n");
                count = 0;
                one_zero = 1;

                // Add any necessary cleanup logic or logging
                return -1;
            }

            count++;
        }

        return 0;
    }

    /**@brief Detecting Mark signal.
     *
     * After the channel seizure, there is a sequence of 180 Mark signals.
     * The purpose of the Channel Seizure Signal and the Mark Signal is to
     * prepare the data receiver in the Customer Premise Equipment (CPE)
     * for the reception of the actual CID message
     *
     * @param fskd Address of pointer to FSK data structure
     * @param buffer Address of pointer to current sample
     * @param len Address of pointer containing pointer of length
     *
     * @return 0 if successful else -1 if error
     */
    public static int GetMarkSignal(FSKData fskd, short[] buffer, int len)
    {
        int olen;
        int res;

        while (len > 0)
        {
            olen = len;
            res = GetBitRaw(fskd, buffer, len); // Getting the current data bit

            if (res == -1)
                return 0; // Number of samples is less than 40
            else if (res != 0)
            {
                count++; // Incrementing for mark signal
                buffer = GetUpdatedBuffer(buffer, olen, len);
            }
            else if ((count > 160) && (res == 0))
            {
                len = olen; // The number of consecutive mark signals can vary.
                // So we wait until we get the start bit of the data frame.
                // But we decrement so that we can start again from the start bit.
                return count;
            }
            else
            {
                System.out.print("\n\nFailed to detect Mark signal\n\n");
                count = 0;
                return -1;
            }
        }
        return 0;
    }

    /**@brief Get the data frame.
     *
     * Data bytes starts after the Mark signal.
     * The message consists of a sequence of 10-bit frames. A start bit(0),
     * 8 bits of information(LSB to MSB) and then a stop bit(1).
     *
     * @param fskd Address of pointer to FSK data structure
     * @param buffer Address of pointer to current sample
     * @param len Address of pointer containing pointer of length
     *
     * @return 0 if successful else -1 if error
     */
    public static int GetDataFrame(FSKData fskd, short[] buffer, int len)
    {
        int a;
        int i, j, n1;
        int olen = len;

        if (GetBitRaw(fskd, buffer, len) == 0)
        {
            buffer = GetUpdatedBuffer(buffer, olen, len);
            j = fskd.nbit;

            for (a = n1 = 0; j > 0; j--)
            {
                olen = len;
                i = GetBitRaw(fskd, buffer, len);
                buffer = GetUpdatedBuffer(buffer, olen, len);

                if (i == -1)
                    return 0; // Number of samples is less than 40

                a >>= 1; // Shifting the bit because bits are transmitted from LSB to MSB
                a |= i;
            }
        }
        else
        {
            System.out.print("\n\n\nStart bit of Data Bytes not Found\n");
            return -1;
        }

        for (i = 0; i < fskd.instop; i++)
        {
            olen = len;
            if (GetBitRaw(fskd, buffer, len) == 0)
            {
                System.out.print("\n\n\nStop bit of Data Bytes not Found\n");
                return -1;
            }
            buffer = GetUpdatedBuffer(buffer, olen, len);
        }

        return a;
    }

    /**@brief Retrieve a serial byte into outbyte.
     *
     * Buffer is a pointer into a series of
     * shorts and len records the number of bytes in the buffer.  len will be
     * overwritten with the number of bytes left that were not consumed.
     *
     * @return return value is as follows:
     * @arg 0: Still looking for something...
     * @arg 1: An output byte was received and stored in outbyte
     * @arg -1: An error occured in the transmission
     */
    int FSKSerialOutByte = 0;
    public int FSKSerial(FSKData fskd, short[] buffer, int len/*,int outbyte*/)
    {
        int i;
        int samples = 0;
        int res;
        int currentSample;

        // Pick up where we left off
        switch (fskd.state)
        {
            case STATE_SEARCH_STARTBIT:

                // Detecting the start of space in the channel seizure.
                // The bandpass value is high when there is some rise in the amplitude, otherwise, it's 0.

                System.out.print("\nSearching for the start bit...\n");
                while (len > 0)
                {
                    currentSample = iget_sample(buffer, len);
                    IDemodulator(fskd/*, fskd.xi2*/, currentSample);
                    fskd.xi2 = IDemodulatorReturn;
                    samples++;

                    // Threshold to detect the start of the FSK data
                    if (fskd.xi2 < 0)
                    {
                        fskd.state = STATE_SEARCH_STARTBIT2;
                        break;
                    }
                }
                break;

            case STATE_SEARCH_STARTBIT2:
            {
                // After Detecting the rise in the waveform, we get to the one-third point
                // of the bit, i.e. reaching 0.33 of the bit.

                System.out.print("\nGetting to the center of the bit...\n");
                i = fskd.ispb / 2;
                for (; i > 0; i--)
                {
                    currentSample = iget_sample(buffer, len);
                    IDemodulator(fskd/*, fskd.xi1*/, currentSample);
                    fskd.xi1 = IDemodulatorReturn;
                    samples++;
                }
                fskd.state = STATE_CHANNEL_SEIZURE;
                System.out.print("\nEntering channel seizure...\n\n");
                break;
            }
            case STATE_CHANNEL_SEIZURE:
            {
                // Detecting Channel seizure
                res = GetChannelSeizure(fskd, buffer, len);
                if (res == -1)
                {
                    fskd.state = STATE_SEARCH_STARTBIT;
                    break;
                }
                else if (res != 0)
                {
                    count = 0;
                    System.out.print("\n\nChannel seizure detected with {res} alternate 1s and 0s.\n");
                    System.out.print("\nDetecting Mark signal... \n");
                    fskd.state = STATE_MARK_SIGNAL;
                }
                break;
            }
            case STATE_MARK_SIGNAL:
            {
                // Detecting Mark signal
                res = GetMarkSignal(fskd, buffer, len);
                if (res != 0)
                {
                    System.out.print("\n\nMark signal detected with {res} consecutive 1s\n");
                    System.out.print("\nGetting Caller ID message data bytes...\n");
                    System.out.print("\nSTART\t\t\t\t8 data bits\t\t\tSTOP\n");
                    fskd.state = STATE_GET_DATA_FRAME;
                }
                break;
            }

            case STATE_GET_DATA_FRAME:
            {
                // For demodulating a byte we require 10 or 11 bits. 12 for the safer side
                if (len > (fskd.ispb * 12))
                {
                    //outbyte = GetDataFrame(fskd, buffer, len);
                    //was outbyte since java does not pass primitive types by reference (it does pass object by reference)
                    FSKSerialOutByte = GetDataFrame(fskd, buffer, len);
                    return 1;
                }
                break;
            }

            default:
                System.out.print("\n\tInvalid State in FSK demodulation\n");
                break;
        }
        return 0;
    }

    private static short[] GetUpdatedBuffer(short[] buffer, int olen, int len)
    {
        short[] newBuffer = new short[len];
        System.arraycopy(buffer, olen - len, newBuffer, 0, len);
        return newBuffer;
    }
}

So yea, I am one step further (I think :smile:)

I hope this frenkenstein fskdemodulator port will work, tried my best to rewrite it into java, since it uses c way of thinking (by passing pointers between functions)

So sorry to bother you like that, but you are the onlyone I can ask this stuff (since stackoverflow usualy does not help at all if u do very wierd and specific things :smile:)

Have to go to sleep now, since my brain cannot think strait anymore, well good night

[iganev]

I am happy to hear that you are making progress!

I will try to find some more time to look into that as it grabbed my attention and I hate to leave an unsolved puzzle...

The difference in what frequencies you see vs what you expect could be because of several factors, probably more than one at play here:

1. Plain old signal distortion due to sample rate discrepancies
2. Wrong setup, meaning we might not be using QuiFFT as it is meant to be used
3. The sample frame might be misaligned due to the fact that we haven't reliably pinpointed where the signal starts and the exact number of samples per bit. That could be shifting the sample window back and forth and could be producing both the frequency mismatch and the extra bits and/or flipped bits that we see.

If I manage to find some time I would try to make a Frankenstein monster out of my FSKDecoder and a simple FFT instead of Zerocrossing. Using the FSKDecoder existing logic to scan for the signal start and cut sample frames to pass to the Zerocrossing check for analysis, but instead of Zerocrossing I will stick a FFT there, using it on a single frame only, not the whole file. Might be worth the try.

I took a look at the C library, they use fancy voodoo magic to apply low-pass and high-pass filters to the signal, effectively drowning out frequencies they are not interested in. That's well above my intellectual level for the time being.

Let me know if you have a breakthrough! :smile:

Happy Holidays!

[veso266]

Hi there, after reading here: https://stackoverflow.com/a/29133798/3368585 that FFT might not be the right tool for this since it doesn't have the time resolution required so maybe thats the reason we don't get the right frequencies and flipped bits at random

so after some thought, I remembered that there is also Goertzel algorithm to detect frequencies in signals its usualy used to detect DTMF frequencies but detecting FSK should be even easier since we only have 2 frequencies to deal with and it seams also this: https://hackaday.io/project/170710-esp32-tnc-and-audio-relay-for-hfvhf-packet-radio/log/176478-researching-fsk-demodulation-methods confirms that FFT is not suited for FSK demodulation

so after looking at some code: https://github.com/tino1b2be/DTMF-Decoder/blob/master/source/com/tino1b2be/dtmfdecoder/DTMFUtil.java#L485 and here: https://stackoverflow.com/a/34137832/3368585

I managed to modify FSKDecoder.java like this (added function calcFrequencyGoertzel )

/**
 *   This file is part of the FSKModem java/android library for
 *   processing FSK audio signals.
 *
 *   The FSKModem library is developed by Ivan Ganev, CEO at
 *   Cytec BG Ltd.
 *
 *   Copyright (C) 2014  Cytec BG Ltd. office@cytec.bg
 *
 *   This program is free software: you can redistribute it and/or modify
 *   it under the terms of the GNU General Public License as published by
 *   the Free Software Foundation, either version 3 of the License, or
 *   (at your option) any later version.
 *
 *   This program is distributed in the hope that it will be useful,
 *   but WITHOUT ANY WARRANTY; without even the implied warranty of
 *   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 *   GNU General Public License for more details.
 *
 *   You should have received a copy of the GNU General Public License
 *   along with this program.  If not, see <http://www.gnu.org/licenses/>.
 */

package si.wolf;

import java.nio.ByteBuffer;
import java.nio.ShortBuffer;
import java.util.*;
import java.util.stream.Collectors;

public class FSKDecoder {

    public interface FSKDecoderCallback {
        /**
         * This method will be called every time there is new data received
         * @param newData
         */
        public void decoded(byte[] newData);
    }

    /**
     * Decoding state-driven thread
     */
    protected Runnable mProcessor = new Runnable() {

        @Override
        public void run() {

            while (mRunning) {

                synchronized (mSignal) {

                    switch (mDecoderStatus) {
                        case IDLE:
                            stop();

                            break;
                        case SEARCHING_SIGNAL:case SEARCHING_START_BIT:
                            processIterationSearch();

                            break;

                        case DECODING:
                            processIterationDecode();

                            break;

                    }
                }
            }
        }
    };

    ///

    protected enum STATE {
        HIGH, LOW, SILENCE, UNKNOWN
    }

    protected enum DecoderStatus {
        IDLE, SEARCHING_SIGNAL, SEARCHING_START_BIT, DECODING
    }

    // /

    protected FSKConfig mConfig;

    protected FSKDecoderCallback mCallback;

    protected Thread mThread;

    protected boolean mRunning = false;

    protected DecoderStatus mDecoderStatus = DecoderStatus.IDLE;

    protected DecoderStatus mDecoderStatusPaused = DecoderStatus.IDLE;

    // /

    protected ShortBuffer mSignal;

    protected ShortBuffer mFrame;

    protected StringBuffer mBitBuffer; // concat 0 and 1 to build a binary representation of a byte

    protected int mCurrentBit = 0;

    protected int mSignalEnd = 0; // where is the end of the signal in mSignal

    protected int mSignalPointer = 0; //where is the current processing poing in mSignal

    protected int mSignalBufferSize = 0;

    // /

    protected ByteBuffer mData;

    protected int mDataLength = 0;

    // /

    /**
     * Create FSK Decoder instance to feed with audio data
     * @param config
     * @param callback
     */
    public FSKDecoder(FSKConfig config, FSKDecoderCallback callback) {
        mConfig = config;

        mCallback = callback;

        mSignalBufferSize = mConfig.sampleRate; // 1 second buffer

        allocateBufferSignal();

        allocateBufferFrame();

        allocateBufferData();
    }

    @Override
    protected void finalize() throws Throwable {

        stop();

        super.finalize();
    }

    // /

    protected void notifyCallback(byte[] data) {
        if (mCallback != null) {
            mCallback.decoded(data);
        }
    }

    protected void start() {
        if (!mRunning) {
            if (!mDecoderStatusPaused.equals(DecoderStatus.IDLE)) {
                setStatus(mDecoderStatusPaused); //resume task
            }
            else {
                setStatus(DecoderStatus.SEARCHING_SIGNAL); //start new process
            }

            mRunning = true;

            mThread = new Thread(mProcessor);
            mThread.setPriority(Thread.MIN_PRIORITY);
            mThread.start();
        }
    }

    /**
     * Stop decoding process
     */
    public void stop() {
        if (mRunning) {
            if (mThread != null && mThread.isAlive()) {
                mRunning = false;

                mThread.interrupt();
            }
        }
    }

    protected void allocateBufferSignal() {
        mSignal = ShortBuffer.allocate(mSignalBufferSize);
    }

    protected void allocateBufferFrame() {
        mFrame = ShortBuffer.allocate(mConfig.samplesPerBit); // one frame contains one bit
    }

    protected void allocateBufferData() {
        mData = ByteBuffer.allocate(FSKConfig.DECODER_DATA_BUFFER_SIZE); // maximum bytes
    }

    protected void nextStatus() {
        switch (mDecoderStatus) {
            case IDLE:
                setStatus(DecoderStatus.SEARCHING_SIGNAL);
                break;
            case SEARCHING_SIGNAL:
                setStatus(DecoderStatus.SEARCHING_START_BIT);
                break;
            case SEARCHING_START_BIT:
                setStatus(DecoderStatus.DECODING);
                break;
            case DECODING:
                setStatus(DecoderStatus.IDLE);
                break;
        }
    }

    protected void setStatus(DecoderStatus status) {
        setStatus(status, DecoderStatus.IDLE);
    }

    protected void setStatus(DecoderStatus status, DecoderStatus paused) {
        mDecoderStatus = status;
        mDecoderStatusPaused = paused;
    }

    protected short[] byteArrayToShortArray(byte[] data) {
        int size = data.length;
        short[] result = new short[size];

        for (int i = 0; i < size; i++) {
            result[i] = (short) data[i];
        }

        return result;
    }

    protected short[] convertToMono(short[] data) {

        short[] monoData = new short[data.length/2];

        for (int i = 0; i < data.length-1; i+=2) {
            int mixed = ((int)(data[i] + data[i+1]))/2;

            monoData[i/2] = (short) mixed;
        }

        return monoData;
    }

    // /

    protected void trimSignal() {

        if (mSignalPointer <= mSignalEnd) {

            short[] currentData = mSignal.array();
            short[] remainingData = new short[mSignalEnd - mSignalPointer];

            for (int i = 0; i < remainingData.length; i++) {
                remainingData[i] = currentData[mSignalPointer+i];
            }

            allocateBufferSignal();
            mSignal.put(remainingData);
            mSignal.rewind();

            mSignalPointer = 0;
            mSignalEnd = remainingData.length;
        }
        else {
            clearSignal();
        }
    }

    /**
     * Use this method to feed 8bit PCM data to the decoder
     * @param data
     * @return samples space left in the buffer
     */
    public int appendSignal(byte[] data) {
        return appendSignal(byteArrayToShortArray(data));
    }

    /**
     * Use this method to feed 16bit PCM data to the decoder
     * @param data
     * @return samples space left in the buffer
     */
    public int appendSignal(short[] data) {
        short first = data[0];
        short second = data[1];
        synchronized (mSignal) {
            short[] monoData;

            if (mConfig.channels == FSKConfig.CHANNELS_STEREO) {
                monoData = convertToMono(data);
            }
            else {
                monoData = data;
            }

            if (mSignalEnd + monoData.length > mSignal.capacity()) {
                //the buffer will overflow... attempt to trim data

                if ((mSignalEnd + monoData.length)-mSignalPointer <= mSignal.capacity()) {
                    //we can cut off part of the data to fit the rest

                    trimSignal();
                }
                else {
                    //the decoder is gagging

                    return (mSignal.capacity() - (mSignalEnd + monoData.length)); // refuse data and tell the amount of overflow
                }
            }

            mSignal.position(mSignalEnd);
            mSignal.put(monoData);

            mSignalEnd += monoData.length;

            start(); //if idle

            return (mSignal.capacity() - mSignalEnd); //return the remaining amount of space in the buffer
        }
    }

    /**
     * Use this method to set 8bit PCM data to the decoder
     * @param data
     * @return samples space left in the buffer
     */
    public int setSignal(byte[] data) {
        allocateBufferData(); //reset data buffer

        clearSignal();

        return appendSignal(byteArrayToShortArray(data));
    }

    /**
     * Use this method to set 16bit PCM data to the decoder
     * @param data
     * @return samples space left in the buffer
     */
    public int setSignal(short[] data) {
        allocateBufferData(); //reset data buffer

        clearSignal();

        return appendSignal(data);
    }

    /**
     * Use this method to destroy all data currently queued for decoding
     * @return samples space left in the buffer
     */
    public int clearSignal() {
        synchronized (mSignal) {
            allocateBufferSignal();

            mSignalEnd = 0;
            mSignalPointer = 0;

            return mSignal.capacity();
        }
    }

    ///

    protected int calcFrequencyZerocrossing(short[] data) {
        int numSamples = data.length;
        int numCrossing = 0;

        for (int i = 0; i < numSamples-1; i++) {
            if ((data[i] > 0 && data[i+1] <= 0)
                    || (data[i] < 0 && data[i+1] >= 0)) {

                numCrossing++;
            }
        }

        double numSecondsRecorded = (double) numSamples
                / (double) mConfig.sampleRate;
        double numCycles = numCrossing / 2;
        double frequency = numCycles / numSecondsRecorded;

        return (int) Math.round(frequency);
    }

    protected int calcFrequencyGoertzel(short[] data) {
        float[] doubleData = new float[data.length];
        for (int i = 0; i < data.length; i++) {
            doubleData[i] = (float) data[i];
        }

        List<Double> frequencies = new ArrayList<Double>();
        //frequencies.add(1200.0); //Mark
        //frequencies.add(2200.0); //Space

        frequencies.add(1200.0); //Mark
        frequencies.add(2200.0); //Space
        List<Map<String, Double>> magnitudes = frequencies.stream()
                .map(frequency -> {
                    Map<String, Double> entry = new HashMap<>();
                    entry.put("Frequency", frequency);
                    entry.put("Magnitude", calculateGoertzel(doubleData, frequency, mConfig.sampleRate));
                    return entry;
                })
                .collect(Collectors.toList());

        //Sort list
        List<Map<String, Double>> sortedMagnitudes = new ArrayList<>(magnitudes);
        Collections.sort(sortedMagnitudes, magnitudeComparator);

        List<Map<String, Double>> highestMagnitudes = sortedMagnitudes.stream()
                .limit(2)
                .collect(Collectors.toList());

        //highestMagnitudes.forEach(System.out::println); //print all frequency and magnitudes
        highestMagnitudes.forEach(entry -> {
            Double frequency = entry.get("Frequency");
            Double magnitude = entry.get("Magnitude");

            int threshold = 350;

            if (magnitude > threshold && frequency == 1200) {
                System.out.println("HIGH");
                DebugTools.WriteBit(true, "src/media/frame.txt");
            }
            else if (magnitude > threshold && frequency == 2200) {
                System.out.println("LOW");
                DebugTools.WriteBit(false, "src/media/frame.txt");
            }

            //System.out.println("Frequency: " + frequency);
            //System.out.println("Magnitude: " + magnitude);
        });

        return 0;
    }

    public Comparator<Map<String, Double>> magnitudeComparator = new Comparator<Map<String, Double>>() {
        public int compare(Map<String, Double> m1, Map<String, Double> m2) {
            return m2.get("Magnitude").compareTo(m1.get("Magnitude"));
        }
    };

    private static double calculateGoertzel(float[] samples, double frequency, int sampleRate) {
        double normalizedFrequency = Math.round(frequency * samples.length / sampleRate);
        double w = (2.0 * Math.PI / samples.length) * normalizedFrequency;
        double cosine = Math.cos(w);
        double sine = Math.sin(w);
        double coeff = 2.0 * cosine;

        double q0 = 0.0;
        double q1 = 0.0;
        double q2 = 0.0;

        for (float sample : samples) {
            q0 = coeff * q1 - q2 + sample;
            q2 = q1;
            q1 = q0;
        }

        return Math.sqrt(q1 * q1 + q2 * q2 - q1 * q2 * coeff);
    }

    protected double rootMeanSquared(short[] data) {
        double ms = 0;

        for (int i = 0; i < data.length; i++) {
            ms += data[i] * data[i];
        }

        ms /= data.length;

        return Math.sqrt(ms);
    }

    protected STATE determineState(int frequency, double rms) {

        STATE state = STATE.UNKNOWN;

        if (mConfig.modemFreqLow > mConfig.modemFreqHigh) //CID_SIG_V23 and CID_BELLCORE_FSK have Logical 0 (space) bigger frequency then Logical 1 (mark)
        {
            if (rms <= mConfig.rmsSilenceThreshold)
            {
                state = STATE.SILENCE;
            }
            else if (frequency > mConfig.modemFreqHighThresholdHigh && frequency <= mConfig.modemFreqLowThresholdHigh) //If frequency is between bigger and smaller frequency then we have a mark
            {
                state = STATE.LOW;
            }
            else if (frequency <= mConfig.modemFreqHighThresholdHigh)
            {
                state = STATE.HIGH;
            }
        }
        else //Other modem standards
        {
            if (rms <= mConfig.rmsSilenceThreshold)
            {
                state = STATE.SILENCE;
            }
            else if (frequency <= mConfig.modemFreqLowThresholdHigh) //If frequency is between bigger and smaller frequency then we have a mark
            {
                state = STATE.LOW;
            }
            else if (frequency <= mConfig.modemFreqHighThresholdHigh)
            {
                state = STATE.HIGH;
            }
        }

        //if (rms <= mConfig.rmsSilenceThreshold) {
        //    state = STATE.SILENCE;
        //}
        //else if (frequency <= mConfig.modemFreqLowThresholdHigh) {
        //    state = STATE.LOW;
        //}
        //else if (frequency <= mConfig.modemFreqHighThresholdHigh) {
        //    state = STATE.HIGH;
        //}

        return state;
    }

    protected short[] getFrameData(int position) {
        mSignal.position(position);

        allocateBufferFrame();

        for (int j = 0; j < mConfig.samplesPerBit; j++) {
            mFrame.put(j, mSignal.get());
        }
        return mFrame.array();
    }

    protected void flushData() {
        if (mDataLength > 0) {
            byte[] data = new byte[mDataLength];

            for (int i = 0; i < mDataLength; i++) {
                data[i] = mData.get(i);
            }

            allocateBufferData();

            mDataLength = 0;

            notifyCallback(data);
        }
    }

    ///

    protected void processIterationSearch() {
        if (mSignalPointer <= mSignalEnd-mConfig.samplesPerBit) {

            short[] frameData = getFrameData(mSignalPointer);

            int freq = calcFrequencyZerocrossing(frameData);
            calcFrequencyGoertzel(frameData);

            STATE state = determineState(freq, rootMeanSquared(frameData));

            if (state.equals(STATE.HIGH) && mDecoderStatus.equals(DecoderStatus.SEARCHING_SIGNAL)) {
                //System.out.println("Searching for the start bit...");
                //found pre-carrier bit
                nextStatus(); //start searching for start bit
            }

            if (mDecoderStatus.equals(DecoderStatus.SEARCHING_START_BIT) && freq == mConfig.modemFreqLow && state.equals(STATE.LOW)) {

                //System.out.println("Getting to the center of the bit...");
                //found start bit

                mSignalPointer += (mConfig.samplesPerBit/2); //shift 0.5 period forward

                nextStatus(); //begin decoding

                return;
            }

            mSignalPointer++;
        }
        else {
            trimSignal(); //get rid of data that is already processed

            flushData();

            setStatus(DecoderStatus.IDLE);
        }
    }

    protected void processIterationDecode() {

        if (mSignalPointer <= mSignalEnd-mConfig.samplesPerBit) {

            short[] frameData = getFrameData(mSignalPointer);

            double rms = rootMeanSquared(frameData);

            int freq = calcFrequencyZerocrossing(frameData);

            STATE state = determineState(freq, rms);

            if (mCurrentBit == 0 && state.equals(STATE.LOW)) {
                //start bit

                //prepare buffers
                mBitBuffer = new StringBuffer();
                mCurrentBit++;
            }
            else if (mCurrentBit == 0 && state.equals(STATE.HIGH)) {
                //post-carrier bit(s)

                //go searching for a new transmission
                setStatus(DecoderStatus.SEARCHING_START_BIT);
            }
            else if (mCurrentBit == 9 && state.equals(STATE.HIGH)) {
                //end bit

                try {
                    mData.put((byte) Integer.parseInt(mBitBuffer.toString(), 2));

                    mDataLength++;

                    if (mDataLength == mData.capacity()) {
                        //the data buffer is full, puke back to application and cleanup

                        flushData();
                    }
                }
                catch (Exception e) {
                    e.printStackTrace();
                }

                mCurrentBit = 0;
            }
            else if (mCurrentBit > 0 && mCurrentBit < 9 && (state.equals(STATE.HIGH) || state.equals(STATE.LOW))) {

                mBitBuffer.insert(0, (state.equals(STATE.HIGH) ? 1 : 0));

                mCurrentBit++;
            }
            else {
                //corrupted data, clear bit buffer

                mBitBuffer = new StringBuffer();
                mCurrentBit = 0;

                setStatus(DecoderStatus.SEARCHING_START_BIT);
            }

            mSignalPointer += mConfig.samplesPerBit;
        }
        else {

            trimSignal(); //get rid of data that is already processed

            flushData();

            setStatus(DecoderStatus.IDLE, DecoderStatus.DECODING); //we need to wait for more data to continue decoding
        }
    }

}

and after playing with a treshold a little my CID_sample-8.wav now looks like

10101001010101010101010101010101010101010101010110101010101010101010101111111110101010101010101010110101101010101011111010101010101010101010101010101010101010111111111111111111110111010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101001010101010110101010101010101010101011101010101010101011111111111111111111111111111111111111010101010111111111111101010100100011111111111111111111111111111111110000000001111111111111111111111111111111111111111111111001010000000000110111111111111111111111111100000000000001101111010111111111111111111111111110101010101110101010101010101010111111010101001010101000000000101010101011111111111101010101010101101010101010101010101010101001010000000000010110101010101010101111111111111111111111010101010101010101010101010101010101010101010100101010100000000000110101010101010010101010101010101101010101010101011111111101010101010101010101101010101010101010101010100101010100000000001010110101010011010101010101010101010101010101111111110101010101010101010110101010101010101010101010101010010101010100000000011010101010101001010101010101010101101010101010101111111101010101010101010101010101010101010101010101010101001010101000000000101010110101010101010101010101010101010101010111111110101010101010101010101010101010101010101010101010101001010101010100000001011010101010101001101010101010101010101010101011111111101010101010101010101010101010101010101010101001010100000000101010101101010101010101010101010101010101010101010101011111110101010101010101010101010101010101010101010101010101010101010010101010101010000000010000000010101010101011010101010101010111111110101010101010101010101010101010101010101010101010101001010100000000010101101010101010101010101010101010101010101010101011111110101010101010101010101010101010101010101010101010101010101010010101010101000000001010101010101001011010101010101010101010101011111111010101010101010101010110101010101010101010101010101001010100000000101010110101010101010101011101101111111011111110101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010010101010101010000001010110101010101010011010101010101010101010101010101111111010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101011010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101011010101010101010101010101010101010101010101010101010101101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101001101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010011101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010100101010101010101010101011010101010101010101010111010101011111110101010101010101010101010101010101010101010101010010101010101010101010101101010101010101010101010100101010000000010101011010101010101010101010101010101010101010101011111111010101010101010101010011010101010101010101011111111010101010101010101010011010101010101000000000000000010101010101010101010101010101000000000000000010101010101010101010101010101010101010101011010101010101010101010101010101010111001010101010101010101010101010101010101010101010101010100000000001011010101010010101010101010101010101011010101010101010101010101010101010101111000000001010101010101010101010101010101010101010101010101010101010101011010101010101010101010101010101010101010101010101010101010101010101010101110110101001010101010101010101010110101010101010101010010101010101010101010110101010101010101011000011010101010101010111010101010101010101010101111111111111001010101010101010101011010101010101001010101011010101010101010101010101010101010101010100101010101010100000000000001111111111111110100101010101010101010101010101010101010101101010101010101010101010101010101010111100101010100000011111111111111111111111110101001010101010101010101010101010101101010101010101010101010101010101010101111000000010101010101010101010101010101010101010101010101010101010101011010101010101010101010101010101010101010101010101010101010101010101010101011110111010100101010101010101010101010101010101010101010101010101010101010110101010010101011010101010101010101010101010101010101010101010101010101010111100101010101010000011111111111111110101001010101010101010101010110100101011010101010101010101010101010101010101010111100000000101010101010101010101010101010101000000000000011010101010101010101010101010101010101010101010101010101010101010101010101010111000000010101010101010101010101010101010101010101010101010101010101010110101010101010101010010101010101010110101010101010101010101010101010101010111111111100101010101010101011010101010101010101001010101011010101010101010100101010101010101010101010101010110101010101010101010101010101010101011100101010101001010101000000000000000101010101010101001100101010101010101010101010101010101010110101010101010101010101010101010101011110000000010101010101010101010101010101010101010101000000100011010101010101010101010101010101010101010101010101010101010101010101010101011100000000101010101010101010101010101010101010101010101010101010101010110101010101011110101010101010101010101010101010101010101010101010101010111111111110100101010101010110101010101010101010101010101010101010101010101001010101010101010110100101010101101010101010101010101010101010101010111100101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010110101010101010101010101010101010101010111100000010101010101010101010101010101010101010101010101010101010101010110101010101010101010101010101010101010101010101010101010101010101010101010111101010100110010101010101010110101010101010101010101010101010101010101010101010101010111010100101010101101010101010101010101010101010101010101010101111111111110100101010101010110101010101010101010101010101010101010101010010101010101010101011010101001010110101010101010101010101010101010101010111100000010101010101010101010101010101010101010101010101010101010101010110101010101010101010101010101010101010101010101010101010101010101010101010101011100000101010101010110101010101010101010101001010101010101010101010110101010101010101110101010101010101010101010101010101010101010101010101010101110111110101001010101010101011010101010101010101010101010101010101010101010101010101010101001000010110101010101010101010101010101010101010101010111111111111100101010101010101101010101010101010101010101010101010101010101010101010101010101011110010101101010101010101010101010101010101001010101010101010101010101010101101010101010101001010110101010101010101010101010101010101010101010101010101011110000101010101010110101001010110101010010101010101010101010101010101101010101010101010101010101010101010101010101010101010101010101010101010101010111000001010101010101010110101010101010101010101010010101010101010101101010101010101011110101010101010101010101010101010101010101010101010101010101111111111010010101010101101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010111100101010110101010101010101001010110101010010101010101010101010101010101101010101010101010100110101010101010101010101010101010101010101010101010101010111100000010101010101010101010101010101010101010101010101010101010101011010101010101010101010101010101010101010101010101010101010101010101010101010111010101101010101010101010101010101010101010101010101010101010101010101010101010101011111010010101101010101010101010101010101010101010101010101010111111111110100101010101011010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101111001010101010101010101010101010101010101010010101010101010101010101011010101010101010101010101010101010101010101010101010101010101010101010101010101011100001100101010110101010101010101010101010101001010101010101010101010110101010101010101010101010101010101010101010101010101010101010101010101010101111011101101010101010101010101010101010101010101010101010101010101010101010101010101110101001010101101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101111000011010101010101010101010101010101010010101010101010101010101010110101010101010101010101010101010101010101010101010101010101010101010101010101010111001010101010101010101010101010101010101010101010101010101010101010101010101010101011111010101010101010101010101010101011111101010010101001010110100101010101010101101010101010101010101010101010101010101010101010101010101010101010101010111100011010101010101010101010101010101010101010101010010101010101010110101010101010101010101010101010101010101010101010101010101010101010101010101010101110011010101010101010101010101010101010101010101010101010101010101010101010101010111110101010101010101010101010101010101111010100101010000101101010010101010101010101101010101010101010101010101010101010101010101010101010101011101010101010101010101010101010101010101010101010101010101011110110110101010101010101010101010101010101010101010101010101010101010101010111111010100101010101101010100101010101010110111010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101110110101010101010101010101010101010101010101010101010101010101010101010101010101010111101010101010101010101010100101101111110101001010101010110101010100101010101010110101010101010101010101010101010101010101010101010101010101011101010101010101010101010101010101111111101010101010101010101010100101010111111010101001010100010101101010010101010101010101101010101010101010101010101010101010101010101010101010101011101010101010101010101010010101011111110101001010101010110101001010101010101010111101010010101000001010101010101010101010101011010101010101010101010101010101010101010101111111111101010101010101010101010101010101111110101001010101010101101001010101010101010110101010101010101010101010101010101010101010101010101010101011010101010101010101010101001010101111111101010101010101010101010010101010101011111010100101010101010101101001010101010101010101101010101010101010101010101010101010101001010110101010101011110101010101010101010101001010101111111010100101010101101010010101010101010101111010100101010100101010110100101010101010101010101101010010101010000101011010101010100101011010101010111101010101010101010101001010101010111111010100101010101010110100101010101010101010111010100101010000010101101010101010101010101010101010111101010101010101010101010100101111111110101010101010101010100101010101010101111110101001010101010110100101010101010101010101101010010101000001010110101010101010101010101010101011110101010101010101010101001010101011111101010100101010101101010010101010101010101101010101001010101010101010101101010101010101010101010101010101010101010101010101010101010010101101010101010111101010101010101010100101010101010111110101001010101010101101010010101010101010101101010100101010100001010110101010101010101010101010101011101010101010101010101010010101010111111111010101010101010101010010101010101011111101010010101010101011010010101010101010101010110101010010101010101010110101010101010101010101111111110101010101010101010100101010101111111010101010101010101010100101010101010101011010101010010101010101010101101010101010101010101010101001010101000101010110101010101001010101011111111110101010101010101010010101010101111110101010010101010110101001010101010101010101101010100101010101010101010110101010101010101010101010101010101010101010101010101010100101010101111111101010101010101010101001010101010101011111010100101010101010110100101010101010101010101101010010101010000101011010101010101010101010101010101011101010101010101010101010010101010111111101010101010101010101010010101010101010111110101010010101010101011010101010101010101010101010100101010100001011010101010101010101010101010101011110101010101010101010010101010101111111010101010101010101010010101010101010101011010101010010101010101010101101010101010101010101011111110101010101010101010101010010101011111110111101010101010101010100101010101010110101010101001010101010101011010101010101010101010101010100101010101010101011010101010101010101010101111111010101010101010101010100101010101111111101010101010101010101001010101010101010110101010101001010101010101011010101010101010101010101010010101010001010101101010101010101010101010101010101110101010101010101010010101010101011111110101010100101101010100101010101010101011010101010100101010101010101101010101010101010101010101001010101010101010101101010101010100101010111111111101010101010101010010101010101010110101010101010010101010110101001010101010101010101011010101001010101010101011010101010101010101010101011111110101010101010101010100101010101011010101010101010101010101010101010010101010101010111111111111111111111111111111111111010101010101010101010101010101010101111111111111111111101010101010101010101010101010101010111111111111111111110101010101010101010101010101010101011111111111111111111010101010101010101010101010101010101111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111110101010101010101010101010101010101011111111111111111111010101010101010101010101010101010101111111111111111111101010101010101010101010101010101010111111111111111111110101010101010101010101010101010101011111111111111111111010101010101010101010101010101010101111111111111111111101010101010101010101010101010101010111111111111111111110101010101010101010101010101010101011111111111111111111010101010101010101010101010101010101111111111111111111101010101010101010101010101010101010111111111111111111110101010101010101010101010101010101011111111111111111111010101010101010101010101010101010101111111111111111111101010101010101010101010101010101010111111111111111111110101010101010101010101010101010101011111111111111111111010101010101010101010101010101010101111111111111111111101010101010101010101010101010101010111111111111111111110101010101010101010101010101010101011111111111111111111010101010101010101010101010101010101111111111111111111101010101010101010101010101010101010111111111111111111110101010101010101010101010101010101011111111111111111111010101010101010101010101010101010101111111111111111111101010101010101010101010101010101010111111111111111111110101010101010101010101010101010101011111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111010101010101010101010101010101010101111111111111111111101010101010101010101010101010101010111111111111111111110101010101010101010101010101010101011111111111111111111010101010101010101010101010101010101111111111111111111101010101010101010101010101010101010111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111110101010101010101010101010100101011010011111111111111111101111101010101111111010101010101010100101010101010101010101101010111101010101010101010101010101010101010100101010100011111111111111111111111111111111111111110101010101010101010101010010101010110101010101010101010101010101010101010101011010101010101010101010101010101011111111111111111111111111111111110101010101010101010101010101001101010101010101010101011111111111010101010101010101011101010101010101011111111111111111111111111111111111111111011010101010101010101010010101010100111111111111111110101011111110101011111111111101010101010101010101101010010101011010010101010101010101010101010101010101010110101010101010101010101010101010101010101010101010101010101010101010101010101010101010101011111111111111111111111111111111111111111111111111111111111111101010101010101010101010101010101010101010111111111111111111010101010101010101101010010101010110101010100101010110101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010100101010101010101010101010101010101010101101010010101010101011010101010101010101010101011111111111111111111111111111111111111111111111110101010101010101010101001010101010111111111101010101010101001010101010101010110101010101010101010101111111111111111111111101010111101111111111111111111111111111111111111111111111111111111111111111111111010101010101010100101010101010101101010101010101010101010101010101010101010101010111111111111111111111111111111111111111111111111110101010101010101010100101010101010101010101011010101010101010101010101010101010101010101010111111111111111111111111111111111111111111111111110101010101010101011010101001010110101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010111111111111111111111111111101010101010101010101010101010101010101001010101000101010101010101010010101010101010101010101010101111111111111111111111111111111110101010101010101010101010101010101010100101010101000111111111111111111111111111111111111111111111111111010101010101010101010100101010101010110101010100101010101000101010101010100101010101010101011111111111111111111111111111110101010101010101010101010101010101010101001010101010101011010101010101010101010101010101010101010101010101010101111111111111111111111111111111111111111111111111010101010101010010101010101010101101010101010101011111111111111111111111111111111010101010101011111111111111111111111111111111111111111111111111111111111111101010101010101001010101010101010111111111111111111111111111111011010101010101010101010101001010101010101101010100101010110101010101010101010101011111111111111101010111111111111111111111111111111111110101010101010101010101010101010101010100101010100010110101010101010101111111111111111111111111111111111111111111111111111111111111111010101010101010100101010101010101011010100101010101010101010111111111111111111111111111111101010101010101010101010010101010101101010101011110101010101010101001010101010101111111101010101010101010010101010100010111111111111111111111111111111111111111111111111111111111111111111111111101010101010101001010101010101011111111111111111111111111111111101010101010101010110101010010101011010111111111111111111111111111111111111111111101010101010101001010101010101010110101010101010101010111111111111111111111111010101010101011111111111111111111111111111111111111111111111111111111111111111111110101010101010101001010101010101010111111111111111111111111111111110101010101010101011010101001101010101010101011110101010101010101001010101010101010101010101011111110101010101010101010101010101111111111101010101010101010101101010010101011010111111111111111111111111111110101111010101010111111010101010101010101001010101010101101010101010101010101011111111111101010101010101010101010101010101010101010101010101111111111111111111111111111111111111111111111111111111111111111010101010101010100101010101010101111111111111111111111111111111101010101010101010101101010101010101010100101010101010101010111111111111111111111111111111111101010101010101010010101010101010101010101010101010101010111111111111101010101010101010101001010101010101111111101010101010101010100101010101010101101010101010101010101010111010101010101011111111111111101010101010101010101001010101010101010111111111111111111111111111111111111111111101010101010101001010101010010101010101111111111111111111111111111111111111111111111111111111111111111111111111111111111111111110101010101010101010111111101010101010101010010101010101101010101010111111111111111111111111111101010101010101010101010101010101010101001010101010010111111111111111111111111111111111110101010101010101001010101010101101010101010101010101010101010101010101010101010101111111111111111111111111111111111111111111111111111111111111111111111111111111111111110101010101010100101010101010111111111111111111111111111111111110101111111110101010101010101001010100000101111111111111111111111111111111111101010101010101001010101010101010110101010101001010101010110101010101111010101010101010101001010101010101101010101010101010101010101010101010101010111111111111010101010101011111111111111111111111111111111110101010101010101010101010010101010101111111111111110101010111111111111111111111111111010101010101010010101010101010101011111111111111111111111111111111101010101010101010100101010101011010101010101010101010101010101010101010101011111111111111111111111111111111111111111111111111111111111111111111111111111111111111111010101010101010101010101010101010101111111111111111111111111111111111111111111010101010101010101001010101010101101010101010101010101010101010101010101010101010111111111111110101010101010101010100101010101010110101010101010101010101010001010101010101010101010101010101010101010101010101010101010000101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010111010101010101010101010010101010100010101010101101010101010101010101010010101010101010101010101010101010101010101010101010101010101011010010101010101010110101010101111111101010101010100101101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010111011101010101010101010101010101010101010101010101010101010101010101010101001011010101010101011111111111111111111010101010101010101010101010010101010101010101010101010101010101010101010101010101010101000000010101010110111101010101010101010101010101010111010101010101010010101010101010101010101010000101010101010101010101010101101010101010101010101010101010101010101010101010101010101010101010101001010100101001101010101010101010100110101010101010101010101010101010101010101010101010101001010101010100101010101111101010101010101010101010101010110101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010100010101001010101010101010101010101010101010101010101010101010101010101011010101010101010101010101010101010101010101010101010101010101010101010000101010101011010101111111111111110101010101000000001101010101010101010101010101010101001010101010101010101010101010101011010101010101010101010010101010101010101010101010101011010101010101001010101010101010101010000000001010110101010101010101001010101010101010101011010101111111111111101010101010010101010101011010101010101010101010101010101010101010101010101010101010101010101010101010101000001010101010101010101101010010101010101010101010101010100010101010101010101010101010110101010101010100101010101010101010101010101010010110101010101010010101010101010101010101010101010101010101010110101010101010101010101010101001101010101010101010101010101010101010101010101010101010101010101010111101010101010101010101010111111111010101011111111111101010101010101011101011110110101010101010101011101001010101010101010101010100000000101010101010100000101010110101010101010101001011010101111111010101010101010101010101010101010101010101010101010010101010101010101010101010101010101010101101010010101011010101010101010101010101010101010101010010001010101010101010101010101011010101111111111010101010101001010000101010101011010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010100110101010101010101010101010101010101010101010101010111101010101010101010101010101001011010101010101001101010101011010101010101010101010101010101010101010101010101010101010100101010101010101010101010101010101101010101011111101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010010101101010101010100101010101010101010101010101101010101010110101010101010010101010101010101010101010110101010101010101010101010101010101001010101010101010100001010110101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010111010101010101010100101010101011010101010101010101010101010101010101010101010101010101010101010101010101010101010101000101010101010101010101010101010101010101010101010101010101010101010101001010101010101010101010101010100101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101110101010101010101010101010101010101010101010101010101010101010101010101010101010101010101011101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101011101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010100101011010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101011101101010101010101010101010101010101010111010101010101010101010101010101010101010101010101010101010101010101010101010101010101110101010101010101010101010101010010101010101101010101001010101010101010101101011010101010110101010101010101010101010101111101010101010101010010101010101010101011010101010101010101010110101010010101010101010101010101010101010101010101010101010101010101010101010101101010101010101010101010101010101111101111010101010101010101010101010111010101010101010101010101010101010010110101010101010101010101010101010101010101010101010101010101010101011010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101001010101010101010101011010101111111110110101010101111111110101010101010101010101010101010101010101010101010101010101010101010100101010101011010101010101010101010101010101010101010101010101010101010101010010101010101010110100101010101010101010101010100010101010101010101010101010110101010100001010101010101010101010101010100101101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010100101101001010101010101010101011010101010101010101010101010101010101010101011010101010101101010101010101010101001010101010101010101010101010101101010010101010101010110101010101010101010101010111101010101010101010101010101010101010101010000000000101000101010100000000101010110101010101010101010101010101010101010101010101010100101010101010101010101010101010101010101011010101010101010101010101010101010101010101010101010101010100101101010101010101010101010101010010101010101010101010110101010101010101010101010101010101010101010101010101011101010101111010101010101001010101010110101010101010101010101010101010101010101010101010101010101010101010101001011111111110101010101010100101010110101111010010101010001011110101010101010101010101101001010101010101010101010101010101011010101010101010101010101010101010101010111101010101010101011101000001101010101010101010101010101010101000000000010101010101010101010101010101010101010101010101010011010101010101010101010101010010110101010101010101010101010101010101010101010101010101010101010101010010101010101010101010101010101010101010101010101010101010100100001010101010110101010101010101010101010101010101010101010101000010101010101010000000000101010101010101010101010101010101010101001010101111010010101010110101010101010101011101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010010101010101010101010100000010100000000000000010101101010101010101010101010101111101010000101101010101010101010101010101010101010101010101010101000010110101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101110101010101010101010101010101010101010101010101011110101010101010101010101010101010101010101010101010101010101010100010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101001101010101011101010101010101001010101010110010101010101010101010101010001010101010101101011111111111111111101010101010101010101010101010101010101010101010101010101010101010101010101101101010101011101010101010101111111111111010101010101010101010101010101010101010101010101010101010101010011010101010101010101010101010101010101010101010101010101010101010101010101010100001010101101010101010101000101010101010101010101010111110101010101010010000000000011110101010101010101111111101010101010101010101010010101010000101010101010101010101010101010111101010101010101010101010101010011001110101010101010101011101010101010101010101010101010101110101111111101010101000000000000000000000101111111111010101010100000101010101010110101010101010101011111111010101010101010101010101010101010101111110101010101010101010010101010101010101010101010000000000000010001010101011010111110101010101001010101010101010101010101010101010110101010101010000001010101011010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101011101010100101010101010101010110101010101010101111111101001000000101010101010101010101010101101010101010101010101010101010101101010101010101010101010101010101010100001011010101010101010101010101010101010001010101010101011111111111010101110101010101010101010101010101010101010101010010101010101010101101010010000000000010101010101010110101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101001101011111010101010101010100100010101010101010101010101010101010101010010101010101011010101010101010101010101010101010101011010101010101010101010101010101001001011010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010100101010101101010101010101010101010101010111101010101010101010101010101010101010101010101010101010101010100101010101010101010101010101101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101011010101010101010101010101010101010101011110101010101010101010101010101010101010101010101010101001010110101010101010101010101110101010101010000000000000010101010101010101010100101010101010101010101011010101010101010101010101010101111101010101010101010101010101010101010101010101010101010101011110101010101010101010101010101010101010101010101010001011101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010110100101010101010101010101000000110101010101010101010101010101010101010101001010101000000000001010101101010101010101010100101010101010101010101010101010101010101011010010101101010101010101001010101010110101010101010101010101010101010101010101010111101111110101010101010101010010101010101010000000000101010110011010101010101010101010101001010101010101010101001010101010110101010101111111111111111111100000000000001010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101011101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010010101010101010101010001011101010101101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010100110101010100110101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101111010101010101010101010101010101010101010101010101011110101010101010101001010101010101101010101010101111111111101000000001010100000001010101010101010101010101010101111010101010100110100101010101010101010101010101010101010101010101010101010110101010101011110010101010101010101010101010101010101010101010101011010101010101010101010101010101011111111111111010101010101011110101010101010100110101010101010101010101010101010101010101010101010110101001010101010101010101010101010101010101101010101010100101010101010101001010101010101010101010101010101010101010101011010101010101010101010101010101010101010101010010101101010100001010101010101010101010101011010101010101010101011110101010101010101010101010101010101010101010101010101010101010101010101010010101010110101010101010101010101010101010101010101010101010101010101010111111110101010101001010110101010101010101010101011101001010111111111111110101010101010101010111101010101010101010101010101010101010101010101001010101010101010101010101011010100101010110101010101010101010101010101010101010101010101010101010101010101010101010111101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010010101101010101010101010101010101010101010101010101010101010101001010101010101000010101010101010101010101010101010101010101010101010101010101010110101010010101010000000001010101010101011010101010101010101010101010101010110101010101010101010101010101010101010101010101110101010100101010101010101010101010101010110111111111101010101010101010101011110101010101010101010101011100010000000000001011010101010101010111111010101010111001010101010101010101010101010101010101010101010110101010101010101010101010101010101010101011110101010101010101010101010101010101010101010101001010101010101010101010101101010101010101010101010101010101010101010101011101010011111010101010101010101010101010010101010111010101101010100101010101010101010101010101010101010101010101101010101011111111111111111111010101010101010101010010101010101010000010101010101010101010101010101010101101010101010101010101010101010100010101101010100101101010101010101010101010101010101011110101010101010101010101010010101010101010110101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101001010101010101010000101101010101010101001011010101010101010100101010101010101010101011010101010101010101010101010101001101010101010101010101010101010101010101010101010101010100100000000000000000000101010101010101010101010101010101101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010001010101010101010101010100101011010010101010101010101010101010101011110101010101010101010101010101011101001010110101010110101010101010101111010100000000110101010101010101010101010010101100101010101010101101010101010101010101010101010101010101010101010101010101010101010101101010101001011010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010111010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101011101010101010101010101010101010101010101010101010101001010101000000001010101010101010110111111010101010010101010101010101010101010101010101010101010101011010100001010101011111110101010101001011010101010101010101010101010101010101010101010101010100101010101010101010100000001010100101010101010101001010001101010101010101010101010101010100101010101010001010101010101010101010101010101010101101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010010110101010101010100101010101101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101110101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010100101010101011010101010101010101010101010101010101010101011101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010100010101010101010101010101010101010101011101010101010101010101010101010111101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010010100101010101010101010101010101010101010101010100101101010101010101001010101010101011010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101011111111101111111010101010101010101010101010101010101101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010100101010101010101010101010101010101011010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010100010100010101010101011010101010101010101010001011111111111111111111111111111011101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101011111111111111111010101010101101010101010101010101010101010101010101010101010101010101010101010101010101010100101010101010101010100110101010101010101010101101101010101010010101010101010101010110101010101010101010101010101010011001010101010101101010101010101010101010101010101010111010101010101010101010101010100101010101011010101010101010101010101010101010101010100101010000000101010000000000000111111111111111010000101011010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101000010101010101010101010101010101101010101010101001010101010101010101010101010110101010101010101010101010101010101010101000001010101010110101010101010101010101000010100010101010110111010101010101011101010101010101010101110101010101010101010101010101010101010100110101010101111110100101010101010101010101010101010101011000010101101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010010110101010101010101010101010111010101010101010100101010101010101010101010100101010101011111111111111111110101010101010101010101010101010010101010101010101010101010101010101010101010101010100001000000000010101010101010001111111111111111101011111110101010101010101010100101010101101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101011101010101010101010101011101011101010101010101010101010100101010101011011111010101010101010101010101010001010101010101010101010101010101010101010101010101010101010101010101010101010101010101010010101010101010101010101000101010110101111111111010101010101010101010101010111010101010100101010101010101010101010101010101101010101010101010101010101010101010101001010101010101010101010110101010101010101010101010101010101010101010101010101010101010101010101010101011111111111111111111111101011010101001010101010101011010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010100110101010101010101010101010101001011010101010101011101010101010101010101010101010100101010101010101010101010101010101010101010101010111111010100101010101010101010001010101010101010110101010101111111111101001010110101010101010010100000101010101010101010101010101010000101001010101101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101011110101010101010101010101010101001101010101010101010101010101011110101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101101010100101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101110101010101010101010101010101010001010101010101010101010101010100000000000000000000011101010101010101010101010101010010101010101010101101010010000101010110101010101010101010101010101010100101010101111101010101010101010101010101010101010101010101010101010100101010101010101010101101010010110101010101010010011111010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010111010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101001101010101010101010101010000110101010101010101010101010101010101010101011110101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010000101101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101000101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010110101010101001101010101010101010101010101010101010101010101111010101010101010101010101010100110101010101010101010101010101111101010101010100010101011010011010101010101010101010101001011010111010010101010101010101010101010101101010101010101010101010100101010101010101010101010101010101010110101010101010101010101010101010101010101010101111010101010101001010110101010101010101010101010101011110101010101010101010101010101010101010101010101010101001010101010110101010101010101010101010101011110101010101011010101010101010101010101010101010101010101010101010101010101010101010100010101001010101010000011010111110101010101010101010101010101010101010101010101010101010101010111111010010101010101010101011010111010101010101010100101010101010101010101010110100101011010101010101010101010111010111010101010101010101010101010101010101010101010101011110101010101010101010101010101010101010101111010101010101010101010101010101011101010101010101010101010101010101010111110101010110101010101010101010101010101010101010111010101010101010101010101010101010101010101010101010101010101010101010101010101111101010101010101011111111111010101010101010101010101010101011110101010101010101010101010101010101010101010101110101010101010101010101010101010101010101010101010101010101011101010101010101010101010101010101010101010110010101101010101011110100101000101010110101010101010101010101010101010101010101010101010101010101010101010010101010101010100110101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010100010101010101010101010101010101010101010101010101010101010101010101010101010111010101010101011101010101010101010100101010100000010101010101010101010101011010101010101010101010101010010101010101010101111010101010101011010111111010101010101010101001010101010101011010101010101010101010101010101010100101010101101010101010101010100101011010101010101100101010101011010101010101010101010101010101010101010101010101010101010100101010110101010101010101101111010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010010101000101010110101011110101010101010101010101010101010101010101111010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010111010101010101010101010101001011010101010101010101010101010101010101111010101010101010101010101010101010101010101010101010101010100101010101010101010110101010101010101010101010101010101000010110101010101010101010101010101010101010100101010101010101010101010101010101010101010101010101101010101010101010110101010101010101010111101010101010101010101010101010101010101010101010101010101010101010101111010101001010101010110111110101010010110101011111111010101010101010101010101010101011101000000101010101010101010101101010101001101010101010101010101010101010101010101010101010101010101010101010101010111101010101010101010101010101011101010101010101010101010101010101010101010101010101010101010101010111111101010101010101010101010101010100010110101010101010101010101010010101010110101010101010101010100101010101010110101010101010101010101010101010111010101010101010101010101010101010101010101010101010101010101010101010111111111101010101010101010101010101010101010101010101010101010101010101010101010101010100111010101010101010101010101010101010101010101010101010101010101010101010101010001010101010101010101010101010101010101010101010010110101010111111111010101110010101010101010101010101010101010101101010101111111101010101110101010101011110101111111111101010101001010101010101011010101010101010101010101010101010101010101011101010101010101010010101010101010101010101010101010101010101010101010101010101010101010110111111111111111111110101010100101010101010101010110101010101010101010101010101010101010100101011011111111111111111010101010101010101010101010101010101010101010100010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101001011010101010101010101010101010101010101010100101010101010101010101010101010101101010101010101010101010101010101010101010101010101010101010101010010101010111101001010101010101010101010101011010101010101010101010101010101010101010101010101000000101010101010110101010101010101011101110101010010101010101011010101010101010101010101010101010101010101010101101010010101101010101010101010101010010101101010101010101010101010101010101010101010101001010101011001010101010101010101010101010101010101010101010101010101010101101010101010101010101001101010101010101010101011111101111111111110101010101010101000001010101000010101010000000001010101010101010101010011011101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010100101010110101010100101010100101010101010101010101010101010101010101010101010101010101010010101010101000111101010101010101010101010111101010101010101010101010100101000000101010101010101010101010101010101010101011010101010101010101010101010101010101001010101010101010101010101010101010101010101010010101010101010101010101010101010101010101001010101010110101010010101010101101001010101010001010101010101010101010101011010101010101010101001010101010101010101010101010101010101010101011010101010101010101010101010101010101010101010101010101010101001010111010101010101010101010101010101010101010101010101011010101001011010101010101010101010101010100101010101010101010101010101010101011010101001010101011010101001010100010101010110101010101010101010101010101011111111111111111111111010101010101010101010101010101010101010101011111111111110110100101010110101010101010101010010101010101010101010101010101010101010101010101010101101010101010101010101010101010101010101010101001010101101010101010101010101110101010101010101010101010101010101010111101010101010101001010101101010111111111010101010101010101010101010101010101010101010101001101010101010101011111111111101010101010101010111111111010010000101010101010101010101010101010101010101101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010111101010101010101010101010101011011110101010101010101001010101010101010101010101010101101010101010101010101010101010101011101010101010101010101011101010101010101010101010101010010101010101010101010101010110101010101010100101101010010101010101010101010010101010101101010101010101010100000000101010110101010101010101010101010101010101010100110101000001010101010101010101010101010101010101010101010101010101010101010101000101010101010101010101010101010101010101001010101010101011010101001011010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010100101010101010101011010101011110101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101011110101010101010101010101010101010101010111101010101010101010101010101010101010101010101010101010101010101010101010101010101010111101010101010101010101010101010101010101010101010101010101010101011110101010

Channel Seizure is finaly starting to get shape, Mark signal is as good as it always was, althoght its still seams something is not write (frames have to much alternating 0101 in it, althogh I can finaly detect some start and stop bits) maybe my threshold is to low or I should also check sorounding frequencies not only 1200 and 2200, idk

but this seams as step forward, not realy sure, better then yesterday it seams :smile:

Happy holidays (new year) to you as well

[iganev]

That sounds promising! Now that you mention DTMF and Goertzel I remember writing a DTMF parser in the early 2000s as a teenager, not knowing what I was doing of course... it was back when WAP was still the hottest trend and was expensive to use WAP, but you could have 3 numbers on your plan with whom you could talk for free with unlimited minutes. So I had an idea to use a GSM modem hooked up to my computer at home and call it to get free internet through the voice channel... had to parse DTMF to basically initialize and authenticate with the thing.. it was ridiculous.

I'll try to poke around the FSKDecoder using your Goertzel code too. But not sure if that will be today. Will let you know if I manage to achieve anything.

[iganev]

Reading the hackaday article you linked I found this:

 Update:  I was able to get this method working, but I'm finding that it's severely constrained by the short window that I have to use in order to discern changes at 1200 baud.  Basically, at 26400 Hz sampling frequency, just 22 samples span each 1200 Hz cycle (the symbol rate at 1200 baud).  The frequency resolution of this method is dF = fs/N = 26400/22 = 1200 Hz, which is terrible for this application, given that I'm trying to differentiate between tones that are separated by only 1000 Hz.  I'm basically locked into this property because I can't extend the Goetzel evaluation window wider than one 1200 baud symbol period.  I can't just double sampling frequency because then I would have twice as many samples per 1200 baud period, resulting in the same ratio and same frequency resolution.  This interpretation is born out by experimentation described below.

If this is correct, this might be a clue to why we keep failing over and over again. My assumption was that the PCM signal in those wav files is at 44100 sample rate. If the sample rate is 26400, that would explain the frequency mismatch and the overlapping sample windows that keep on flipping or duplicating bits. Is like listening to music sped up and trying to play the notes by ear... It could also explain the weird baudrate of 1200 that is not a common divisor of 44100 and yields a weird 36.75 samples per bit instead of a round number... will look into that as well... 22 samples per bit sounds much more reasonable than 36.75 samples per bit.... although it might not be enough for a zerocrossings calculation (not enough crossings in that sample size) it could be a clue to what's wrong...

[iganev]

I also completely forgot that WAV has some headers at the beginning of the file. Not that it's all that critical, a good algorithm should basically ignore them, but during development it's helpful to work with the raw PCM data directly and not having to deal with headers and stuff like that. To do that you can re-export the wav from audacity like so:

Also, if you notice, I set the sample rate to 48000 instead of the default of 44100 because a baudrate of 1200 is a common divisor and thus makes each sample frame exactly 40 samples per bit. Using that sample rate fixes the frequency deviation and now you'd see 1200 and 2200 as expected. Unfortunately, some bits are still flipped, even with Goertzel. In fact I managed to achieve the exact same result with Goertzel as with Zerocrossing.

The next clue to why that might be came from one of the documents you linked, not entirely sure which one right now, but somewhere I read that the frequency shift is not sudden (like my FSKEncoder/Decoder expects), but is gradual. That messes up the frames a little bit because both frequencies have significant amplitudes and you can only guess the direction the signal is heading by comparing previous frames to see which amplitude is rising and which is falling. Needs more research.

[veso266]

OK, I guess one step closer again (FFT now shows correct frequencies), thats great

I turned to the official documentation, I only found the european one (EN 300 659-1 - V1.3.1

Its the same as Bellcore one, just different Mark and space frequencies if Bellcore has (whats inside my samples)

1200Hz - Mark
2200Hz - Space

ETSI has

1300Hz - Mark
2100Hz - Space

it appears its just a regular V.23 / Bell 202 Modem

and I think I found the picture (Figure 5 Maximum Out of Band Tx Line Energy Limits in CMX624_CML.pdf that describes non sudden changes)

I also found this if it helps: https://github.com/DSheirer/sdrtrunk/blob/master/src/main/java/io/github/dsheirer/dsp/afsk/AFSK1200Decoder.java#L40

since what I was able to read is that CallerID uses AFSK which is not the same as regular FSK

so yea, not sure when do you have new year, we have it in 1hour here in Slovenia (UTC+1) so I should probably sing a carol or something :smile:

[iganev]

I throw confetti in UTC+2 and UTC-7 :dancers: .

I was thinking yesterday... it's a little bit weird to have a baud rate of 1200 using these frequencies... my crappy FSK code for example does around 10 times lower baud rate at similar frequencies.

That's because basically what happens is that you have up to 1 zero crossing (wave cycle) in 1 sample window for "mark" (HIGH) and up to 2 zero crossings (wave cycles) in 1 sample window for "space" (LOW).

Even a tiny miss-alignment of the sample window could and probably would produce bit flipping and errors.

Therefore I think it is absolutely crucial to use a more sophisticated frequency detection algorithm that determines the frequency by the wave slope rather than zero crossings or any other short-cut approaches. I am not sure what that algorithm should be though, not too familiar with DSP math and all that voodoo.

Will research when I have a chance.

Happy New Year! :confetti_ball:

Edit 1: Check this out: http://www.krten.com/~rk/articles/rtenough.html This fellow builds a template of sine/cosine values rate of change for space/mark (LOW/HIGH) frequencies and detects them with very few samples just by correlating the results in the rate of change of the signal.

Edit 2: Check this out too: https://sites.google.com/site/wayneholder/attiny10-assembly-ide-and-device-programmer/bell-202-1200-baud-demodulator-in-an-attiny10 This is the same approach, it's just that the coefficients aren't normalized and he's assessing all the samples. In this case 8 samples, because the sample rate is just 9600. (If you extrapolate to 44100 sample rate that makes 36.75 samples and 40 samples as 48000, so same thing practically, just worse signal)

[veso266]

Hi there hopefully new year went well

Some update, after trying to implement the FSK detector from here: https://sites.google.com/site/wayneholder/attiny10-assembly-ide-and-device-programmer/bell-202-1200-baud-demodulator-in-an-attiny10

first I resampled my CID_sample-8.wav to 9600 and converted it to raw samples and then fed this into the demodulate like this

InputStream input = new FileInputStream(new File("src/media/CID_sample-8_9600.raw"));
  //Feed raw data direcly
  byte[] data = new byte[input.available()];
  input.read(data, 0, data.length);
  ByteBuffer pcm = ByteBuffer.wrap(data);

  //the decoder has 1 second buffer (equals to sample rate),
  //so we have to fragment the entire file,
  //to prevent buffer overflow or rejection
  byte[] buffer = new byte[1024];

  //feed signal little by little... another way to do that is to
  //check the returning value of appendSignal(), it returns the
  //remaining space in the decoder signal buffer
  while (pcm.hasRemaining()) {
      if (pcm.remaining() > 1024) {
          pcm.get(buffer);
      }
      else {
          buffer = new byte[pcm.remaining()];

          pcm.get(buffer);
      }
      mDecoder.appendSignal(buffer);

      Thread.sleep(100);
  }

catch (IOException e) {
  e.printStackTrace();
catch (InterruptedException e) {
  e.printStackTrace();

initialized fsk decoder like this: mConfig = new FSKConfig(FSKConfig.SAMPLE_RATE_44100, FSKConfig.PCM_8BIT, FSKConfig.CHANNELS_MONO, FSKConfig.CID_BELLCORE_FSK, FSKConfig.THRESHOLD_20P); (as I did want to keep samples in bytes)

    private static byte[]   coeffloi = { 64,  45,   0, -45, -64, -45,   0,  45};
    private static byte[]   coeffloq = {  0,  45,  64,  45,   0, -45, -64, -45};
    private static byte[]   coeffhii = { 64,   8, -62, -24,  55,  39, -45, -51};
    private static byte[]   coeffhiq = {  0,  63,  17, -59, -32,  51,  45, -39};

    private static int demodulate (byte[] data, int idx) {
        short outloi = 0, outloq = 0, outhii = 0, outhiq = 0;
        for (int ii = 0; ii < 8; ii++) {
            byte sample = data[ii + idx];
            outloi += sample * coeffloi[ii];
            outloq += sample * coeffloq[ii];
            outhii += sample * coeffhii[ii];
            outhiq += sample * coeffhiq[ii];
        }
        return (outhii >> 8) * (outhii >> 8) + (outhiq >> 8) * (outhiq >> 8) -
                (outloi >> 8) * (outloi >> 8) - (outloq >> 8) * (outloq >> 8);
    }

    protected short[] byteArrayToShortArray(byte[] data) {

        for (int i = 0; i < data.length-8; i++) {
            var result = demodulate(data, i);
            if (result > 0) //2200 -> (Space (0)
            {
                DebugTools.WriteBit(false, "src/media/frame.txt");
                System.out.println("LOW");
            } else if (result < 0) //1200 -> (Mark (1)
            {
                DebugTools.WriteBit(true, "src/media/frame.txt");
                System.out.println("HIGH");
            }
        }

        return new short[]{0,0,0}; //just so the rest of the code does not complain, since function (demodulate) wanted bytes, and here we still have bytes I decided to hijack it
    }

I got this out

1101100111100111111111000001111111111111100000001111111000000111111110000100000111100000011110011111111110100000011111111111111111111110000000100111110001111011111111111000000001011111111100000000000001111001000001111000111110000000011110000111100001111111101100111111111111000000000111010010000111111111000001111111111110010000000111111100000111111110000001100111001000010000011011000111111111100111110000000000001101111101000000011111000011110111111110011111111111100001111000011110000001001111110000001111111111101101111000000111111111011110000000000001111110010111100000001000000001001110010001000111100000000001111111111111111000000001111111111100000111111111111111101111101000000101000000000111111011001101111111100001111000000111111111111111100110001011111000010111011110000100011000000000000111111100000111011001000000000001001110000000010111111111111111111110110001100001011000000000011111111111111111001000011111100110000111111011111101111110000111100000100000001100000000110000111111111111111110011100111111110000111111111111001111111111111111110010011100110000001000101111111111111111110000000000111111100011111111111100000000000011111110001111111100000000111111101110000000000011111111110100111111111111111111101111111111111110000010101111110000111111111111111000111011111111110010111111111111111111110000000000000000000110110010011101110000000100001111111100000000001111111100111110010000000000000011110000101100111111111111000011111100000000111011110000010000111111110011111101111111011011111111000000111111111111110001000001110011000000111111000011110011111100111111110001000000011000000011111111111011111111000000000000011110001111011001001111111111000000011111111111111111111101000011111111100111100011111111111111111100000000000110000001111111111111000011111111111110000000001111111111100010111100000000000000011100100110111001101100011111110011111100000011111000111111111100001011111111110000000000001111111100001111111111110011111111111111111111111111111111111111110111111111111100000011011100000000001011111000000000001111000011111111110000000000011111111111110000001100011101100011110000111100000111111111000000110000101110010001111000111111111111111011111111111011111111111111111111110011111111110000100000000000001100111110011111111111111100001111111100011111111110000000000000001000010001100000000000010110100111111101100111011111100111111110000010111111111111100000001111111111111000011111111111111111111111111111111110001010011111110001111011111111111110000000000000000111111111100000000000111111111000000001101000011111111110000111111111111111111110011111011011101110010111111111111111111111111111001101111111111100111000000111101111000000111100101111111111111011111111111011110010111111110000101111000011111111111111111111111110111100001111111111111100000110001111110001110111110000000000111111110000110100111000001111110000000011111110100101110111111111111111111111111111110111111110000110001000011111111101011111111000011110111010010001101111111111111111111111111111010111101110011111111110000000000011111111111111111111111111011101001000011111100000111110010111100000000111100001110011111000011110000000000111100001100111011110000111101111101000000111111111100111111111001001111010011111111110000101111110111111111000000000000111111000000000111111111111111111110110100001011011101000000000000000000001100000010000000000000111110011011101011000011100000111111111100000011111111111100000011110010100101001111111100000000000011100010111111111100000011111111111110010000000001100111111111111111111111000011110011000000111100100111001100110011001111110010011111111111000000000000001011111111000011111111110000101111111111111100001111111111111111111110111111100001111111111110000000000111100001111100000001111111111000011000001001110111100101111001001111111111100000011000000001111110001101111110100000001111100101011011111111111000000000011111110001111111111001111111100001111100111111111000101111011001111111111111101001111111100001111111100000000000011001111111110000000000000000011111111110000001111111100000101111100000000010100000011111111001111111111111111111111111100111111011111111111111100000011111111111110110011100000000000111100001101110011101100000001111111111000011111111111101101111000000000000000000000000001111111100011111111100111111111111011011111101110010001111111111000000111100000011100000001001000000000000111101000000111111100000001111000011111111111001100000000000111111111100001111000011111111111111101001111111110001011110011101111111111111100000000110100001111000000100011110000111100011111000000000000000000001111111111000000001000111100010011010111110111110000111111111111111111110011111111111111111111110000000011111111000011110000001111000000111100001111111111111111111101000000001111111111000010111111111100010001101111111000111100111111111100001110011111000000101111111111000001111111111111001111111111111000111111111111000011111100000000111111111100000000111100110000001110000000001111011011111110000001101011010111110000101111011111101001000111010111111000111111111000111111101100000100000001111111100000011111000111110000111111110110000011111111111111100011111111000111111111111100100001001110010011111000000000111000000101101111111111110111001110000111111011111000011111111000000100111111110100000000011000011110100000011111111111000001111000000000000000111000011111111100100011011111111100111111100001110001101111111111111111111110001111010011111111111111110011111011111111110111111011111111101000000011111110000000011111110001111111000000000001111111111111111111111111110100111011100011111111100000000001000111111111001111111100011110000001111100011110000000111111100011111111111100011110000110000000000000001111110011111111110000000111111000000001111001001000000111111111111111111111110111011110000000011111111111111111000111010011111111000111001000111100011111010000000100010000011111111110010000001000000001000110000000101111111111001000111111111100000100111000011101001111100011111111000000011110100000001011111111111100011100100111000011111110100001001111110111111111100001111111111110000111111111001000111111111011100001111111001111100001111111111100000001111111011000101111111111111100111111110001111111111110000000000110001111100000111111111111110011011111100001001100010000011011111111111111111011000000000111111100001101000010111111000000010011111011111111111111111100000000111110010011111111111111110010001110000011111111001111110011111111110101001100010111111101000000000011011111111111000011111000010010101101111110000000000111111010011100110101000000001010000011111110011111111001111100100100001101111100000000100111110000111111111111110011111111111110010000001111110000111000001111110111111111111100001111110000000000111001111111111100001111111100000000000000011110000000000111100011111111000000010001110001110101000000010000011111111011111111111110001111100001100001000111111111111111011111111100111100011110000000000111011011111011111000001111100011111111111000111111110000011101101111110000000001111010111001001111100010000001111111110000011111011101011111001111100001100001111011011011100000011111101110111110010000011111111001001111111111000001011110000001110010000001000001110100101111111001111000000000011101000100101111111101110100100000000000111111111100101110110111101011111110000000001001111111110001111001001011100011100010000000111110010111111111111110000000010011111111101111100101111111111111111111111011111011111100111111110000000000011111111111110000001100111111111101011111101011000001111000000100000000011100111000000011111100000011011001100110110010000000000001110011111111111000000000011111100000010111001001111101100010100001001111111100000001000111011001010010011101101110010111111111111000000000100001101111000100100000000000000100111111000001111110000111111111010000100111111111111110100000000111111110110001111111111111111110000000011111111111100001100001111110011000011111100001101110000111111010000100111111111111111111111111111110010000010010000000000011111111100000010000001111111111101100000001111011011010000000011111111000000101111100110000011110000111111110000111111110000000010011111110000010010111111000000000000000000001001000000000001110000111100000000111111111111111100111111100000000001100011000000000000000011111111111111111111111100111111111100001111000000000011111111111100111111111110111111111101111001111001011011110101111111001000010010011111111101111000000001110111111111100111111110000001111111100111111111111011111111111000110111111111111110000011110111111111100110011111111111100011111111111111111001111111110011111111100001111111111110111111110001110111101011000000000000000001000000111110000011111000000011111111101111111111111111111000001000000001111000011111101111111001100001111111111110111000000000001001101111100001111110011111111111000011111110111111000000111000000111111111110000000011111101000001111100000000111111111111111100000011111111111111100111110000001110000011100000000001111111101111111110100111111110010111100000000010111001011111111110000000000010111111111111001111111111110000000000100101011111110111000111111111111111111111111011111100111000100011111111111110101110000011100000000000000001111110000001111110111111111000000001100111111110010100011110000011101000000110010111111111001001111111100011111111100011000111100000001110010001110000000001111100000001101110111111100001111111111111111001111111111111111111111111001110100111111111111100101111111111000011111111111000110111110111111111111110011000000111111011101010000000000111111110000111111111111011100111011111100000001111110001110001110001000000111110000011010001001000110011100011111111011101111111110001111011001000011111110000000000111001011111111110010011101111111000000111100001111000000111111000011111000001111000111010000001111111110001110000010011100011111111100001111111111111111111111000000001111111111111000000111111111111111110110010011111100111100111101001011111001000011110000111110111000000000111100110011001111111100000000000011111100111111000000011111000011111111101101111111110101100000111111110000111011111011111100000111110111111111111111001111100000011111011110111111111111110011111100001111111100001011010000001111110000001111111111010011111111111111001011111111000000011111000001111111111111000010010000111111110000111111111100000111110000111101101111111111111110111111011000111110000000111100000011101111111001001111111111111111111110101111111000000111111000000000000000000110011101111110101011011111111111110011110000011111110000111101001111000000000011111111111111111110100011111110001001111111001111111111111111111111111111111111000000111111100000001111111111111110110000110000110111000011111110111111011110111000011110011111111111111111111100000000000111110000000111111000000000000111011111111111111111111111111111111111100111111111110001100001111111111101111011111000001111111111111111111111111111100000000000000000010001110011111110000111111111111111111000011001111111010110011011111000011010000000000000001000011100100111100001101111111100100000000000000001111001111111111011111011011000001011111100111111110001111111111110100001111100111111111111001111111100000000001111111110001110000100100000000000011111111111100111111111111110000111100000000001110001111110011111111100100000001011100101111111111100000011110000111100000000011111001111110111100000001110100000110100011011000001111100000000111111111111011100011111111111111001000111111101111111111111000011110000000000000111000011111111001111000000100111111100000001111111111100001111000000001111111111101111010011110000110000001111110000000001111110001011111111110000111110111111111111111110000111111111111111100001111111101000001111111110000100111111111111111000000111111100011111111110111111111111110011011001110110010010011111111100100111111111100000000000000011100000000111111001111111111111111100100000000110100111100000000000000001011111111111111001111111111111011110000011111111111000001111111110100001111000000111111000000000011111111100101111100001110001111010001110101110000111111110000000011000011111111111111111111111100111111001111111111111000001111000011110000111000000010110000111111111111111111111110111111111111111111111100001111111111100110111111111110000000111111111100000000111111111111101111110000111100000001111000000010001100111111111101100011000000111011110100100011011011111111110001111111110011110000111100000010001111111100001111111111100000011000111000111000001001111111111111111111111011111110000011110011000000001111111111111100111111111111010011000001111111111111110000111111111111101110010011111110000011111100000011111100000000000000000011111111111111111111100111111111110111111111100011100100011110000011111111111111111111000011110000000111001111000100000011111111100000001000000100111111111111000000000000111100000000000000111111110000001100000100001100000100111111000011111100111111110000001110111111111111110000110000000011111111101001000000001111100000110000111100000000111111111110010111111111000011111110110110001111001010000000111111110000111111111101001110000011100100000001110011110000000110001100111111111001111111111111100010111100001000000000001001111110000000001111110000111100011111100111111100000000000000111111001111111111111111111111011111110000000011111111011111111111000000000011111111001011111111000000000111111101111111111111000011111011110011111111000100000000001001111100111111110000111111110100011110000000111111110011111111111111111111111111111111111100110011000011000011000011011111001001110111111111111111111111100000111111000011111111111111110000111000001111000000000000011100001011111111000011110000111111111011000011111111111100000010011101101100011111111111110000001100111111110000000000000011111000101111111101111100001111111110111111111101000000001111011111111100000000001111100001111111100001111000000000000000011111010011100010011110001010011111100011111111100100111110011111111000001000000111100000000000011111111111111100111000011101111011111111111111001110111111111000000011000111111111111111111111111010000010111011111001111110000111111011100011100000000111101000000111101000000111111111111111111101111111111111110000111000000000000000000111111000011111011111111110000111111111100101111001111000011100000011111100111001111111111100000001101111110000011100100001111111100000011011111000000111000011111111110000010111111111111111111000111111111111111111111111110001111111010011000011110000111011111111111100000000000000000000000000111111110000110100000000000000000000111111111100000

which looks realy wierd, there is not even a channel seizure and mark signal this time...

Then I found this DSP wodoo: https://github.com/mobilinkd/afsk-demodulator/blob/master/afsk-demodulator.ipynb

at least it explains the process in details very well and after folowing it (using python just to see if it would work with it, so if it doesn't I wouldn't waste time translating it to java), I didn't even get the proper cats eye, so not even PLL locked to the signal right, so bits were all over the place..

from scipy.io.wavfile import read
import numpy as np
import matplotlib.pyplot as plt
from scipy.signal import lfiltic, lfilter, firwin
from DigitalPLL import DigitalPLL

#audio_file = read('CID_sample-8_26400.wav')
#audio_file = read('TNC_Test_Ver-1.102-26400-1sec.wav')
#audio_file = read('CID_sample-8_simulated_26400.wav')
audio_file = read('fsk_only_no_cs-8_26400.wav')
sample_rate = audio_file[0]
audio_data = audio_file[1]

# Total time
T = 10.0 / 1200.0
# Sampling frequency
fs = sample_rate
# Number of samples
n = int(T * fs)
# Time vector in seconds
t = np.linspace(0, T, n, endpoint=False)

class fir_filter(object):
    def __init__(self, coeffs):
        self.coeffs = coeffs
        self.zl = lfiltic(self.coeffs, 32768.0, [], [])
    def __call__(self, data):
        result, self.zl = lfilter(self.coeffs, 32768.0, data, -1, self.zl)
        return result

bpf_coeffs = np.array(firwin(141, [1100.0/(sample_rate/2), 2300.0/(sample_rate/2)], width = None,
        pass_zero = False, scale = True, window='hann') * 32768, dtype=int)

#print(bpf_coeffs)

bpf = fir_filter(bpf_coeffs)

filter_delay = len(bpf_coeffs)//2

delay = 12

# Select the audio data to demodulate (first 60 bits)
s = audio_data[:22*60 + delay]
# Band-pass filter the audio data
f = np.append(bpf(s), bpf(np.zeros(len(bpf_coeffs)//2)))[len(bpf_coeffs)//2:]
# Digitize the data
d = np.array([int(x > 0) for x in f])
# Delay the data
a = d[delay:]
# XOR the digitized data with the delayed version
x = np.logical_xor(d[:0-delay], a)
# Low-pass filter the PWM signal
lpf_coeffs = np.array(firwin(101, [760.0/(sample_rate/2)], width = None,
        pass_zero = True, scale = True, window='hann') * 32768, dtype=int)
#print(lpf_coeffs)
lpf = fir_filter(lpf_coeffs)
c = np.append(lpf(x-0.5), lpf(np.zeros(len(lpf_coeffs)//2)))[len(lpf_coeffs)//2:]

# Digitize the correlated output.
dx = np.array([int(x > 0) for x in c])

pll = DigitalPLL(sample_rate, 1200.0)

locked = np.zeros(len(dx), dtype=int)
sample = np.zeros(len(dx), dtype=int)

for i in range(len(dx)):
    sample[i] = pll(dx[i])
    locked[i] = pll.locked()

# Total time
T = 10.0 / 1200.0
# Number of samples
n = int(T * fs)
# Time vector in seconds
t = np.linspace(0, T, n, endpoint=False)

class NRZI:

    def __init__(self):

        self.state = False

    def __call__(self, x):

        result = (x == self.state)
        self.state = x
        return result

nrzi = NRZI()

data = [int(nrzi(x)) for x,y in zip(dx, sample) if y]
print(data, len(data))

# Save the plot as an image (you can choose the format, e.g., PNG, JPEG, PDF, etc.)
#plt.savefig('correlated_signal_plot.png')
plt.show()

after almost giving up, I found this: https://github.com/maxirmx/minimodem4j/blob/main/src/main/java/minimodem/fsk/Fsk.java which looks very promising, because if u look here: https://github.com/maxirmx/minimodem4j/blob/main/src/main/java/minimodem/databits/DataBitsCallerId.java it claims it decodes callerID which means if it does, then FSK demodulator inside has to be real good

Sadly right now I am too tired to try this :) (yea, not sure how but I started when the sun was starting to come up and its dark outside now :) )

If it turns out that the FSK demodulator works out, we should probably adapt it to your library, because even if zero crossing is not good for everything, your library is way easier to understand then this minimodem4j one :smile:

Well good night, so sorry if I bother u too much

[iganev]

I am afraid my library expects 3 start bits (for good omens) which needs to be changed before you can expect any results with it and also the fact that you initialize it with sample rate of 44100 on a 9600 bitrate file is also a reason for it not to work as expected.

For my library to work for bell 202 both the state machine and the frequency detection need to be changed for sure.

Lmk if you have any success with that last one. If not I can give it a try during the weekend.

[veso266]

Sucsess

Here is the smallest example I can throw together :smile: FskDemodulationExample.zip

I guess FFT worked after all, it still has problems with some noisy samples and I have no idea how it would perform on a real phone line (have to make this first phone line to sound card interface first :smile: )

Thanks so much for giving me hope to continue :smile: I am very happy now PS: Checksum is still missing, but to test this properly I have to first find some broken samples :smile:

[iganev]

Nice! I'll have a look during the weekend and try to understand the example code you provided. Good job!

[veso266]

FskDemodulationExample_crc.zip

Added CRC,

for some reason everything goes faster once u are not stuck in a loop anymore :smile: