occasionally corrupted data when sending on macOS + CP2102

thkusch commented 7 years ago

I observed occasionally corrupted data (about every 20 times) when sending a simple String (about 10 chars) as byte array via writeBytes(serialHandle,data) on macOS 10.12.3 using CP2102 USB-Serial converter.

The same code using Bluetooth serial connection works (to HC-05 BT-UART-Dongle). The same code on Win10 works with both, CP2102 and Bluetooth.

When using jssc, there are no issues with the combination macOS + CP2102

I assume some issue with the driver communication.

RishiGupta12 commented 7 years ago

What is your test setup. I mean who sends data and who receives it (MAC + CP2102).

thkusch commented 7 years ago

Hello Rishi!

I did some investigation. The problem seems to be the fineTuneReadBehaviour method. I call my receive method and pass a timeout to it. When I call fineTuneReadBehaviour, the read fails sometimes with gibberish data. Tested with the CP2102 as loopback device, so it is really pinned down to the CP2102 device. As I wrote you, other devices (I tried BT) and other OS (tried with Win) works. Also the CP works with jssc and the RXTX lib.

Here is my simplified code. It is a port of my code which used jssc before so it is not polished yet to your lib, but it reliably reproduces the problem. It occurs after 10-30 transmits. See attachment for the console output. It shows "RECEIVED DATA" if something went wrong.

If would be great if you could fix it (also maybe allow to set the size of the internal buffer. 2k is pretty small...). Jssc sucks so much when trying to use BT with it..

Best regards, Thomas

public static void main( String args[] ) { String sData="&abcdefghi\n\r"; String sRecv; byte bData[]=new byte[sData.length()]; try { scm=new SerialComManager(); serialHandle=scm.openComPort("/dev/tty.SLAB_USBtoUART", true, true, true);
scm.configureComPortData(serialHandle, SerialComManager.DATABITS.DB_8, SerialComManager.STOPBITS.SB_1, SerialComManager.PARITY.P_NONE, SerialComManager.BAUDRATE.B38400, 0); scm.configureComPortControl(serialHandle, SerialComManager.FLOWCONTROL.NONE, 'x', 'x', false, false); for(;;) { scm.writeBytes(serialHandle, sData.getBytes());

            bData=recv(100);
            sRecv=new String(bData);
            System.out.println(sRecv);
            if(!sData.startsWith(sRecv))
            {
                System.out.println("\n"+sData+sRecv);
                System.exit(0);
            }
            Thread.sleep(10);
        }

    } catch (Exception e) {
        e.printStackTrace();
    }
}

public static byte[] recv(int iTimeout) throws SerialException
{
    try
    {
        byte res[]=new byte[256];
        int iPos=0;
        byte b;

        try{
            scm.fineTuneReadBehaviour(serialHandle, 0, iTimeout/10, 0, 0 , 0);              
        } catch (Exception e) {
            Logger.print("RECEIVE: failed", e);
            throw new SerialReadException("cannot receive data");   
        }
        while(true) {

            try{
                b=scm.readBytes(serialHandle, 1)[0];
            } catch(Exception e){                           
                System.out.println("RECEIVED DATA:"+ new String(res));                      
                throw new SerialTimeoutException("data receive timeout");
            }
            if(b=='\n' || b=='\r') {
                if(iPos!=0){
                    byte newRes[]=new byte[iPos];
                    System.arraycopy(res, 0, newRes, 0, iPos);
                    return(newRes);                     
                } else continue;

            }
            res[iPos++]=b;                                          
            if(iPos==res.length) {
                byte newRes[]=new byte[res.length+256];
                System.arraycopy(res, 0, newRes, 0, res.length);
                res=newRes;
            }
        }                       
    }
    catch(Exception e){
        if(e instanceof SerialTimeoutException)
            throw new SerialTimeoutException("data receive timeout");
        Logger.print("RECEIVE: failed", e);
        throw new SerialReadException("cannot receive data");           
    }
}

Am 18.02.2017 um 13:50 schrieb Rishi Gupta notifications@github.com:

What is your test setup. I mean who sends data and who receives it (MAC + CP2102).

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

RishiGupta12 commented 7 years ago

What exactly you want to do. For ex; send string "abcdef\r\n" and receive 10 bytes. Please clarify about receive part. What is your dependency on 2k buffer size.

thkusch commented 7 years ago

the 2k may be too low because I simply need a lib that is able to handle larger data chunks than 2k (in my case these are indeed strings. The whole thing is an embedded application). Beside this, the data are sent and received asynchronously, so it is not unlikely that there are cases where there may be more than 2k in the receive buffer. I don't see a reason for such a limitation. Nevertheless it is not as important as reliable data handling which isn't in my case. Even the problem seems to be the fineTuneReadBehaviour method, it doesn't seem to occur on receive part but on send part. The data on the TX line is wrong (checked with a logic analyzer). Do you have any idea? Is it possible that if the fineTuneReadBehaviour method is called WHILE there is still data in the send buffer it fails?

Best regards, Thomas

Am 18.02.2017 um 17:45 schrieb Rishi Gupta notifications@github.com:

What exactly you want to do. For ex; send string "abcdef\r\n" and receive 10 bytes. Please clarify about receive part. What is your dependency on 2k buffer size.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

RishiGupta12 commented 7 years ago

The code you mentioned above has many wring things. Leave the timeout and sending part. Just tell me what you want to do in receive part. For ex; do you know how many bytes of data will come or whenever data comes it will be terminated by \n\r etc. When the data arrives it will keep coming but there is maximum limit for ex; 5k etc.

thkusch commented 7 years ago

Hello!

The code you mentioned above has many wring things. Leave the timeout and sending part. Just tell me what you want to do in receive part. For ex; do you know how many bytes of data will come or whenever data comes it will be terminated by \n\r

The length of a single data chunk is undetermined. All data chunks and ends with \n\r. That's the reason why I read it byte by byte. If there is no data, the method should return after an adjustable time out.

RishiGupta12 commented 7 years ago

In this example modify while (totalNumberOfBytesReadTillNow <= 10) { line for \n\r condition. https://github.com/RishiGupta12/SerialPundit/blob/master/applications/d1-length-nonblocking-read-com/src/example/ComPollDataRead.java
Set timeout only once after you call scm.configureComPortControl or you can use thread.sleep way. For ex; call read method, if it does not return any data then seleep again and call read again. if it still does not return you can return completely. Notice in above code how i have used Thread.sleep(10);

Or much better you define what is the total timeout for ex; you call read method and waiting for data to be received. Call a while loop and set initTimeOut = 0, then call read method if no data is received sleep for some time call read again, but here you check if final timeout has been reached or not. Tis way you will not have to use fineTune API at all. See this ex https://github.com/RishiGupta12/SerialPundit/blob/master/modules/serial/src/com/serialpundit/serial/ftp/SerialComXModem.java#L148

thkusch commented 7 years ago

Am 18.02.2017 um 18:40 schrieb Rishi Gupta notifications@github.com:

• In this example modify while (totalNumberOfBytesReadTillNow <= 10) { line for \n\r condition. https://github.com/RishiGupta12/SerialPundit/blob/master/applications/d1-length-nonblocking-read-com/src/example/ComPollDataRead.java

This is not a good example for my application.. What if there is MORE than one chunk of data in the RX-buffer? When I call readBytes I get all data. So if I return the data up to the first \n\r only the remaining data is lost. As I wrote in the last mail, I read the RX buffer byte by byte to prevent this. Maybe streams are the better solution. Do you have an example?

RishiGupta12 commented 7 years ago

more examples https://github.com/RishiGupta12/SerialPundit/tree/master/applications
stream class is there please read javadocs https://github.com/RishiGupta12/SerialPundit/blob/master/modules/serial/src/com/serialpundit/serial/SerialComInByteStream.java
to prevent rx buffer overflow use a reader thread that call back main app whenever chunk data is received, till then it keep allocating buffers dynamically. take a look at this example for hid to understand about this approach more https://github.com/RishiGupta12/SerialPundit/blob/master/applications/hid-reportthread-hotplug/src/example/HIDDemo1.java

thkusch commented 7 years ago

thanks!

Am 18.02.2017 um 18:57 schrieb Rishi Gupta notifications@github.com:

• more examples https://github.com/RishiGupta12/SerialPundit/tree/master/applications

• stream class is there please read javadocs https://github.com/RishiGupta12/SerialPundit/blob/master/modules/serial/src/com/serialpundit/serial/SerialComInByteStream.java

• to prevent rx buffer overflow use a reader thread that call back main app whenever chunk data is received, till then it keep allocating buffers dynamically. take a look at this example for hid to understand about this approach more https://github.com/RishiGupta12/SerialPundit/blob/master/applications/hid-reportthread-hotplug/src/example/HIDDemo1.java

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

RishiGupta12 / SerialPundit

occasionally corrupted data when sending on macOS + CP2102 #19