michalmonday / CSV-Parser-for-Arduino

It turns CSV string into an associative array (like dict in python)
MIT License
58 stars 12 forks source link

Strange behavior when read after the buffer used befor #19

Open Dork57 opened 1 year ago

Dork57 commented 1 year ago

Hello,

I read a csv-File after sending over html (AsynchronWebserver). When I read row after row and write the result to my array all works fine.

But when I read the result and check if the value == Null, the value shall not write in my array, the csv- parser seems to break. I don't why.

this is my code:

     `CSV_Parser cp((char *)buffer, "udududud",true, ';', """");
      //cp.parseLeftover(); 
      uint16_t *Drehzahl =         (uint16_t*)cp[0]; 
      uint16_t *Kurve1 =           (uint16_t*)cp[1];
      uint16_t *Kurve2 =           (uint16_t*)cp[2];
      uint16_t *Kurve3 =           (uint16_t*)cp[3];

      cp.print();

      for (uint row = 0; row < cp.getRowsCount(); row++) {
          uint8_t Drehzahli = Drehzahl[row]/100;

        //NULL nicht beachten
       if(Kurve1[row]){Winkel[0][Drehzahli] = Kurve1[row];}
       if(Kurve2[row]){Winkel[1][Drehzahli] = Kurve2[row];}
       if(Kurve3[row]){Winkel[2][Drehzahli] = Kurve3[row];}
        `

The output of cp.print (important section):

_CSV_Parser content: rows_count = 221, cols_count = 4 Header: Drehzahl | Kurve 1 | Kurve 2 | Kurve 3 Types: uint16_t | uint16_t | uint16_t | uint16_t Values: 0 | 0 | 0 | 150 100 | 0 | 0 | 150 . . . 19600 | 0 | 0 | 130 19700 | 0 | 0 | 130 19800 | 0 | 0 | 130 19900 | 0 | 0 | 130 20000 | 0 | 0 | 130 130 | 130 | 13800 | 130 130 | 130 | 13900 | 130 130 | 130 | 14000 | 130 130 | 130 | 14100 | 130 130 | 130 | 14200 | 130 130 | 130 | 14300 | 130 130 | 130 | 14400 | 130 130 | 130 | 14500 | 130 130 | 130 | 14600 | 130 130 | 130 | 14700 | 130 130 | 130 | 14800 | 130 130 | 130 | 14900 | 130 130 | 130 | 15000 | 130 130 | 130 | 15100 | 130 130 | 130 | 15200 | 130 130 | 130 | 15300 | 130 130 | 130 | 15400 | 130 130 | 130 | 15500 | 130 130 | 130 | 15600 | 130 130 | 130 | 15700 | 130 Memory occupied by values themselves = 1801 sizeof(CSVParser) = 40

The File ends normally after. 20000 | 0 | 0 | 130 But suddenly there are ghost values after.

Is there any explaination? My csv.File attented. MeineKurven.csv

Dork57 commented 1 year ago

The right result with the following code:

_Kurven Dateiname: MeineKurven.csv CSV_Parser content: rows_count = 201, cols_count = 4 Header: Drehzahl | Kurve 1 | Kurve 2 | Kurve 3 Types: uint16_t | uint16_t | uint16_t | uint16_t Values: 0 | 0 | 0 | 150 100 | 0 | 0 | 150 200 | 0 | 0 | 150 300 | 0 | 0 | 150 . . . 19700 | 0 | 0 | 130 19800 | 0 | 0 | 130 19900 | 0 | 0 | 130 20000 | 0 | 0 | 130 Memory occupied by values themselves = 1641 sizeof(CSVParser) = 40

       `CSV_Parser cp((char *)buffer, "udududud",true, ';', """");
      cp.parseLeftover(); 
      uint16_t *Drehzahl =         (uint16_t*)cp[0]; 
      uint16_t *Kurve1 =           (uint16_t*)cp[1];
      uint16_t *Kurve2 =           (uint16_t*)cp[2];
      uint16_t *Kurve3 =           (uint16_t*)cp[3];

      cp.print();

      for (uint row = 0; row < cp.getRowsCount(); row++) {
          uint8_t Drehzahli = Drehzahl[row]/100;

   Winkel[0][Drehzahli] = Kurve1[row];
   Winkel[1][Drehzahli] = Kurve2[row];
   Winkel[2][Drehzahli] = Kurve3[row];
   `
Dork57 commented 1 year ago

Okay, now i am a step further.

The issue appears in the second run of code. If I always reset the controller befor I send the csv-File, it seems to work without the issue.

Dork57 commented 1 year ago

So now I have got found the problem.

The problem is that the csv_parser tries to read the whole buffer. If the buffer has larger contents from the previous run, it doesn't break at string terminators: \00

Problem in buffer (Here I tried to insert some \00 marks at the end of the string. It doesn't work): 19800;130;;<\r><\n> 19900;130;;<\r><\n> 20000;130;;<\r><\n> <\0><\0><\0><\0><\0><\0><\0><\0><\0><\0>3800;130;130;130<\r ><\n> 13900;130;130;130<\r><\n> 14000;130;130;130<\r><\n>

My solution is now to clear the whole buffer after the imported content.