Large amounts of data (over 50 bytes) unreadable

[simonbarker]

I've been trying to read any considerable amount of data off the card for a few hours now and can't read more than about 50 chars before the message becomes rubbish.

The data is stored correctly on the SD card but using the supplied Read example code it doesn't come through onto my terminal correctly.

I have tried following the advice in issue #108 but that only solves the problem for the first transmission block - after than its back to jibberish.

I assume it is possible to retrieve large amounts of data off the cards but right now the provided example code isn't working for me.

I've changed the timeout limit to 100 as when set to 1000 the loop never completes

void readFile(char *fileName) {
  //Old way
  openLog.print("read ");
  openLog.print(fileName);
  openLog.write(13); //This is \r

  //New way
  //OpenLog.print("read ");
  //OpenLog.println(filename); //regular println works with OpenLog v2.51 and above

  //The OpenLog echos the commands we send it by default so we have 'read log823.txt\r' sitting 
  //in the RX buffer. Let's try to not print this.
  while(1) {
    if(openLog.available())
      if(openLog.read() == '\r') break;
  }  

  //This will listen for characters coming from OpenLog and print them to the terminal
  //This relies heavily on the SoftSerial buffer not overrunning. This will probably not work
  //above 38400bps.
  //This loop will stop listening after 1 second of no characters received
  char inBuff[32];
  for(int timeOut = 0 ; timeOut < 100 ; timeOut++) {
    int n = openLog.available();
    if(n > 0) {
      for(int x = 0 ; x < n && x < 32 ; x++) //Read the incoming characters into a local buffer
        inBuff[x] = openLog.read();

      for(int x = 0 ; x < n && x < 32 ; x++)    
        Serial.write(inBuff[x]); //Take the character from OpenLog and push it to the Arduino terminal
      timeOut = 0;
    }
    delay(1);
  }
  //This is not perfect. The above loop will print the '.'s from the log file. These are the two escape characters
  //recorded before the third escape character is seen.
  //It will also print the '>' character. This is the OpenLog telling us it is done reading the file.  

  //This function leaves OpenLog in command mode
  Serial.println(data);
}

[simonbarker]

It seems this issue can be solved by using the hardware serial port on the Arduino but in software serial it's a no go - even using buffers - the openLogger just starts dumping rubbish in to the buffer.

Get the feeling like this project has been abandoned though so hopefully someone can work out why this is in the future.

[nseidle]

Grr. I submitted a comment earlier this morning but it seems it was lost. Probably my own error. Here it is again:

I am assuming you can get the read operation to work correctly when attached directly to an OpenLog. This would prove to me the OpenLog is ok but the Arduino with its buffers is the problem. Please confirm.

If you are able to, please include the file that is giving you the problems. Might give some insight.

Be sure to slow down OpenLog as much as possible (9600bps) and speed up the Arduino as much as possible (57600 or 115200).

[simonbarker]

Thanks for getting back, so I've connected it up to a keyspan adapter for the first time and when set to 5V I can control the openLogger create files, append, change files etc but I can't read information off it - in fact when first powering up the openLog I also don't receive the expected 12< but I do receive ...

See image below for my setup

At 3.3V i get a continuous ..11..11..11..11..11..11..11 and pressing CTRL-Z three times doesn't do anything. At 5V I can control the card but can't read anything - I know the openlog can output stuff though as if I turn it on without a card in I get an "Error card init" message in the terminal.

Setup is 9600, parity 1, no flow control, using CoolTerm on Mac. I've tried with and without local echo, terminal mode is RAW mode and Enter Key Emulation has been tried in CR+LF, CR and LF modes - all produce the same results described above. Baud is set to 9600 in the config.txt file.

Also, with the hardware serial port method I mention previously I was using a 500 byte buffer to completely take in information from the openLog to then be sent out in one go on a software serial port (via the photographed KeySpan) and it worked fine. In this instance using asymmetric serial port speeds wouldn't have made any difference - the problem occurs during the read from the openLog into the software buffer that I created.

I'll post code once we've worked out this problem as it feels important to get it working through the kingspan.

[simonbarker]

For reference, this is the kind of output I get when interfacing the openlog with my Arduino

430,33216.00,26.44,25.94,4.79,0.25,509.87, 430,33646.00,26.44,25.94,3.35,0.25,495.80, 430,34076.00,26.44,26.00,5.74,0.25,506.74, 430,34506.00,26.44,25.94,1.44,0.25,480.16, 430,34936.00,26.44,26.00,-5.91,0.25,456.70, 430,35366.00,.25,508.31, 430,37086.00,26.44,26.00,2.55,0.25,484.85, 430,37516.00,26.44,26.00,1.28,0.25,472.34, 430,,26.44,25.94,5.5unknown command: 1493,40299.00,26.44,25.94,5.5 !>

[simonbarker]

PROBLEM SOLVED;

I was using an interrupt service routine during which I wrote to the card. Turns out I wasn't disabling this interrupt when reading, so eventually the read gets interrupted by a command to write which it never leaves.

Apologies for what is really quite a stupid error on my part and thanks for the help

[nseidle]

Thank you for coming back and letting us know what the solution was.