Code does not write binary file correctly

[marcelvanherk]

Hi Paul,

I am reading an historic floppy disk image and dumping to to SD in binary format, code snippet:

    File myFile;
    myFile = SD.open(fname.c_str(), FILE_WRITE);
    myFile.seek(0);

    for (int sector=0; sector<18*80; sector++) {
      byte dat[512];
      readit(sector, dat);
      dumpFast(dat, 512);
      myFile.write(dat, 512);
    }
    myFile.close();

My dumpFast routine shows on Serial port, e.g.,

41 55 54 4F 45 58 45 43 42 41 54 20 00 00 00 00 AUTOEXECBAT .... 00 00 00 00 00 00 00 00 21 00 2B 00 80 00 00 00 ........!.+.�...

The file on the SD shows in notepad++ hex editor (something like):

41 55 54 4F 45 58 45 43 42 41 54 20 00 00 00 00 AUTOEXECBAT .... 00 00 00 00 00 00 00 00 21 00 2B 00 d0 90 00 00 ........!.+.�... 00

I.e. non-ascii characters like 0x080 are replaced by UTF8 0xd0 0x90. This seems to happen consistently.

Using teendyduino 1.57 in arduino 1.8.19, on Teensy 3.5

Can you shed light on why this happens and how to avoid this, I expected write() to be binary.

Thanks, Marcel

[PaulStoffregen]

I tried but could not reproduce the problem. Since you didn't get a complete program I can actually run, I created this small program with the data you showed.

#include <SD.h>

byte dat[512] = {
  0x41, 0x55, 0x54, 0x4F, 0x45, 0x58, 0x45, 0x43,
  0x42, 0x41, 0x54, 0x20, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x21, 0x00, 0x2B, 0x00, 0x80, 0x00, 0x00, 0x00
};

const int chipSelect = BUILTIN_SDCARD;

void setup()
{
  // Open serial communications and wait for port to open:
  Serial.begin(9600);
  while (!Serial) {
    ; // wait for serial port to connect.
  }

  Serial.print("Initializing SD card...");

  // see if the card is present and can be initialized:
  if (!SD.begin(chipSelect)) {
    Serial.println("Card failed, or not present");
    while (1) {
      // No SD card, so don't do anything more - stay stuck here
    }
  }
  Serial.println("card initialized.");

  File myFile;
  myFile = SD.open("TEST.BIN", FILE_WRITE);
  myFile.seek(0);
  myFile.write(dat, 32);
  myFile.close();
  Serial.println("wrote data to card");

}

void loop()
{
}

When I run it on a Teensy 3.5 with a SD card in the socket, I get this in the serial monitor.

Then when I remove the card and put it into a card reader connected to my PC, this is what I see when I read the card.

The file was stored as exactly 32 bytes, with 0x80 in the proper place. I could not reproduce this UTF8 translation problem.

Could you please give this simple test program a try and see if it results in 32 or 33 bytes written to the file when you read with a PC?

[marcelvanherk]

Hi,

Thanks for your extensive answer. I have been trying to strip down the program, only to discover that my code does not overwrite data in an existing file, i.e. I saw no change whatever I did, wasting a bit of time. Do I need to delete files as per my example above before I can write into again? No clue yet what triggers the issue - it shows in the full program (using interrupts and u8g2 library) but not in stripped down versions. I do define the buffer as volatile.

Marcel

[marcelvanherk]

Testcpu86.zip This is the full code if you are interested. It intends to emulate a 40 year old WD1002 controller, but it fails when dumping a floppy on button press - the Serial output is correct by the SD card file is not.

[PaulStoffregen]

Can you create a program which reproduces the problem without requiring extra hardware beyond the SD card?

[marcelvanherk]

Hi,

I am still trying. It seems that myFile.seek(0) misbehaves sometimes, I just saw it insert one character into the file when using on a just created file. So that call may be involved. For now if I delete the file first and then write it, without seeking things seem go alright. I'll let you know if I see other strange behavior. Thanks a lot.

[PaulStoffregen]

Sorry, I can't put more time into this, at least not without a program which I can run on a Teensy (without needing other hardware) to reproduce the problem.

[marcelvanherk]

I understand. One theory that I have is that my buffer is volatile, and that therefore it is maybe calling the wrong write function. When I copy the data to a non volatile buffer first I see no issues. Thank for your help!

On Wed, 15 Feb 2023, 00:12 Paul Stoffregen, @.***> wrote:

Sorry, I can't put more time into this, at least not without a program which I can run on a Teensy (without needing other hardware) to reproduce the problem.

— Reply to this email directly, view it on GitHub https://github.com/PaulStoffregen/SD/issues/46#issuecomment-1430563120, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAVDWJKRCRMPFUPSSMB4IYLWXQNNBANCNFSM6AAAAAAUTWP23E . You are receiving this because you authored the thread.Message ID: @.***>