Bug in buffered reading?

[ullix]

I am using your lib with an ESP32-Wroom-Dev and an EEPROM ATMEL188 24C256. Reading and Writing generally ok, but:

When I do reads like x256PRM.get(i, buf), with buf being a byte buffer from 16, 32, 64, 128, ..., 16k, I almost always (95% of the time) get a sequence of 0xFF somewhere in the bytes read from the buffer. This sequence length is varying, ranging from 3 to 15 - clearly not always powers of 2 - and they seem to appear an random places within the EEPROM memory.

When I read that very modul via single-byte-reads, no FFs are present. Do I then follow with a buffered read, FFs are there again, but perhaps at a different location, and different sequence length.

I tried adding delays, but that did not help. I use page_size 64, write_time 5. My reading speed turns out at ~0.1ms/byte.

Can you help?

[ullix]

Just an example: 2 consecutive reads from the same EEPROM; look for the 2 different fff sequences in what is supposed to be identical text (a 0xFF byte read from EEPROM was replaced with 'f'):

`<1st read>

DateTime from AMEpocheTime UTC: 2020-02-28 13:48:44 f80 ESP32-WROOM-Dev , Saves, Chip t[s], d[us], Sum, NTP-AM,(mi,ma), NTP-RTC,(mi,ma), dur[ms],( mi, ma), Ipts, Heapfree[kB] 1000 2020-02-28 13:49:44, 1, 71, -2, -2, 9,( 9, 9), 0,( 0, 0), 139,( 1, 139), 0(2), 148(107,278) 1080 2020-02-28 13:50:44, 2, 131, -1, -3, 9,( 9, 9), 0,( 0, 0), 138,( 1, 139), 0(2), 148(107,278) 1100 2020-02-28 13:51:44, 3, 191, 1, -2, 9,( 9, 9), 0,( 0, 0), 140,( 1, 140), 0(2), 148(107,278) 1180 2020-02-28 13:52:44, 4, 251, 0, -2, 9,( 9, 9), 0,( 0, 0), 142,( 1, 142), 0(2), 148(107ffffff1200 2020-02-28 13:53:44, 5, 311, 0, -2, 9,( 9, 9), 0,( 0, 0), 138,( 1, 142), 0(2), 148(107,278) 1280 2020-02-28 13:54:44, 6, 371, 0, -2, 9,( 9, 9), 0,( 0, 0), 138,( 1, 142), 0(2), 148(107,278)

<2nd read> DateTime from AMEpocheTime UTC: 2020-02-28 13:48:44 f80 ESP32-WROOM-Dev , Saves, Chip t[s], d[us], Sum, NTP-AM,(mi,ma), NTP-RTC,(mi,ma), dur[ms],( mi, ma), Ipts, Heapfree[kB] 1000 2020-02-28 13:49:44, 1, 71, -2, -2, 9,( 9, 9), 0,( 0, 0), 139,( 1, 139), 0(2), 148(107,278) 1080 2020-02-28 13:50:44, 2, 131, -1, -3, 9,( 9, 9), 0,( 0, 0), 138,( 1, 139), 0(2), 148(107,278) 1100 2020-02-28 13:51:44, 3, 191, 1, -2, 9,( 9, 9), 0,( 0, 0), 140,( 1, 140), 0(2), 148(107,278) 1180 2020-02-28 13:52:44, 4, 251, 0, -2, 9,( 9, 9), 0,( 0, 0), 142,( 1, 142), 0(2), 148(107,278) 1200 2020-02-28 13:53:44, 5, 311, 0, -2, 9,( 9, 9), 0,( 0, 0), 138,( 1, 142), 0(2), 148(107,278) 1280 2020-02-28 13:54:44, 6, 371, 0, -2, 9,( 9, 9), 0,( 0, 0), 138,( 1, 142), 0(2), 148(107,278) ... 2000 2020-02-28 13:49:44 appendBinFile: FFat: /logBin size:8880 + 148 T[ms]: O:1.9 W:0.2 F:99 C:0.2 T:102 (101,101) 2080 2020-02-28 13:50:44 appendBinFile: FFat: /logBin size:9028 + 148 T[ms]: O:2.0 W:0.1 F:99 C:0.1 T:102 (101,101) 2100 2020-02-28 13:51:44 appendBinFile: FFat: /logBin size:9176 + 148 T[ms]: O:2.0 W:0.1 F:101 C:0.1 T:104 (101,103) 2180 2020-02-28 13:52:44 appendBinFile: FF**ffffffffffffffffffffff**+ 148 T[ms]: O:2.0 W:0.1 F:103 C:0.1 T:106 (101,105) 2200 2020-02-28 13:53:44 appendBinFile: FFat: /logBin size:9472 + 148 T[ms]: O:2.0 W:0.1 F:99 C:0.1 T:102 (101,105) 2280 2020-02-28 13:54:44 appendBinFile: FFat: /logBin size:9620 + 148 T[ms]: O:2.0 W:0.1 F:99 C:0.1 T:102 (101,105) `

[ullix]

closing was not intended

[nseidle]

Thanks for reporting! Sounds like you've stumbled onto something bad.

Do you have a full part# so I can pull the datasheet?

Do you have a minimal example sketch that I can use that demos the problem? I'll try to replicate.

[nseidle]

Here's the datasheet I'm looking at but the part looks obsolete so I'd like your full part #.

[ullix]

This is the complete imprint on the chip: ATMEL188 24C256 PU27 D.

Code to follow shortly.

[nseidle]

PU is DIP 27 is 2.7V so the max I2C clock is 400kHz

ATMEL188 I'm trying to make into the mfg year and week but that doesn't seem to fit their spec:

If the chip is from 2011 I might be a bit suspicious. If it's from 2018 I doubt the hardware is the problem.

[ullix]

It is this product (though I doubt it helps any further): https://www.amazon.de/gp/product/B07XTBXGXW/ref=ppx_yo_dt_b_asin_title_o06_s00?ie=UTF8&psc=1

[ullix]

I believe this is all the relevant code. Same effect no matter what the buffer size. I went from very big buffer to smaller because of timing issues, but that is a different matter.

`#include "SparkFun_External_EEPROM.h"

define XCFG256_START 0 // Start position in EEPROM to store setting structure

//#define XCFG256_SIZE 32768 // Size of EEPROM; 24C256 has 256k bit= 32768 byte //TESTING

define XCFG256_SIZE 16384 // Size of EEPROM; 24C256 has 256k bit= 32768 byte

define XCFG256_PAGE_SIZE 64 // Page Size of EEPROM; 24C256 has 64 byte

define XCFG256_PAGE_WRITE_TIME 5 // Page Write Time of EEPROM; 24C256 has 5 ms

define XCFG256_ADRESS 0x50 // I2C address of EEPROM 24C256 at 0x50

ExternalEEPROM x256PRM; // external EEPROM

//--------------------------

const uint16_t ddfull = 0x4000; // = 16384 = 128 * 128 -- only half size used for testing const uint16_t sbufsize = 64; // try 16, 32, 64, 128, 256, ... all the same problem byte smallbuf[sbufsize]; byte myBuffer[ddfull]; // use buffer to read full EEPROM into

for (int i = 0; i < ddfull; i += sbufsize){ x256PRM.get(i, smallbuf); for (int j = 0; j < sbufsize; j++){ // read chunks myBuffer[i + j] = smallbuf[j]; } yield(); }
`

The problems show when I print chars from myBuffer, but not when I read single bytes from EEPROM.

[ullix]

I became curious and lifted one DIP out of its socket. On its underside I see the text 18 18 33. Helpful?

[ullix]

I just converted my code to support another EEPROM, ATHYT532 24C32N SU27 D (part of an RTC), and the same problem shows, note the prominent ffff:

DateTime from NTP-Epoche180 ESP32-WROOM-Dev , Saves, Chip t[s], d[us], Sum, NTP-AM,(mi,ma), NTP-RTC,(mi,ma), dur[ms],( mi, ma), Ipts, Heapfree[kB] 200 2020-02-28 17:11:01, 1, 68, 0, 0, 5,( 5, 5), 0,( 0, 0), 135,(135, 135), 0(2), 160(122,290) 280 2020-02-28 17:12:01, 2, 128, 0, 0, 5,( 5, 5), 0,( 0, 0), 203,( 1, 203), 0(2), 160(116,290) 300 2020-02-28 17:13:01, 3, 187, 1, 1, 5,( 5, 5), 0,( 0, 0), 131,( 1, 203), 0(2), 159(116,290) ffffffffffffffffffffffffffffffff 400 2020-02-28 17:11:01 appendBinFile: FFat: /logBin size:36704 + 148 T[ms]: O:1.9 W:0.1 F:98 C:0.1 T:101 (100,100) 480 2020-02-28 17:12:01 appendBinFile: FFat: /logBin size:36852 + 148 T[ms]: O:2.0 W:51.1 F:116 C:0.1 T:170 (100,169) 500 2020-02-28 17:13:01 appendBinFile: FFat: /logBin size:37000 + 148 T[ms]: O:2.0 W:0.2 F:95 C:0.1 T:98 (97,169)

[nseidle]

I've got a 24LC256 and can't replicate the issue on Uno or Artemis.

Have you tried the Examples->Ex5_InterfaceTest? If that's failing for you I suspect it is platform dependent and it's time to break out the logic analyzer.

Here's my test sketch for reference.

include

include "SparkFun_External_EEPROM.h" // Click here to get the library: http://librarymanager/All#SparkFun_External_EEPROM

ExternalEEPROM myMem;

void setup() { Serial.begin(115200); Serial.println("I2C EEPROM example test 123");

//Pick any unconnected analog pin randomSeed(analogRead(A29)); //randomSeed(analogRead(A15));

unsigned int randomLocation;

Wire.begin(); Wire.setClock(400000); //Wire.setClock(1000000);

if (myMem.begin() == false) { Serial.println("No memory detected. Freezing."); while (1); } Serial.println("Memory detected!");

Serial.println(""); Serial.println("Buffer Write Test");

long startTime; long endTime; bool allTestsPassed = true;

//Buffer write test //-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- //char myChars[242] = "Lorem ipsum dolor sit amet, has in verterem accusamus. Nulla viderer inciderint eum at. Quo elit nullam malorum te, agam fuisset detracto an sea, eam ut liber aperiri. Id qui velit facilisi. Mel probatus definitionem id, eu amet vidisse eum."; char myChars[88] = "Lorem ipsum dolor sit amet, has in verterem accusamus. Nulla viderer inciderint eum at."; randomLocation = random(0, myMem.length());

Serial.print("Calculated time to record array of " + (String)sizeof(myChars) + " characters: ~"); Serial.print((uint32_t)sizeof(myChars) / myMem.getPageSize() * myMem.getPageWriteTime()); Serial.println("ms");

startTime = micros(); //myMem.put(randomLocation, myChars); myMem.put(0, myChars); endTime = micros(); Serial.println("Time to record array: " + (String)(endTime - startTime) + " us");

char readMy[sizeof(myChars)];

startTime = micros(); //myMem.get(randomLocation, readMy); myMem.get(0, readMy); endTime = micros(); Serial.println("Time to read array: " + (String)(endTime - startTime) + " us");

Serial.println("Location " + (String)randomLocation + " string should read:"); Serial.println(myChars); Serial.println(readMy); if (strcmp(myChars, readMy) != 0) { Serial.println("String compare failed"); allTestsPassed = false; } //-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

const uint16_t ddfull = 1024;//128128; // = 16384 = 128 128 -- only half size used for testing const uint16_t sbufsize = 64; // try 16, 32, 64, 128, 256, ... all the same problem byte smallbuf[sbufsize]; byte myBuffer[ddfull]; // use buffer to read full EEPROM into

for (int i = 0; i < ddfull; i += sbufsize) { myMem.get(i, smallbuf); for (int j = 0; j < sbufsize; j++) { // read chunks myBuffer[i + j] = smallbuf[j]; } }

for (int j = 0; j < ddfull; j++) { Serial.write(myBuffer[j]); } Serial.println();

Serial.println(); Serial.print("Memory Contents:"); for (uint16_t x = 0; x < 32 * 4; x ++) { if (x % 16 == 0) Serial.println(); Serial.print(" 0x"); if (myMem.read(x) < 0x10) Serial.print("0"); Serial.print(myMem.read(x), HEX); } Serial.println();

if (allTestsPassed == true) Serial.println("All tests PASSED!"); else Serial.println("Something went wrong. See output."); }

void loop() {

}

and output

[ullix]

Tested your code and it worked fine. Shrunk down my own code to the bare minimum - and it also worked fine. Yet, my big code still has the problem! Hmmm.

Eventually I got it. I tried to get some timing advantage by putting the EEPROM-reading code in a separate task (on Core 1, just as setup and loop). And this - what is it: Multitasking? Multithreading? - does interfere with reading the EEPROM. Code is below.

My wild guess: it is the Wire lib. Is that possible? Or could you put some remedy into the EEPROM lib?

The reason why I tried this tasking is that the time it takes to read the EEPROM gives me some difficulties. How can this be sped up? Does your lib work also for FRAM or only the good- ol' EEPROMs? FRAM can be written faster, but as long as it is I2C based, I am afraid they may not be faster to read? (Writing time is not an issue for me).

Any better ideas?

The code follows: A full 32k Bytes of '.' (dot) is written to EEPROM in setup and then read and printed in the loop. All ok. Next is to read the EEPROM via a task started in the loop. The problem is easily seen in the printing.

Comment out the very last code line to see it working just fine.

` /*

• Demonstrating problems with Multi-Tasking/Threads in Wire (EEPROM?) */

  include

  include "SparkFun_External_EEPROM.h"

  ExternalEEPROM myMem;

  define XCFG256_START 0 // Start position in EEPROM to store setting structure

  define XCFG256_SIZE 32768 // Size of EEPROM; 24C256 has 256k bit= 32768 byte

  define XCFG256_PAGE_SIZE 64 // Page Size of EEPROM; 24C256 has 64 byte

  define XCFG256_PAGE_WRITE_TIME 5 // Page Write Time of EEPROM; 24C256 has 5 ms

  define XCFG256_ADRESS 0x50 // I2C address of EEPROM 24C256 at 0x50

  ExternalEEPROM x256PRM; // external EEPROM

  byte Buffer1[XCFG256_SIZE]; byte Buffer2[XCFG256_SIZE];

  bool flagBadChar = false; uint16_t loopCounter = 0; uint16_t badCharCounter = 0;

  void initEEPROM(){ Serial.println("Initializing external EEPROM with SparkFun library"); if (x256PRM.begin(XCFG256_ADRESS) == false) { Serial.println(F(" No external EEPROM detected. ")); return; } // Settings for this EEPROM x256PRM.setMemorySize (XCFG256_SIZE); x256PRM.setPageSize (XCFG256_PAGE_SIZE); x256PRM.enablePollForWriteComplete (); // valid for 24C256 ??? x256PRM.setPageWriteTime (XCFG256_PAGE_WRITE_TIME);

  Serial.printf("   EEPROM memory size [bytes]: %i \n", x256PRM.length());

  }

  void printBuffer(byte *buffer){

  Serial.print("printBuffer: ");
  for (int i = 0; i < XCFG256_SIZE; i++) {
      byte rbyte = cleanByte(buffer[i]);
      if (i % 192 == 0) Serial.printf("\n%6i (%4X): ", i, i);
      Serial.printf("%c", rbyte);
  }    
  Serial.println();

  }

  byte cleanByte(byte rbyte){

  if      (rbyte >= 32 && rbyte < 128);       // ASCII, ok
  else if (rbyte == 0x0A) ;                   // LF, ok
  else if (rbyte == 0x00) {flagBadChar = true; rbyte = 'O';}
  else if (rbyte == 0xFF) {flagBadChar = true; rbyte = 'F';}
  else                    {flagBadChar = true; rbyte = 'X';}
  
  if (flagBadChar) {
      badCharCounter++;
      flagBadChar = false;
  }
  
  return rbyte;

  }

  void fillBufferFromEEPROM(void * parameter){ // in SEPARATE TASK: read EEPROM and put values into Buffer1

  Serial.println("TASK: fillBufferFromEEPROM Begin:");
  
  x256PRM.get(0, Buffer2); 
  
  // clean the bytes 
  for (int i = 0; i < XCFG256_SIZE; i++) Buffer2[i] = cleanByte(Buffer2[i]);
  
  printBuffer(Buffer2);
  
  vTaskDelete(NULL);

  }

  void makeEEPROMTask(){ // use specific core

  xTaskCreatePinnedToCore(
      fillBufferFromEEPROM,   // Function that will be called
      "EEPROM_TASK",          // Name of the task
      10000,                  // Stack size (bytes)
      NULL,                   // Parameter to pass
      2,                      // Task priority 
      NULL,                   // Task handle
      1);                     // pin task to core 1 (it crashes on core 0 sometimes)

  }

  void setup(){ Serial.begin(115200); delay(1599);

  Wire.begin(25,26);
  //Wire.setClock(100000);
  //Wire.setClock(400000);
  Wire.setClock(1000000);
  Serial.printf("Wire setting: Clock: %i Hz\n", Wire.getClock());
  
  initEEPROM();
  
  auto tA = micros();
  for (int i = 0; i < XCFG256_SIZE; i++)  Buffer1[i] = '.';
  auto tB = micros();
  Serial.printf("Buffer made in %1.3f ms\n", (tB-tA)/1000.0);
  
  auto t0 = micros();
  myMem.put(0, Buffer1);
  auto t1 = micros();
  Serial.printf("Buffer put to EEPROM in %1.1f ms  (ms / byte: %1.3f)\n", (t1-t0)/1000.0, (t1-t0)/1000.0/XCFG256_SIZE);
  printBuffer(Buffer1);    
  
  Serial.println("******************** Setup done ************************************\n\n");

  }

  void loop(){ loopCounter++; Serial.printf("\nLoop count is: %i ----------------------------------------------------------------------\n", loopCounter);

  auto t2 = micros();
  myMem.get(0, Buffer2);
  auto t3 = micros();
  Serial.printf("Buffer got from EEPROM in %1.1f ms  (ms / byte: %1.3f)\n", (t3-t2)/1000.0, (t3-t2)/1000.0/XCFG256_SIZE);
  printBuffer(Buffer2);
  
  Serial.printf("Number of Bad chars detected: %i \n", badCharCounter); // war 0 auch nach ~2000 loops
  
  // inactivate next line to see it reading EEPROM just fine :-/
  makeEEPROMTask();

  }

`

[nseidle]

Works for me. Closing stale issue.