Read data from CircularBuffer without removing from buffer

[aaraujo11]

Hi @m-mcgowan

I'm using the circular buffer method to save msg blocks of uint8_t data. And i notice that i'm only able to read it once, and after that the buffer went empty. There is any way to control if you want or not remove the data from the buffer, by reading from the buffer queue without removing from it.

I already try to manipulate the size_ variable in: https://github.com/m-mcgowan/spark-flashee-eeprom/blob/master/firmware/flashee-eeprom.h#L246

Thanks for develop this extremely useful library :)

[m-mcgowan]

It's fairly easy to implement a peek() method that works likeread() but doesn't advance any of the internal pointers after the data has been returned, so the data can be re-read.

uint8_t buf[100];
CircularBuffer* cb;
page_size_t available = cb->peek(buf, 100);

This would read up to 100 characters into buf, without advancing the read pointer. (So subsequent reads would return the same data.)

I hope that gives you what you need!

I have my hands really full, but give me a couple of evenings and I should have this available for you.

[aaraujo11]

Hi @m-mcgowan,

Thanks for the answer, but some how i misunderstand the purpose of the circularbuffer, because i understand that is used for permanent storage.

I basically want to save sequential blocks of 3626 bytes time to time (or the full page of 4096 bytes) , if i don't have connection to my server, when i have connection i simply want to read and send that msg blocks in the same order as saved in memory. Also i need to allocate in memory the idx of the number saved of blocks. And i want to use all memory available from 0x80000 to 0x200000.

What is your opinion of the best approach to keep a good usage of the memory.

Thanks!

[m-mcgowan]

Hi,

Just some background - you don't need to write a full page, the system will work whatever size you write, although writing a full page at a time is more efficient. If you chose to do this, use pageSize() to find out how large a page is - it may change in future!

The circular buffer is not persistent - the data isn't available after reset - while the data is saved to flash, the read and write pointers are not saved. (Writing these to flash after each change would wear the flash quickly.)

Your use case is the exact reason why I created the circular buffer - to allow data to be buffered when the system is offline. So you have one part of your code put data into the buffer, and another part read it from the buffer and send to the network (when online.) Data you write to the buffer comes out in the same order when you read from it.

The flashee library frees you from working with absolute memory addresses. (So forget about 0x80000 etc..) To create a buffer using all the available user flash you can write

CircularBuffer* buf = Devices::createCircularBuffer(0, Devices::userFlash().length());

The basic way to making this work online or offline is:

1. generate a block of data to send over the network
2. append the data to the buffer
3. when the core is online and there are data blocks in the buffer, then read each block and send over the network.

You can expand the scheme so that when the core is online and the buffer is empty, the data isn't put into the buffer at all but sent directly over the network. This avoids writing to the flash when it's not needed.

You can know if data is in the buffer because read() will return 0.