Open technoblogy opened 3 years ago
Hi!
I wrote this crude sketch five years ago, so a lot has happened to my coding skills since since then. I haven't really used this functionality myself either, but I'll try to answer your questions as good as I can. Note that this sketch relies on optiboot.h, which is mostly written by @majekw. This is the sketch](https://github.com/majekw/optiboot/blob/spm/optiboot/examples/flash_program/flash_program.ino) SerialReadWrite.ino is based on. As you can see in this sketch, here's where I've copied all the "important things to know" text.
I'm not sure what you're referring to by 'buffer' in the documentation in the header of SerialReadWrite.ino. When you say: Buffer must be page aligned (see declaration of flash_buffer) There is no flash_buffer in the code. I assume this means the block of flash you're writing to, flashSpace[]?
Yes, I'm referring to flashSpace[] here.
In: Writing to EEPROM destroys temporary buffer You can write only once into one location of temporary buffer By 'temporary buffer' do you mean: uint8_t ramBuffer[SPM_PAGESIZE]; and by EEPROM do you mean flash? Why does it get destroyed by writing it to flash?
When looking at the code now, there is no reason why the ramBuffer
should be destroyed when writing to EEPROM. ramBuffer
is just an array living in RAM, and should not be affected. But it would be great if you could try this just to check if the statement from majekw still is true (for some reason).
Why might you want to do fill-erase-write rather than erase-fill-write?
I am doing erase-fill-write. See code here.
Instead of allocating the page(s) of flash in the middle of PROGMEM as you've done here, could I specify an explicit address for flashSpace? For example, if I understand correctly the bootloader occupies two pages from word addresses 0xFF00 to 0xFFFF. I would like to write to one page of flash just below that, from word addresses 0xFE80 to 0xFEFF. So could I do this?
I've not tried this myself, and I don't have a good answer to your question. However, you can always try this and use Avrdude to dump the flash and look at its content afterward to see if it did work? Perhaps maybe @majekw can give us a straight answer? 🙂
Is it essential to use a RAM buffer, ramBuffer[256], or if I'm short of RAM could I write the data to flash byte by byte, treating it more like an SD card? I realize I would have to provide my own alternative to optiboot_writePage().
As far as I know, you can only write a full page to the flash memory at once. You could of course create a wrapper that treats each page as a byte array, but you'll always have to "write" all the bytes, even though only one has changed.
Finally, I'm puzzled by the code in optiboot_readPage() in optiboot.h: ... Why is it skipping bytes if they are 0 or 255?
The code is based on the read routine from the original sketch. It may be that 0x00 or 0xff represents an empty space? But I do agree that 0 and 255 should be read out as well. If not, it's pretty much useless for anything else than characters.
I can change the optiboot.h and the SerialReadWrite sketch based on what you figure out what works and what don't. Looking forward to seeing what you'll come up with! Is it something that you'll write about on your blog?
Thanks for the answers! I'll let you know what I find out after I've done some experiments.
The application I've got in mind is to improve my version of uLisp for the ATmega1284P. Currently it allows you to save the workspace from RAM to the EEPROM on the chip, which limits you to saving 4Kbytes of the 16Kbytes. By saving it to flash instead I could save the entire RAM.
I use optiboot.h in the ArduinoOTA library to store the uploaded sketch binary in upper half of the flash memory. I don't use a memory buffer. The MCU has a buffer for the page. https://github.com/jandrassy/ArduinoOTA/blob/master/src/InternalStorageAVR.cpp
btw: fill-erase-flash is only possible in last 4 flash pages to support advanced bootloader burning
@jandrassy Thanks, I'll take a look at your code too.
Lot of questions :-)
OK, I've got it to work! Here is some feedback. This generally relates to the ATmega1284P and may need changing for the smaller chips.
I couldn't get anything to work at first until I remembered that my ATmega1284P board had an old bootloader on it from Jack Christensen's Mighty 1284P core. After updating the bootloader from your MightyCore everything started to work!
So I think it might be useful to provide a function like the one in Spence Konde's DxCore called Flash.checkWritable() which checks for obvious things. I'm not sure how to write this.
I got very confused about whether I was working with byte addresses or word addresses. It doesn't help that the memory map in the Atmel datasheet shows the top of memory on an ATmega1284P as 0xFFFF. It would be worth saying that all the addresses used by the Optiboot Flash routines are byte addresses.
The demo program SerialReadWrite.ino makes it appear that you have to allocate your own 256-byte RAM buffer, but this isn't necessary as there's a temporary buffer in the ATmega1284P that you can fill on the fly with optiboot_page_fill().
I don't think the function optiboot_readPage() should be included in optiboot.h as it's more of a demo function, and it wouldn't be useful for reading real data as it ignores 0x00 and 0xFF bytes. Also it wouldn't work in the upper half of flash; you need to use pgm_read_byte_far() for that. It's not really needed anyway as you can just use pgm_read_byte() or pgm_read_byte_far().
In the documentation there's a bit of confusion over the terminology. I suggest:
Flash buffer: the temporary page in flash filled by optiboot_page_fill(). Flash area: the page(s) of flash you are writing to (which must be page aligned). Don't call it a buffer (it's not temporary). RAM area: the area in RAM you're transferring to flash, although I suggest dropping this concept.
Finally, here are my demo programs which I've tried to keep as simple as possible.
/*
Test Optiboot Flash on ATmega1284P - explicit page
*/
#include "optiboot.h"
// Bootloader takes from 0x1fc00 to 0x1ffff (byte addresses).
// Write in page immediately below that
uint32_t BaseAddress = 0x1fb00;
void setup() {
Serial.begin(9600);
}
void loop() {
Serial.println("Erase");
optiboot_page_erase(BaseAddress);
Serial.println("Write data");
uint32_t addr = BaseAddress;
// Fill flash buffer. Page is 128 words
for (unsigned int i=0; i<128; i++) {
optiboot_page_fill(addr, i); addr++; addr++;
}
// Write flash buffer to flash area
optiboot_page_write(BaseAddress);
Serial.println("Read data");
addr = BaseAddress;
for (unsigned int i=0; i<128; i++) {
Serial.print(pgm_read_word_far(addr));
addr++; addr++; Serial.print(' ');
}
Serial.println();
delay(20000);
}
/*
Test Optiboot Flash on ATmega1284P - allocate page in PROGMEM
*/
#include "optiboot.h"
// Allocate one page of 256 bytes
const uint8_t flashSpace[256] __attribute__ (( aligned(256) )) PROGMEM = { };
uint32_t BaseAddress = (uint32_t)flashSpace;
void setup() {
Serial.begin(9600);
}
void loop() {
Serial.println("Erase");
optiboot_page_erase(BaseAddress);
Serial.println("Write data");
uint32_t addr = BaseAddress;
// Fill flash buffer. Page is 128 words
for (unsigned int i=0; i<128; i++) {
optiboot_page_fill(addr, i); addr++; addr++;
}
// Write flash buffer to flash area
optiboot_page_write(BaseAddress);
Serial.println("Read data");
addr = BaseAddress;
for (unsigned int i=0; i<128; i++) {
Serial.print(pgm_read_word(addr));
addr++; addr++; Serial.print(' ');
}
Serial.println();
delay(20000);
}
I forgot to say: thanks for providing this really useful feature!
@technoblogy thanks for the feedback! I'll go through it later and apply some of the improvements you've suggested.
BTW you can write to the upper part of flash by using the PROGMEM1 attribute:
const uint8_t flashSpace[SPM_PAGESIZE * NUMBER_OF_PAGES] __attribute__ (( aligned(SPM_PAGESIZE) )) PROGMEM1 = {
"This some default content stored on a page far away"
};
EDIT: Not, that this doesn't work for some reason. Haven't had the time to investigate why. I did change pgm_read_byte_far
to pgm_read_byte_far
in optiboot_readPage()
, but still no success. For reference, PROGMEM1 is defined here.
Here's another example, which is what I'm using in uLisp. You simply keep calling FlashWriteInt() to write values to the flash (up to 16Kbytes in this example), and the only other thing you have to do is call FlashEndWrite() at the end. It takes care of the correct sequence of optiboot_page_erase, optiboot_page_fill, and optiboot_page_write calls for you:
const uint32_t BaseAddress = 0x1bc00;
void FlashWriteInt (uint32_t *addr, int data) {
if (((*addr) & 0xFF) == 0) optiboot_page_erase(BaseAddress + ((*addr) & 0xFF00));
optiboot_page_fill(BaseAddress + *addr, data);
if (((*addr) & 0xFF) == 0xFE) optiboot_page_write(BaseAddress + ((*addr) & 0xFF00));
(*addr)++; (*addr)++;
}
void FlashEndWrite (uint32_t *addr) {
if (((*addr) & 0xFF) != 0x00) optiboot_page_write((BaseAddress + ((*addr) & 0xFF00)));
}
void loop() {
uint32_t addr = 0;
for (int word = 0; word<500; word++) FlashWriteInt(&addr, word);
FlashEndWrite(&addr);
for(;;);
}
Is there a "proper way" to read content that's stored in "far mem" (64-128kB)? I've tried to modify the optiboot_readPage function
, but it seems like the pgm_get_far_address
macro requires its parameter to be known at compile time. I'm getting this error:
/var/folders/6l/ypg6qbw172v1s4vtt6g990tw0000gn/T/arduino_build_544907/sketch/SerialReadWrite.ino.cpp.o: In function `optiboot_readPage(unsigned char const*, unsigned char*, unsigned int, char)':
/Users/hans/Documents/Arduino/hardware/MightyCore/avr/libraries/Optiboot_flasher/src/optiboot.h:137: undefined reference to `r30'
/Users/hans/Documents/Arduino/hardware/MightyCore/avr/libraries/Optiboot_flasher/src/optiboot.h:137: undefined reference to `r30'
/Users/hans/Documents/Arduino/hardware/MightyCore/avr/libraries/Optiboot_flasher/src/optiboot.h:137: undefined reference to `r30'
collect2: error: ld returned 1 exit status
Using library Optiboot_flasher at version 1.0
// Function to read a flash page and store it in an array (storage_array[])
void optiboot_readPage(const uint8_t allocated_flash_space[], uint8_t storage_array[], uint16_t page, char blank_character)
{
uint8_t read_character;
for(uint16_t j = 0; j < SPM_PAGESIZE; j++)
{
read_character = pgm_read_byte_far(pgm_get_far_address(allocated_flash_space) + j + SPM_PAGESIZE*(page-1));
if(read_character != 0 && read_character != 255)
storage_array[j] = read_character;
else
storage_array[j] = blank_character;
}
}
RAMZ register
The idea is to see if I can create a usable example where I utilize PROGMEM1
.
// This array allocates the space you'll be able to write to
static const uint8_t flashSpace[SPM_PAGESIZE * NUMBER_OF_PAGES] __attribute__ (( aligned(SPM_PAGESIZE) )) PROGMEM1 = {
"This some default content stored on page one"
};
I want to see if there's possible to read and write to the array when it's located in the far (64kB+) memory space. So far I've been able to read from it by "manually" passing it's memory address, but it's not very elegant. When I try to write to it I either corrupt the bootloader area or the program gets permanently corrupted.
Preferably, this is what I want to acheive:
// This array allocates the space you'll be able to write to
static const uint8_t flashSpace[SPM_PAGESIZE * NUMBER_OF_PAGES] __attribute__ (( aligned(SPM_PAGESIZE) )) PROGMEM1 = {
"This some default content stored on page one but still far away!"
};
// ...
// Write content to page 1:
optiboot_writePage(flashSpace, ramBuffer, 1);
// Read page 1 of the allocated space:
optiboot_readPage(flashSpace, ramBuffer, 1);
// Print page content
Serial.print(F("\nContent of page 1: "));
Serial.println((char*)ramBuffer);
maybe my copy_flash_pages function's source code can help https://github.com/MCUdude/MightyCore/blob/09909376fbf39305e1047bbc2efeb8f67f441620/avr/bootloaders/optiboot_flash/optiboot_flash.c#L1190-L1209
Preferably, this is what I want to achieve:
OK, I got it working, but I still can't get around the fact that the address to the array has to be known at compile-time and thus passed to the optiboot_readPage
and optiboot_writePage
functions. But this isn't really a deal-breaker. I'll continue working on some of the suggestions @technoblogy came up with! There may be breaking changes to optiboot.h, but these will be minor, and absolutely for the best (like getting rid of the 0 and 255 guards in optiboot_read_page for instance).
In the SerialReadWrite example I've just updated, you can now easily place all your data in the far memory using PROGMEM1.
All you have to do is to add pgm_get_far_address
when you're reading or writing:
const uint8_t flashSpace_near[SPM_PAGESIZE * NUMBER_OF_PAGES] __attribute__ (( aligned(SPM_PAGESIZE) )) PROGMEM1 = {
"This some default content stored on page one, really near!"
};
const uint8_t flashSpace_far[SPM_PAGESIZE * NUMBER_OF_PAGES] __attribute__ (( aligned(SPM_PAGESIZE) )) PROGMEM1 = {
"This some default content stored on page one, far, far away..."
};
// write to near mem
optiboot_writePage(flashSpace_near, ramBuffer, pageNumber);
// write to far mem
optiboot_writePage(pgm_get_far_address(flashSpace_far), ramBuffer, pageNumber);
// Read from near mem
optiboot_readPage(flashSpace_near, ramBuffer, page);
// Read from far mem
optiboot_readPage(pgm_get_far_address(flashSpace_far), ramBuffer, page);
I hope these changes to optiboot.h makes it more suitable for real-world applications 🙂 Feel free to bring some feedback! I'd love to further improve it.
Personally I find this a bit confusing, and would prefer it in a comment:
"This some default content stored on page one, really near!"
Why would you want to have some default content there? Also, it implies that you're only going to be storing text.
It's intended to show what the SerialReadWrite sketch is capable of. In the sketch, you can dump the entire allocated space, and one can see where the text is. As a demo, I think it helps to illustrate what the user is working with. It doesn't make sense in a read-world application, that's for sure.
I'm planning to create other examples as well that show how other data can be handled, with and without a buffer in RAM.
Do you have any ideas on what other examples could be useful to help users utilize the flash read/write feature in their own applications?
Perhaps my example?
https://github.com/MCUdude/MightyCore/issues/210#issuecomment-808773024
@technoblogy I will include a modified version of your example to show how one can write 16-bit integers straight to the internal buffer. It's very fast and light-weight, but not as user-friendly as I want it to be.
That's why I've created a wrapper library that utilizes Optiboot.h but acts very much like the Arduino EEPROM library! This means that you can easily store all sorts of things like variables, structs, strings, etc, and the library will take care of the low-level stuff. It is template-based, so the compiled size is still very small.
The compiled size without LTO for the example below is 6056 bytes for an ATmega1284P. Approx. 1.7kB is due to printing floats.
Any thoughts? I will publish the library after I've fine-tuned it even more 👍
// Flash_put_get.ino
#include <Flash.h>
struct MyObject
{
float field1;
uint8_t field2;
char name[10];
};
// RAM buffer needed by the Flash library
uint8_t ram_buffer[SPM_PAGESIZE] = {0x00};
// Allocate two flash pages for storing data
#define NUMBER_OF_PAGES 2
const uint8_t flashSpace[SPM_PAGESIZE * NUMBER_OF_PAGES] __attribute__((aligned(SPM_PAGESIZE))) PROGMEM = {};
// Flash constructor
Flash flash(flashSpace, sizeof(flashSpace), ram_buffer);
void write_data()
{
float f = 123.456f;
uint8_t buffer_address = 0;
// First, make sure there are no content in out buffer
flash.clear_buffer();
// One simple call, with the address first and the object second
flash.put(buffer_address, f);
Serial.println(F("Written float data type!"));
// Data to store
MyObject customVar =
{
3.14f,
65,
"MCUdude"
};
// Move address to the next byte after float 'f'
buffer_address += sizeof(float);
flash.put(buffer_address, customVar);
// Write buffer to the first allocated flash page (page 0)
flash.write_page(0);
// Now let's set a flag on another flash page to indicate that the flash memory contains content
// Here we're treating the object as an array
flash.clear_buffer();
flash[5] = 'X';
flash.write_page(1);
Serial.println(F("Written custom data type!\nReset your board to view the contents!\n"));
}
void read_data()
{
Serial.println(F("Read float from flash: "));
// fetch first flash page
flash.fetch_page(0);
float f = 0.00f; // Variable to store data read from flash
uint8_t buffer_address = 0; // Buffer address to start from
// Get the float data from flash at position 'buffer_address'
flash.get(buffer_address, f);
Serial.print(F("The value of f is now: "));
Serial.println(f, 3);
buffer_address += sizeof(float); // Move address to the next byte after float 'f'
MyObject customVar; // Variable to store custom object read from flash.
flash.get(buffer_address, customVar);
Serial.println(F("Read custom object from flash: "));
Serial.println(customVar.field1);
Serial.println(customVar.field2);
Serial.println(customVar.name);
}
void setup()
{
delay(2000);
Serial.begin(9600);
// Fetch flash page 1, where we may have a flag
flash.fetch_page(1);
// Check if our flag is present
if(flash[5] == 'X')
{
Serial.println(F("Content found! Content:"));
read_data();
}
else
{
Serial.println(F("No content found! Writing new content..."));
write_data();
}
}
void loop()
{
}
Serial monitor output after upload:
No content found! Writing new content...
Written float data type!
Written custom data type!
Reset your board to view the contents!
(I press the reset button)
Content found! Content:
Read float from flash:
The value of f is now: 123.456
Read custom object from flash:
3.14
65
MCUdude
I'm not sure why you've provided:
uint8_t ram_buffer[SPM_PAGESIZE] = {0x00};
because it suggests that you need a RAM buffer to use the Optiboot Flash, which isn't the case; a point I made in my comments at:
https://github.com/MCUdude/MightyCore/issues/210#issuecomment-808702140
One does not need a RAM buffer to read/write to flash, but it makes it much more convenient to have a pool you can read and write to, just like I've shown in the example.
If you have lots of structs and variables scattered around on a page, how would you work with it if you didn't have a buffer you can dump the content into before parsing the data? You may be able to reduce the RAM usage a little, but I bet the code is not going to be as pretty.
Optiboot.h does not require a RAM buffer, and I'm going to create an example showing this Flash.h will most likely require a RAM buffer because It's supposed to be easy to use and act much like the EEPROM library. I'd love to get rid of the buffer, but I don't really see how that could be accomplished. Is it possible to read from the internal flash buffer, so this could be used as a "pool"?
Fair enough.
Is it possible to read from the internal flash buffer, so this could be used as a "pool"?
I'm not sure.
Do you think it will be possible to implement this (from my earlier comments):
So I think it might be useful to provide a function like the one in Spence Konde's DxCore called Flash.checkWritable() which checks for obvious things. I'm not sure how to write this.
It's pretty important, because currently if you try to use Optiboot.h with the wrong bootloader it just crashes your program in a major way. It should at least check that you've got the correct bootloader.
Yes, that looks perfect.
The only concern I have is that, in order to use Flash::check_writable(), I have to create a Flash instance which appears to carry an overhead of 10 bytes, which I don't need. I would prefer it not to be a C++ routine.
I assume you're leaving optiboot.h and optiboot.cpp unchanged?
I was already thinking about this. See my latest commit.
BTW the reason why I created optiboot.cpp was because just having a header file made it a pain to include in other files. I constantly got "multiple definitions of..." errors. I also went with cpp since a few functions are overloaded, which is not supported in C.
OK, great. Now I can call optiboot_check_writable(). I assume optiboot_page_erase(), optiboot_page_fill(), and optiboot_page_write() haven't changed?
OK, great. Now I can call optiboot_check_writable(). I assume optiboot_page_erase(), optiboot_page_fill(), and optiboot_page_write() haven't changed?
Correct! Nothing has changed in these low-level functions, apart from me adding more comments in the code.
@technoblogy this is the example I'm going to provide that's based on your example code. It isn't identical at all, but it borrows your principles. I'm also demonstrating how to write 8-bit data to flash:
/***********************************************************************|
| Optiboot Flash read/write |
| |
| Read_write_without_buffer.ino |
| |
| A library for interfacing with Optiboot Flash's write functionality |
| Developed in 2021 by MCUdude |
| https://github.com/MCUdude/ |
| |
| In this low-level example we write 16-bit values to one flash page, |
| and 8-bit values to another page. What's different about this |
| example is that we do this without using a RAM buffer where the |
| contents are stored before writing to flash. Instead, the internal, |
| temporary buffer is used. By doing this we reduce RAM usage, but it |
| is not nearly as user friendly as using the Flash library. |
| |
| IMPORTANT THINGS: |
| - All flash content gets erased after each upload cycle |
| - Allocated flash space must be page aligned (it is in this example) |
| - Writing to EEPROM destroys temporary buffer so make sure you call |
| optiboot_page_write before reusing the buffer or using EEPROM |
| - You can write only once into one location of temporary buffer |
|***********************************************************************/
#include <optiboot.h>
// Workaround for devices that has 64kiB flash or less
#ifndef pgm_read_byte_far
#define pgm_read_byte_far pgm_read_byte_near
#endif
#ifndef pgm_get_far_address
#define pgm_get_far_address
#endif
// Allocate one flash pages for storing data. If you want to allocate space in the high progmem (>64kiB)
// You can replace PROGMEM with PROGMEM1
#define NUMBER_OF_PAGES 2
const uint8_t flashSpace[SPM_PAGESIZE * NUMBER_OF_PAGES] __attribute__((aligned(SPM_PAGESIZE))) PROGMEM = {};
// Function for writing 16-bit integers to a flash page
void flash_write_int(uint32_t base_addr, uint16_t offset_addr, int16_t data)
{
// Start by erasing the flash page before we start writing to the buffer
if ((offset_addr & 0xFF) == 0)
optiboot_page_erase(base_addr + (offset_addr & 0xFF00));
// Write the 16-bit value to the buffer
optiboot_page_fill(base_addr + offset_addr, data);
// Write the buffer to flash when the buffer is full
if ((offset_addr & 0xFF) == 0xFE)
optiboot_page_write(base_addr + (offset_addr & 0xFF00));
}
// Function to write bytes to a flash page
void flash_write_byte(uint32_t base_addr, uint16_t offset_addr, uint8_t data)
{
static uint8_t low_byte = 0;
if ((offset_addr & 0xFF) == 0)
optiboot_page_erase(base_addr + (offset_addr & 0xFF00));
if (!(offset_addr & 0x01)) // Address is 2, 4, 6 etc.
low_byte = data;
else // Address is 1, 3, 5 etc.
optiboot_page_fill(base_addr + offset_addr, (data << 8) | low_byte);
if ((offset_addr & 0xFF) == 0xFE)
optiboot_page_write(base_addr + (offset_addr & 0xFF00));
}
// Function to force a flash page write operation
void flash_end_write(uint32_t base_addr, uint16_t offset_addr)
{
// Write the buffer to flash if there are any contents in the buffer
if ((offset_addr & 0xFF) != 0x00)
optiboot_page_write(base_addr + (offset_addr & 0xFF00));
}
void setup()
{
delay(2000);
Serial.begin(9600);
static uint16_t addr = 0;
Serial.print(F("Filling up flash page 0 with 16-bit values...\n"));
// Fill the first flash page (page 0) with 16-bit values (0x100 to 0x01FF)
for(uint8_t data = 0; data < (SPM_PAGESIZE / 2); data++)
{
flash_write_int(pgm_get_far_address(flashSpace), addr, data + 0x0100); // Write data
addr += 2; // Increase memory address by two since we're writing 16-bit values
}
// Force an end write in case it hasn't already been done in flash_write_int
flash_end_write(pgm_get_far_address(flashSpace), --addr);
Serial.print(F("Filling up flash page 1 with 8-bit values...\n"));
// Fill the second flash page (page 1) with 0-bit values (0x00 to 0x0FF)
for(uint16_t data = 0; data < SPM_PAGESIZE; data++)
{
addr++; // Increase memory address by one since we're writing 8-bit values
flash_write_byte(pgm_get_far_address(flashSpace), addr, data); // Write data
}
// Force an end write in case it hasn't already been done in flash_write_byte
flash_end_write(pgm_get_far_address(flashSpace), addr);
Serial.print(F("Flash pages filled. Reading back their content.\nPage 0:\n"));
for(uint16_t i = 0; i < SPM_PAGESIZE; i += 2)
Serial.printf(F("Flash mem addr: 0x%05lx, content: 0x%04x\n"), pgm_get_far_address(flashSpace) + i, (pgm_read_byte_far(pgm_get_far_address(flashSpace) + i + 1) << 8) + (pgm_read_byte_far(pgm_get_far_address(flashSpace) + i)));
Serial.println(F("\n\nPage 1:"));
for(uint16_t i = 0; i < SPM_PAGESIZE; i++)
Serial.printf(F("Flash mem addr: 0x%05lx, content: 0x%02x\n"), pgm_get_far_address(flashSpace) + SPM_PAGESIZE + i, pgm_read_byte_far(pgm_get_far_address(flashSpace) + SPM_PAGESIZE + i));
}
void loop()
{
}
I think worth mentioning in the comment that you must call flash_end_write() when you've finished, otherwise you may end up with the last page not written.
I think worth mentioning in the comment that you must call flash_end_write() when you've finished, otherwise you may end up with the last page not written.
Thanks! I'll point this out
For some reason, the example sketch I just posted doesn't work with devices that has a flash page size of 128 bytes or less, which is all chips with 32kiB flash or less. It does compile just fine, but the data isn't written to flash properly. I guess I have a late night ahead of me...
For some reason, the example sketch I just posted doesn't work with devices that has a flash page size of 128 bytes or less, which is all chips with 32kiB flash or less. It does compile just fine, but the data isn't written to flash properly. I guess I have a late night ahead of me...
don't use _far_
for those devices.
don't use far for those devices.
_far_
isn't even defined for these devices, that's why I added:
#ifndef pgm_read_byte_far
#define pgm_read_byte_far pgm_read_byte_near
#endif
#ifndef pgm_get_far_address
#define pgm_get_far_address
#endif
...at the beginning of the sketch. And the code works fine on an ATmega644P, which only has "near mem", but has 256-byte flash pages.
I think it might be the tests for the first and last word in the page:
if ((offset_addr & 0xFF) == 0)
if ((offset_addr & 0xFF) == 0xFE)
These assume the page size is 0xFF. For a page size of 0x7F the second test should be:
if ((offset_addr & 0xFF) == 0x7E)
Also, the test in the flash_write_byte() function for a page size of 0xFF should be:
if ((offset_addr & 0xFF) == 0xFF)
and for a page size of 0x7F it should be:
if ((offset_addr & 0xFF) == 0x7F)
@technoblogy thanks for the input, but for some reason, I'm still getting garbage in the Serial monitor.
In the first place, I'm writing 16-bit integers starting from 0x0100 to 0x17F. The output looks like this:
Page 0:
Flash mem addr: 0x00180, content: 0x0100
Flash mem addr: 0x00182, content: 0x0302
Flash mem addr: 0x00184, content: 0x0504
Flash mem addr: 0x00186, content: 0x0706
Flash mem addr: 0x00188, content: 0x0908
Flash mem addr: 0x0018a, content: 0x0b0a
Flash mem addr: 0x0018c, content: 0x0d0c
Flash mem addr: 0x0018e, content: 0x0f0e
Flash mem addr: 0x00190, content: 0x1110
Flash mem addr: 0x00192, content: 0x1312
Flash mem addr: 0x00194, content: 0x1514
...
...
Flash mem addr: 0x001fc, content: 0x7d7c
Flash mem addr: 0x001fe, content: 0x7f7e
For the second page I'm writing 8-bit values from 0x00 to 0x7F. This is the output:
Page1:
Flash mem addr: 0x00200, content: 0x00
Flash mem addr: 0x00201, content: 0x00
Flash mem addr: 0x00202, content: 0x00
Flash mem addr: 0x00203, content: 0x00
Flash mem addr: 0x00204, content: 0x00
Flash mem addr: 0x00205, content: 0x00
Flash mem addr: 0x00206, content: 0x00
Flash mem addr: 0x00207, content: 0x00
...
...
Flash mem addr: 0x0027e, content: 0x00
Flash mem addr: 0x0027f, content: 0x00
What chip (with what page size) are you using?
This test was done on an ATmega16 with a page size of 128 bytes. But I have tried with an ATmega324P (128 bytes page size), and I'm getting the exact same output. But I have every MightyCore compatible chip in my drawer if that could help...
Are there conditions under which it does work?
dump whole flash with avrdude and check if it is problem with writing or reading
Are there conditions under which it does work?
I have not been able to make it work on any chip with 64 or 128 bytes page size.
dump whole flash with avrdude and check if it is problem with writing or reading
Here are the relevant parts of the flash dump:
:200160002D6269742076616C7565732E2E2E0A0000000000000000000000000000000000CF
:20018000000102030405060708090A0B0C0D0E0F101112131415161718191A1B1C1D1E1F6F (Addr: 0x0180 - first page)
:2001A000202122232425262728292A2B2C2D2E2F303132333435363738393A3B3C3D3E3F4F
:2001C000404142434445464748494A4B4C4D4E4F505152535455565758595A5B5C5D5E5F2F
:2001E000606162636465666768696A6B6C6D6E6F707172737475767778797A7B7C7D7E7F0F
:200200000000000000000000000000000000000000000000000000000000000000000000DE (Addr: 0x0200 - second page)
:200220000000000000000000000000000000000000000000000000000000000000000000BE
:2002400000000000000000000000000000000000000000000000000000000000000000009E
:2002600000000000000000000000000000000000000000000000000000000000000000007E
:20028000700511241FBECFEFD8E0DEBFCDBF11E0A0E0B1E0E2E9F2E102C005900D92A631CB
Now I look at it again I realise that my example needs changing quite a bit for page sizes other than 256.
optiboot_page_erase(base_addr + (offset_addr & 0xFF00));
and
optiboot_page_write(base_addr + (offset_addr & 0xFF00));
should be changed to this for 128-byte pages:
optiboot_page_erase(base_addr + (offset_addr & 0xFF80));
and:
optiboot_page_write(base_addr + (offset_addr & 0xFF80));
Well, that certainly did the trick, thank you @technoblogy! I've tested it on an ATmega8535 (64 byte page), ATmega324P (128 byte page), ATmega644P (256 byte page) and ATmega1284 (256 byte page). I can also confirm that it works excellent in high progmem too. It can easilly be tested by defining the flash space in PROGMEM1 rather than PROGMEM.
On the other side, the code now looks more cryptic than ever. But at least these functions will work on pretty much any classic AVR without the need for modification:
// Function for writing 16-bit integers to a flash page
void flash_write_int(uint32_t base_addr, uint16_t offset_addr, int16_t data)
{
// Start by erasing the flash page before we start writing to the buffer
if ((offset_addr & (SPM_PAGESIZE - 1)) == 0)
optiboot_page_erase(base_addr + (offset_addr & (0xFFFF - SPM_PAGESIZE + 1)));
// Write the 16-bit value to the buffer
optiboot_page_fill(base_addr + offset_addr, data);
// Write the buffer to flash when the buffer is full
if ((offset_addr & 0xFF) == (SPM_PAGESIZE - 2))
optiboot_page_write(base_addr + (offset_addr & (0xFFFF - SPM_PAGESIZE + 1)));
}
// Function to write bytes to a flash page
void flash_write_byte(uint32_t base_addr, uint16_t offset_addr, uint8_t data)
{
static uint8_t low_byte = 0;
if ((offset_addr & (SPM_PAGESIZE - 1)) == 0)
optiboot_page_erase(base_addr + (offset_addr & (0xFFFF - SPM_PAGESIZE + 1)));
if (!(offset_addr & 0x01)) // Address is 2, 4, 6 etc.
low_byte = data;
else // Address is 1, 3, 5 etc.
optiboot_page_fill(base_addr + offset_addr, (data << 8) | low_byte);
if ((offset_addr & 0xFF) == (SPM_PAGESIZE - 2))
optiboot_page_write(base_addr + (offset_addr & (0xFFFF - SPM_PAGESIZE + 1)));
}
// Function to force a flash page write operation
void flash_end_write(uint32_t base_addr, uint16_t offset_addr)
{
// Write the buffer to flash if there are any contents in the buffer
if ((offset_addr & 0xFF) != 0x00)
optiboot_page_write(base_addr + (offset_addr & (0xFFFF - SPM_PAGESIZE + 1)));
}
Great. Glad it works, and sorry it took me a couple of attempts to work out what the problem was!
Just one thing that people should be warned about: I'm not sure what would happen if flashSpace[] happens to overlap from near memory to far memory. Do you think that would work?
Great. Glad it works, and sorry it took me a couple of attempts to work out what the problem was!
No worries. To be honest I haven't been working on this the entire evening, we've also had some delicious easter lamb tonight that took a while to prepare.
Just one thing that people should be warned about: I'm not sure what would happen if flashSpace[] happens to overlap from near memory to far memory. Do you think that would work?
I'm not sure that could even happen as long as they work with a pre-allocated space and not just an arbitrary start address. If you stick with PROGMEM you can only allocate what's left in the near mem. If you use an AVR with 256kiB flash and you're using PROGMEM1, you can still only allocate an array with 0xFFFF places.
I modified the sketch to take a memory address instead, 0xFF00, and filled two pages from this address. This means that the first page will be in low mem, and the other one in high mem. The only "real" modification I had to do apart from commenting out the allocated flash space array and creating a constant with the same name that contains the start address, I had to remove pgm_get_far_address
becuase it would fetch the pointer to the memory address I provided, not the address itself.
Anyways, here's the sketch and output if you're interested:
Sketch:
Output:
Mmm.. the lamb sounds good!
Mmm.. the lamb sounds good!
It absolutely was!
It turns out I've discovered another bug that seems to have always been there but not yet discovered. When using the sketch I provided in this comment on an ATmega2561 (or an ATmega2560 I'm sure), I can write to any page in the first flash section (0x0000-0xFFFF) and the second section (0x10000-0x1FFFF). However, I try to write to the third section (0x20000-0x2FFFF) or fourth section (0x30000-0x3FFFF) the program crash and goes into a boot loop. I am able to upload new programs with the bootloader, but it has lost its flash writing capabilities, and I'll have to re-burn the bootloader to get to write to flash again. This one probably goes deeper than the other issues I've run into and is probably something @majekw knows more about.
EDIT: It turns out I'm able to erase a page and write to the buffer using optiboot_fill_page, it's only the write command that causes this:
// Function to force a flash page write operation
void flash_end_write(uint32_t base_addr, uint16_t offset_addr)
{
// Write the buffer to flash if there are any contents in the buffer
if ((offset_addr & 0xFF) != 0x00)
optiboot_page_write(base_addr + (offset_addr & (0xFFFF - SPM_PAGESIZE + 1)));
}
I thought you might be interested to see what I'm using the Optiboot Flasher for. It's an AVR assembler, written in a combination of C and Lisp, that runs on an ATmega1284P and allows you to write machine-code programs using assembler mnemonics and run them on the processor:
@technoblogy I'm not familiar with Lisp, but that is very impressive!
About the issue I was having regarding the 256kiB chips, it turned out that EIND has to be used:
void do_spm_cli(optiboot_addr_t address, uint8_t command, uint16_t data)
{
uint8_t sreg_save;
sreg_save = SREG; // Save old SREG value
asm volatile("cli"); // Disable interrupts
#ifdef RAMPZ
RAMPZ = (address >> 16) & 0xff; // Address bits 23-16 goes to RAMPZ
#ifdef EIND
uint8_t eind = EIND;
EIND = FLASHEND / 0x20000;
#endif
do_spm((address & 0xffff), command, data); // do_spm accepts only lower 16 bits of address
#ifdef EIND
EIND = eind;
#endif
#else
do_spm(address, command, data); // 16-bit address - no problems to pass directly
#endif
SREG = sreg_save; // Restore SREG
}
@jandrassy I believe this might be relevant for your copy_flash_pages function as well.
Glad you solved it!
I'm planning to use your Optiboot Flasher on the ATmega1284P, but there are a few things I don't understand:
I'm not sure what you're referring to by 'buffer' in the documentation in the header of SerialReadWrite.ino. When you say:
Buffer must be page aligned (see declaration of flash_buffer)
There is no flash_buffer in the code. I assume this means the block of flash you're writing to, flashSpace[]?
In:
Writing to EEPROM destroys temporary buffer You can write only once into one location of temporary buffer
By 'temporary buffer' do you mean:
uint8_t ramBuffer[SPM_PAGESIZE];
and by EEPROM do you mean flash? Why does it get destroyed by writing it to flash?
Why might you want to do fill-erase-write rather than erase-fill-write?
Instead of allocating the page(s) of flash in the middle of PROGMEM as you've done here, could I specify an explicit address for flashSpace?
For example, if I understand correctly the bootloader occupies two pages from word addresses 0xFF00 to 0xFFFF. I would like to write to one page of flash just below that, from word addresses 0xFE80 to 0xFEFF. So could I do this?
const *uint8_t flashSpace PROGMEM = 0xFE80;
Is it essential to use a RAM buffer, ramBuffer[256], or if I'm short of RAM could I write the data to flash byte by byte, treating it more like an SD card? I realise I would have to provide my own alternative to optiboot_writePage().
Finally, I'm puzzled by the code in optiboot_readPage() in optiboot.h:
if(read_character != 0 && read_character != 255)
storage_array[j] = read_character;
Why is it skipping bytes if they are 0 or 255?
Sorry about the flood of questions!