Add menu options for EEPROM size selection

[dbuezas]

What do you think about this @LaZsolt ?

Relates to #243

[dbuezas]

Board setup for 0 EEPROM if the lib is not included

if EEPROM.h is not included, the eeprom emulation registers are never initialised and the board is therefore (I think) setup to emulate zero EEPROM

int lgt_eeprom_size(){
  if ( ECCR & 0x40 ) return  ( 1020 << ( ECCR & 0x3 ) );
  return 0;
}

void setup() {
  Serial.begin(115200);
  Serial.print("lgt_eeprom_size:");
  Serial.println(lgt_eeprom_size());
  delay(1000);
}
void loop() { }

• Prints: lgt_eeprom_size: 0
• When setting up EEPROM Emulation to 0 KB, compilation shows: Sketch uses 1940 bytes (6%) of program storage space. Maximum is 31744 bytes. (I came up with this value by adding 1024*2 to the old upload.maximum_size)

If the lib is imported

#define USE_LGT_EEPROM_API
#include <EEPROM.h>

void setup() {
  Serial.begin(115200);
  lgt_eeprom_init(EEPROM_SIZE); // this is the constant set through the menu
  Serial.print("lgt_eeprom_size: ");
  Serial.print(lgt_eeprom_size());
  Serial.println();
}

void loop() { }

Prints lgt_eeprom_size: 8160 (or whatever is set in the menu )

[dbuezas]

@dwillmore @jg1uaa @jayzakk, your feedback (and that of anybody else) very welcome :)

Also, I need some help writing a test that:

• fills the program part of the flash
• fills EEPROM
• tests if everything is ok and the EEPROM reads correctly and doesn't overwrite the program

Any willing volunteer? :)

[dbuezas]

Uhm... I just read in our wiki that the bootloader is stored right after the 2 kb reserved for the EEPROM, this whole thing may be cursed...

I don't find any explicit offset set in platforms.txt though

[LaZsolt]

the bootloader is stored right after the 2 kb reserved for the EEPROM

Not true. The bootloader is stored before the 2kB reserved EEPROM space. So

making the EEPROM space configurable via menu would add another 2k

not true.

The factory burned bootloader started at address 0x7400 and the emulated EEPROM start address is 0x7800 which ending at the end of the FLASH.

Ergo for the different emulated EEPROM size require different bootloader.

All in all if the EEPROM size menu exist it is only could reseve ROM space for such a type of EEPROM emulation software which not using the hardware emulation. In this case the bootloader must start at address 0x7C00 and e.g. 1kB emulater EEPROM start at 0x7400

[LaZsolt]

Edit:

the bootloader is stored right after the 2 kb reserved for the EEPROM

Not true. The bootloader is stored before the 2kB reserved EEPROM space.

[LaZsolt]

Somehow I unmarked the draft attribute this pull request. But I don't know how.

[dbuezas]

Somehow I unmarked the draft attribute this pull request. But I don't know how.

it's on the top right. Marked as draft again

[dbuezas]

Ok, I'm thinking about closing the PR then.

What if one is using an ISP? There's no bootloader in that case so maybe adding a new board type for it would allow using all the space

[SuperUserNameMan]

BTW, I did not follow the impact of the recent updates, but if one of you change the address of the bootloader according to the EEPROM size picked in the menu, please update the https://github.com/dbuezas/lgt8fx/wiki/EEPROM-of-the-LGT8F328p-:-the-important-things-you-need-to-know

[SuperUserNameMan]

Uhm... I just read in our wiki that the bootloader is stored right after the 2 kb reserved for the EEPROM, this whole thing may be cursed...

As mentioned by @LaZsolt , when i wrote the wiki, the bootloader was actually stored BEFORE the EEPROM :

• bootloader from 0x7400 to 0x77ff (1kB)
• EEPROM from 0x7800 to 0x7fFF (2kB)

Extending the EEPROM at runtime would overwrite the bootloader, and might also overwrite a part of the program space.

Setting the EEPROM to 0 by default, would leave 2kB of unused blank space AFTER the bootloader, unless you relocate the bootloader.

---

Here is something I did not think about when i rewrote the EEPROM library : maybe it is possible to swap the bootloader and the EEPROM.

It is possible to modify the EEPROM library to protect the 2 last 1kB pages in which would be relocated the bootloader. In consequence, we would waste 1kB (since the bootloader is 1kB long), but the size of the EEPROM could be extended without overwriting the bootloader. Also, setting the EEPROM to 0 by default would leave 3kB of extra space for the program.

edit : and when the EEPROM is enabled, the unused 1kB page just before (or just after) the bootloader could be used as a emulated-fuse space. (as it is possible to write once into each blank cells without triggering destructive page swap)

[dbuezas]

@SuperUserNameMan you are back! :) I sent you Collaborator rights. This allows you to merge PRs and allows ppl in comments to link you even if you are not part of the thread yet.

I did not follow the impact of the recent updates

Nothing related to the bootloader or eeprom except this PR.

maybe it is possible to swap the bootloader and the EEPROM.

It probably is, but this will make this core incompatible with the default bootloader these boards typically ship with. Most likely a bad idea.

Extending the EEPROM at runtime would overwrite the bootloader

I don't understand this: the bootloader is in lower ram than the emulated EEPROM space. Does the EEPROM grow backwards?

---

I think the only option available would be to make an option to not reserve EEPROM space specifically to be used without the bootloader (i.e while burning via ISP). WDYT?

[SuperUserNameMan]

@SuperUserNameMan you are back! :) I sent you Collaborator rights. This allows you to merge PRs and allows ppl in comments to link you even if you are not part of the thread yet.

My availability might be unreliable for major contribution, though i might still be of use regarding EEPROM questions as long as my memory is not outdated.

maybe it is possible to swap the bootloader and the EEPROM.

It probably is, but this will make this core incompatible with the default bootloader these boards typically ship with. Most likely a bad idea.

Do we have access to the bootloader source-code ? Isn't the bootloader burned at each sketch upload ? (edit : indeed these two questions show how rusty i became ...)

Extending the EEPROM at runtime would overwrite the bootloader

I don't understand this: the bootloader is in lower ram than the emulated EEPROM space. Does the EEPROM grow backwards?

Yes, it is extended backwards so that everything remains within the 32kB memory space.

I think the only option available would be to make an option to not reserve EEPROM space specifically to be used without the bootloader (i.e while burning via ISP). WDYT?

Yes, i think you could set the size of the EEPROM to 0 by default, so when the sketch is uploaded using an ISP, the whole 32KB of flash is available for the sketch.

And when uploading a sketch using the bootloader, setting the size of the EEPROM to 0 by default should have no impact other than wasting 2kB of space at the end of the flash. (however, you might want to check if it does not interfere with the initialization code of the EEPROM library)

[SuperUserNameMan]

I don't understand this: the bootloader is in lower ram than the emulated EEPROM space. Does the EEPROM grow backwards?

Do you mean the bootloader is copied into the RAM ?

[LaZsolt]

The bootloader runs from flash started at address 0x7400. From these address the code can be read with the pgm_read_byte().

But the bootloader code also readabe from RAM address started at 0xB400.

[LaZsolt]

Crazy idea, but if the EEPROM emulated by software it could be jump up the bootloader code.

Another crazy idea, for that case when the EEPROM size is 0, is that define a fixed adressable read only 1k data block with starting address is the same as the bootloader then the compiler could avoid to put any code here, but the last 2k became useable.

[dbuezas]

Very interesting! I don't see any offsets in the upload scripts, so it must be the bootloader the one that adds the offset. If I'm correct, it means that if one uses EEPROM in a sketch uploaded via ISP, the source code would ve overwritten. Right?

[SuperUserNameMan]

If I'm correct, it means that if one uses EEPROM in a sketch uploaded via ISP, the source code would ve overwritten. Right?

Unless the compiler is aware of the maximum available memory, yes.

[SuperUserNameMan]

Crazy idea, but if the EEPROM emulated by software it could be jump up the bootloader code.

Yes, i was thinking about the same. I think it is possible to tell to the EEPROM library to avoid the two 1kB pages encompassing the bootloader. (as they work in pair, we must skip 2kB (edit : the EEPROM libray should just skip the before last 1KB page, and the hardware EEPROM module will convert it to a 2KB skip. See next messages bellows )

We then would have EEPROM with theoretical sizes of 1KB, 3KB or 7KB.

number of 1KB pages of the EEPROM	theoretical size of the EEPROM	actual available bytes and emulated continuous space	occupied space into main Flash memory	beginning address into Flash memory
0	0	0	0	n/a
1	1024	1020	2048	0x7800
2	n/a	n/a	n/a	0x7000
3	3072	3060	8192	0x6000
7	7168	7140	16384	0x4000
default	1024	1020	2048	0x7800

Instead of :

number of 1KB pages of the EEPROM	theoretical size of the EEPROM	actual available bytes and emulated continuous space	occupied space into main Flash memory	beginning address into Flash memory
0	0	0	0	n/a
1	1024	1020	2048	0x7800
2	2048	2040	4096	0x7000
4	4096	4080	8192	0x6000
8	8192	8160	16384	0x4000
default	1024	1020	2048	0x7800

Another crazy idea, for that case when the EEPROM size is 0, is that define a fixed adressable read only 1k data block with starting address is the same as the bootloader then the compiler could avoid to put any code here, but the last 2k became useable.

If it is possible to tell the compiler to skip the bootloader, that's a good idea !

[SuperUserNameMan]

Oh wait, I think i need to refresh my memory about how the EEPROM is emulated before misleading you on that topic.

[SuperUserNameMan]

Well if you want to skip the bootloader, you just have to hack : uint16_t lgt_eeprom_continuous_address_to_real_address( uint16_t address )

This function is used to recalculate the EEPROM address so that the library automatically skips the last 4 bytes of each 1KB pages of EEPROM. It is thus possible to tell it to skip the before last 1KB page of the EEPROM.

Note that the EEPROM library is not aware of the hardware page-swap algorithm which consume 2 Flash pages per 1KB EEPROM page.

Also note that the EEPROM library does NOT think in term of the Flash memory space : it does not care about the actual address into the Flash memory, because it is handled by the hardware EEPROM module which will convert the EEPROM space addresses into Flash space addresses.

So when the EEPROM library will skip the before last 1KB page of its EEPROM space, the hardware module will actually skip 2KB (2 pages) of the Flash memory space (from 0x7000 to 0x7fFF ). Which should preserve the bootloader (and leave 1KB of unused flash memory)

[wollewald]

Hi, I have seen this is closed. I have read it, but still I do not understand if there is a way to get completely rid of the EEPROM in order to use the full 32 KB flash for program code.

[LaZsolt]

I think if the program uploaded using an ISP then all the 32 KB flash will be available for program code.

[wollewald]

Thanks for the reply.

I tried uploading via LArduinoISP, it does not seem to change anything:

I also tried to allocate 0 KB for the EEPROM using:

ECCR = 0x80;
ECCR = 0;

It had no effect. Same with:

lgt_eeprom_init(0);

[LaZsolt]

The displayed size of uploadable code size comes from here: https://github.com/dbuezas/lgt8fx/blob/df986b57d2424452255e5ec975149948ae1ae7de/lgt8f/boards.txt#L19

It can be exceeded but we get a warning.

[wollewald]

I am used from the AVR Arduinos that this max. value is flexible. E.g. if you use the Optiboot bootloader the value is smaller compared to what you get when you don't use it. If it's just a fixed value which uses the default setting of 1 KB EEPROM this explains my confusion. Displaying the real max available flash would be helpful. Many thanks!!!!

[dbuezas]

So displaying the full 32KB? We can do that if the community agrees

[wollewald]

The main reason for my question is that some people asked if they could use the Arduino Optiboot for the LGT8F328P, because they think it's smaller than the bootloader for the LGT8F328P. Like me, they just at value you get during upload.

[LaZsolt]

So displaying the full 32KB? We can do that if the community agrees

I think displaying the real available flash will not be easy because uploadig with ISP cannot be detected. I will study the Arduino core solution.

An Optiboot support wish for LGT8F328P: https://github.com/Optiboot/optiboot/issues/359 The current LGT8Fx's bootloader version is made from a very old Optiboot version. I think if someone could make the short version of the Optiboot for LGT8Fx, then it could be bigger than the ATmega version because of different flash programmig method.

[wollewald]

Just to be sure that I have understood it correctly: it's just the bootloader that takes ~ 3KB and this does not include the 2 KB which are needed for the 1 KB EEPROM?

And thanks for the link!

[LaZsolt]

The LGT8F328P bootloader less than 1 kB.

[wollewald]

< 1KB - that's great.

If it's difficult to implement a method to display the exact flash which is available, then some documention how the 29696 bytes are calculated and under which conditions this value applies would be helpful. I am by far not so deep in this matter like you, but still I would say I am not an absolute beginner. I simply thought I did something wrong when I changed the size of the EEPROM and the max available flash size didn't change. Therefore, there are probably quite a number of people who have similar problems. I also have not yet finally understood if there is flash reserved for the EEPROM by default, even if I do not include EEPROM.h. From what I read now I would assume that there are roughly 31 KB Flash available if I don't use the EEPROM and 29696 bytes if I use 1 KB of EEPROM. Sorry, to bother you with all these questions, but would really like to understand this.

[dbuezas]

From the wiki article about EEPROM written by @SuperUserNameMan, the flash layout is:

[0, Program..., ...bootloader..., ...EEPROM, 32000]

Trying to use more than 1k of eeprom (taking 2kb of flash) will overwrite the bootloader, because the eeprom "grows backwards".

This means that the only way to have bigger programs is to forego the bootloader (upload via isp).

[wollewald]

Hi, this is helpful. Sorry to be so tenacious, but there is still one thing I am not sure of:

"By default, the size of this emulated EEPROM is set to 1KB, and it can be changed at run-time to 0KB, 1KB,2KB, 4K or 8KB."

Does the default (1KB EEPROM) only apply if I include EEPROM.h? In other words: If I use the bootloader and I don't include EEPROM.h, will no flash be reserved for the EEPROM and therefore I should have ~31KB flash?

[LaZsolt]

Displaying the real max available flash would be helpful.

I checked the AVR Arduino core.

• It looks like no possibile way to displaying different available flash size when uploading with ISP or with bootloader.
• Normally you don't want to overwrite the bootloader when using ISP for upload so the available flash size is the same as in normal case. 29696 bytes.
  (Using ISP for upload there is possibilities to overwrite the bootloader and the emulated EEPROM space. In this case the full flash size available for program, but you get a warning message.)
• The emulated EEPROM space is always reserved regardless of whether it is used or not. In other words the bootloader won't jump to the end of flash when 0 KB EEPROM is set in runtime.
  (The reserved emulated EEPROM space is at the end of program flash memory. The bootloader precedes the reserved emulated EEPROM space.)

So in my opinion no need to change the displayed maximum upload size.

In summary: The bootloader stored in a fixed location in program flash memory followed by emulated EEPROM space. If EEPROM space changed bigger than 1 KB the bootloader will be overwritten after writing some bytes to the EEPROM spaces. If someone want to use bigger (or smaller) EEPROM space the bootloader must be relocated. This relocation method is not yet invented.

[wollewald]

Very clear - that answers my question perfectly. Thank you!

The other good news: no more questions from side! Thanks for your patience.

[dbuezas]

So to use the full 32kb:

• Make a program that uses 32Kb
• Upload via ISP
• Ignore warnings
• Don't attempt to write anything to EEPROM

Right?

[LaZsolt]

Yes.

[dwillmore]

And that goes for ATMEL and LGT?

On Sun, Jul 2, 2023 at 11:37 AM LaZsolt @.***> wrote:

Yes.

— Reply to this email directly, view it on GitHub https://github.com/dbuezas/lgt8fx/pull/261#issuecomment-1616704878, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACPEX7FOOJZGR2AFE3XOOTDXOGITDANCNFSM6AAAAAAWNZA6DY . You are receiving this because you were mentioned.Message ID: @.***>

[dbuezas]

No, ATMEL has real EEPROM independent of flash