Expected behavior of dhara_map_trim

Dermiste commented 1 week ago

Hi,

First off, great project, I have been using a top layer build on it for a while and everything is working fine.

Couple of days ago I tried using dhara_map_trim and I am a bit confused about the results, scenario described bellow, can you please shed some light on this ? is what I see the expected results or is there something wrong with my code?

Page size is 2048

1 - Initial state:

4 records

Page ID	Content
0	`0x00, 0x00, 0x00 ...`
1	`0x01, 0x01, 0x01 ...`
2	`0x02, 0x02, 0x02 ...`
3	`0x03, 0x03, 0x03 ...`

2 - Action

dhara_map_trim( _ , 2)

Final state:

Expected results

3 records, the ligne with only 2's was removed and garbaged collected

Page ID	Content
0	`0x00, 0x00, 0x00 ...`
1	`0x01, 0x01, 0x01 ...`
2	`0x03, 0x03, 0x03 ...`

Actual results:

3 records, but when I try to read the one at index 2 I get 0xff's (= DHARA_E_NOT_FOUND) instead of 0x03's

Page ID	Content
0	`0x00, 0x00, 0x00 ...`
1	`0x01, 0x01, 0x01 ...`
2	`0xff, 0xff, 0xff ...`
3	`0x03, 0x03, 0x03 ...`

Yaxit commented 2 days ago

AFAIK trim will remove the sector from the journal, which marks it as free. So what you see is expected. Sector 2 is now free.

Dermiste commented 2 days ago

Thanks @Yaxit for your help :) in this case how should I proceed to "recycle" free sectors? I am trying to get to the "expected result". I assume what I am trying to do is a common case, it could be summed up like:

Fill up memory with 1000 records
Remove 1 record every two, so only remains records at 0,2,4,6,8
"Recycle" empty sectors
We now have used sector from 0 to 500, all empty ones moved from 500 to 1000

Yaxit commented 2 days ago

What do you mean with recycling?

Dermiste commented 2 days ago

Well if a sector is removed from the journal, I would like the next sector to move up one page, and be located where the removed sector was

If I have sector ID's 0,1,2 and they points to data A,B,C, if I delete sector 1 I need to find a way so that deleted sector 1 now points to C instead of B.

Maybe "trim" is not what I need ?

I hope that makes sense.

Yaxit commented 1 day ago

I understand what you want, although not why. Dhara is a FTL that provides an abstraction over the physical pages of the memory. It is your responsibility to track which sectors contain which data. What you want seems more in the scope of a file system in my opinion. You can use a file system on top of Dhara of course, like FATfs.

Hope this helps!

Dermiste commented 1 day ago

For my project, I need my flash to contain contiguous records of data, of various size. Once the flash is full old records will be recycled to make room for new ones. Each new one will eventually occupy the space used by 1 or more records. Since records vary in size, adjustment will have to be made in order to keep records contiguous, hence my initial questions.

You said this lib is an abstraction layer, which means I can copy pages to trimmed spaces in order to achieve what I need & the physical arrangement of pages should not change, only the journal keeping track of sectors, am I right?

Yaxit commented 1 day ago

What you want to achieve is what a filesystem would do. Keeps track of the free and used sectors, and offers you a file-based API that is agnostic with respect to which sectors are used for each file.

You said this lib is an abstraction layer, which means I can copy pages to trimmed spaces in order to achieve what I need & the physical arrangement of pages should not change, only the journal keeping track of sectors, am I right?

No, copying pages will actually copy data in and out. In addition, it will also need to update the journal. Again, your problem is solved at the FS level, not in Dhara. You cannot achieve what you want with this library alone, sorry.

dlbeer / dhara