(in)validity of block allocation bitmap

[t-w]

Reviewing some pieces of bitmap allocation code (for #63) revealed a couple of potential issues:

1. (in)validity flag (of the bitmap) is not checked while mounting a volume

   • it rather should be checked and if the bitmap is not valid - it should be rebuilt/reconstructed from the data on the volume (so it should basically work like Amiga's DiskValidator); otherwise, if an invalid bitmap is just read (as it is now), eg. an already used block (but not marked so in bitmap) can be (again) allocated for another purpose; all in all it can lead to hard to detect data corruption

2. the use of the (in)validity flag is limited to only adfUpdateBitmap (one function), while it seems is should be used to mark "dirty" bitmap while writing any block (data, directory, link) until the bitmap is updated accordingly, as the block can be (de)allocated by those operations (meaning the bitmap will be invalid if the operation gets interrupted for some reason)

(All this have to be checked further, this is just a brief note for now).

[t-w]

ad. 2. In fact, the current code seems to be fine. In adfSetBlockUsed/Free(), it marks the status of each allocated/deallocated block to "changed". Then, adfUpdateBitmap(), before updating the bitmap blocks, sets BM_INVALID flag in root block, updates the bitmap writing bitmap blocks that changed, and then, if all is successful, sets BM_VALID in root block.

BUT! If there was some change written to the disk, which changed the block allocation, and root block for some reason is not read (there is an error) in adfUpdateBitmap(), after which the function aborts(!), there is a situation where there are changes in the blocks allocation done, but neither block allocation bitmap is updated nor the rootblock's bitmap validity flag set to invalid. What, further, can lead to data corruption due to use of not up-to-date bitmap.

To minimize this possibility, the invalid flag should be set (and written to the disk!) immediately after any change is done in regards of block allocation.

However, the current code does not keep the rootblock contests in memory, it would be necessary to read it (and, if necessary, write) each time block allocation/deallocation occurs.

So either additional flag has to be added to the internal bitmap structure (to keep this information), or the rootblock should be kept in the memory all the time (and written when necessary).

[t-w]

ad. 2 (cont.) In general, there seem to be 3 ways of minimizing the possibility of occurring the case when the bitmap on the disk is invalid and the bitmap invalidity flag in root block is not set as invalid:

1. read rootblock / set the bitmap flag "invalid" / write rootblock - immediately when there is any change of the bitmap is going to occur (before making the change)
2. completely ignore the bitmap (in)validity flag when mounting the volume, always reconstruct the bitmap, compare it with the one on the disk, and, if different, update it (and set the bitmap (in)validity flag).
3. completely ignore bitmap allocation blocks: always build the bitmap from the block data when mounting (like when the bitmap is always invalid) and keep it up-to-date in memory; set bitmap (in)validity flag always as invalid (so that the orig. Amiga OS reconstructs it by itself)

Not sure which way is better... so comments welcomed.

[t-w]

ad. 2 (cont.) Some pros and cons:

1. cons: potentially, a lot of reads/writes to rootblock (it might overly wear out some media, like real floppies, flash drives etc.)
2. and 3. cons: 2. and 3. might be problematic in case of very slow media (real floppies?) and/or large filesystems (larger hard disks or hard disk images) - acquiring allocation data by traversing all metadata blocks (so all directory blocks, dir. entry blocks and file ext. blocks) can make mounting slow (not sure how slow - whether it would be acceptable or not...).

At the moment, I am leaning towards 3. - so just ignore bitmap blocks and its data completely (do not read or write it) and just make sure that bitmap allocation data is set as invalid in root block and let (eventually) a real Amiga OS rebuild it (or, eventually, update the bitmap only when the volume is being unmounted, and then mark it as "valid").

Kind of a related matter - I haven't found any way that an OFS or FFS volume could have bad blocks marked (except maybe a special file named eg. BADBLOCKS, which would have all the bad blocks assigned... some utilities seem to do that: https://forum.amiga.org/index.php?topic=1068.0 https://comp.sys.amiga.hardware.narkive.com/S0P0ALdP/mark-bad-block-in-hdd ) But in case something like this existed - to consider along with the things discussed here...

[t-w]

For the moment, the code works as follows (after the merge above):

• by default it always reads the bitmap from volume (when mounting)
• the validity of the bitmap must be checked in the user code (in principal, if invalid, it should be rebuild to prevent volume corruption and data loss)
• if the bitmap is marked invalid - the lib gives only a warning on mounting
• bitmap reconstruction can be done with adfReconstructBitmap() and is done in-memory, it is not written until other write operations (so as before) or an explicit call to adfUpdateBitmap()

Eventual further improvement would be to somehow split volume mount with or without filesystem. In the first case, the bitmap (as any other volume metadata) would not be read or used (to support a custom data layout, just on the volume blocks level).