Support links for reading (eventually also writing?) files (and directories)

[t-w]

I understand from your explanation that the implementation of the links in Amiga's OS is better to avoid... However, I was thinking about adding the support at least in some part (like reading linked files).

I investigated this a bit, it seems that reading hardlinked files should not be difficult. I think, It is a matter of (optional) opening the block of with file entry in place of the hard link in the function adfOpenFile(), then the API would work as with regular file (and, if I understand well, in Amiga filesystems, a hardlink is just a pointer to real file/directory, so there is no metadata to update in it - but please correct me if I am wrong).

Would this be OK. to add something like this? (with some test I guess) If so, i prepare a PR.

Btw. For now, I am not sure about directories and the softlinks (maybe I resolve this on my side...) - but just let know if you'd eventually be willing to merge such changes as well (that is, if I figure out anything useful...).

[lclevy]

Yes, it is ok. But I won't be able to help, as this code is 20 years old to me.

[lclevy]

Does it work with a chain of links?

[t-w]

I was using testffs.adf from the set of test disks in the repo, there is no such case there (I think...), so it is rather hard to implement not having an example... As for now, it does work with 1 level, just goes to what is pointed by realEntry in the hard link data structure and expects to be at the real entry (as the name...).

So far, I haven't seen a case where realEntry points to another link... If there are such cases - this basically would have to be extended to traverse to the "real" real entry (probably just wrap it in a loop) - so it is a good start, anyway.

If you have an example of such chain of links for test, I can have a further look at that.

[t-w]

Actually, it seems to me that chain of links should not matter - if I understand well (looking at this description), such chain is built with the next_link entry in the hard link structure, while real_entry should point to a real entry (file or directory). If it is so - the chain should not matter.

(I am no expert on Amiga filesystems though, so correct me if I am wrong...)

[lclevy]

The best would be to test on real system, emulated or not

[t-w]

Well, I do not have any disks with such "chained" links... do you have one?

To make it clear about a "real" system where I use it - I build it on a Linux box and, besides the tests written for ADFlib, I do all this to use it for fuseadf, a filesystem which allows to mount an ADF image into a linux directory and read (maybe later write) the files without extracting them from the image. This is basically why I need some way to get into the structure of ADF disks, and ADFlib seems suitable for it. So another test that I do is mounting a few different images (including testffs,adf from your set) to see how it works. For now, it works properly with files and directories (for reading). Including changes from #25, it works also for hard links (both files and directories - from testffs.adf, as it is the only example I got...). So as for me, after these changes the library works properly.

Btw. I just got to the softlinks (which are the last which I am having trouble with). I use adfNameToEntryBlk() (from adf_dir.c) to get the block entry (with softlink) as it can be done by name - but the block gets corrupted because adfReadEntryBlock() always is doing byte swapping (endian change, adf_raw.c/swapEndian()) for SWBL_ENTRY (while it should read it as link block, doing byte swapping with SWBL_LINK). I have made a simple change:

RETCODE adfReadEntryBlock(struct Volume* vol, SECTNUM nSect, struct bEntryBlock *ent)
{
    uint8_t buf[512];

    if (adfReadBlock(vol, nSect, buf)!=RC_OK)
        return RC_ERROR;

    memcpy(ent, buf, 512);
#ifdef LITT_ENDIAN
    int32_t secType = swapLong ( ( uint8_t * ) &ent->secType );
    if ( secType == ST_LFILE ||
         secType == ST_LDIR ||
         secType == ST_LSOFT  )
    {
        swapEndian((uint8_t*)ent, SWBL_LINK);
    } else { 
        swapEndian((uint8_t*)ent, SWBL_ENTRY);
    }
#endif

which fixes the problem for links.

But the question here is how the API is supposed to work. For me, function like adfNameToEntryBlk() should work for any directory entry type (which can be a file header block, link block, or a directory block) as you just give it a name of the entry and the function give back a correct block (which then can be just casted to whatever entry type this is). But maybe I am doing it in a wrong way and this change will break something for other cases??? So what is exactly the idea behind this? Can this be extended to support different entry types or should it remain specialized? If so - how should properly I retrieve the block of a link (by its name)?

[lclevy]

you can find amiga OS here : http://lclevy.free.fr/drd/amigaroms.zip A cshell here: https://github.com/lclevy/ADFlib/blob/master/arccsh.adf and an emulator : https://www.winuae.net/

related work : https://lallafa.de/blog/2012/01/mastering-adfhdf-images-with-xdftool/

[t-w]

Thanks for CSH, I was using Zshell but this is more friendly (anyway, links can be also created with makelink from Amiga utilities). I had a brief look, and made some image doing hardlink to hard link (to hard link) to something (file or dir.) - it is always shown by ls command of csh as a hardlink to the real file/directory (not the other hardlink), so the realEntry field in the link block structure points to the block of the real file/directory. It means the code will work for such case - and it does when I mount the image with fuseadf (using updated ADFlib). What interesting - the same hardlinks does not work when mounted with affs module from Linux kernel (possibly because there are some security concerns, so they cannot risk...). Anyway, OK., I will extend also the test code to cover this.

[lclevy]

Fuseadf is using adflib? Where is it hosted?

[t-w]

Well, It is a prototype that I am working on, so far in a private repo on GitLab, until it is in some decent state for basic use. It might be soon as in general reading files and directories is fine. As for the links, with the changes to the ADFlib, they seem to work too (as least for these cases I tried). If you want to have a look, I can grant you permissions (but you need an account on gitlab.com).

[t-w]

I have created a new test image regtests/Dumps/test_link_chains.adf with hard linked ("chained") files and directories (using ln tool from CSH, I think the original makelink would do exactly the same...) and added some simple tests:

• for files, it opens a file (adfOpenFile()) using a hardlink (or a hardlink to a hardlink .... to a file - they always point to the same real file, as stated before) and reads it - work as with a regular file

• for directories - adfChangeDir() to hard-linked directories works in the same way as with a regular directory (after the operation, vol->curDirPtr just points to the original directory).

(see the new commits in the PR)

[lclevy]

Excellent !

[t-w]

There is one more thing about links - I would like to to add some minimal support for softlinks - just proper resolving the destination. Currently this is broken, at least on Intel CPUs: directory entry blocks are always read and treated as a directory entries (struct bEntryBlock) while sometimes they are links. The problem is that adfReadEntryBlock() is using incorrect mapping (SWBL_ENTRY) for modifying byte order with swapEndian() (for links it should use SWBL_LINK, see adf_raw.c). In result, for instance dir_1 becomes _rid...1 (there can be other broken fields as well...).

I have pushed the simple change for adfReadEntryBlock() (with a simple test). It is just detecting type of entry being read and swapping bytes for links with proper mapping.

[lclevy]

Hi, may I close this issue?

[t-w]

Yes, I guess we are done here. I haven't used or tested writing linked files - but this should be avoided anyway. The relevant part is reading them, and that seems fine to me. (I close the issue).

[lclevy]

Thank you! I'm currently converting the faq and readme to markdown...

[lclevy]

Can I reach you via email ou Twitter ?