lclevy / ADFlib

A free, portable and open implementation of the Amiga filesystem
GNU General Public License v2.0
89 stars 30 forks source link

Issue with adding files to adf (floppy) image #3

Closed emoon closed 9 years ago

emoon commented 9 years ago

Hi,

I'm trying to use this lib to add a (executable) file to a floppy. The floppy is a standard DD with a standard Bootblock with S directory, Startup-Sequence file which has "test_exe" in it.

I use this code (roughly) to add the file:

int main(int argc, const char* argv[])
{
    struct Device* flop = 0;
    struct Volume* vol = 0;
    struct File* file = 0;
    size_t fileSize = 0;
    void* data = 0;

    const char* output = argv[1];
    const char* input = argv[2];

    if (!(data = loadToMemory(input, &fileSize)))
    {
        printf("Unable to load file \"%s\" to memory\n", input);
        return -1;
    }

    if (!(flop = adfMountDev((char*)output, 0)))
    {
        printf("%s : unable create newdev: %s\n", argv[0], output);
        return 1;
    }

    if (!(vol = adfMount(flop, 0, 0)))
    {
        printf("%s : unable to mount floppy\n", argv[0]);
        return 1;
    }

    if (!(file = adfOpenFile(vol, "test_exe", "w")))
    {
        printf("%s : unable to create test_exe\n", argv[0]);
        return 1;
    }

    printf("fileSize %d\n", (int)fileSize);

    if (adfWriteFile(file, fileSize, (uint8_t*)data) != fileSize)
    {
        printf("%s : unable to write test_exe\n", argv[0]);
        return 1;
    }

    adfFlushFile(file);
    adfUnMount(vol);
    adfUnMountDev(flop);

    return 0;
}

But when I try to boot it it simple will not boot the image. If run the executable manually it works, also if do install df0: it will work as expected. (Notice that I have already installed the floppy before adding the executable to it)

If I take a copy of the floppy and run install df0 on it again it works as expected. If I do a binary diff of the two images there are some places where they differ which is likely the cause of why the Amiga can't boot the image.

I compile this under MacOSX 64-bit (I have also made sure that LITT_ENDIAN is set to 1)

Any ideas of the cause of this?

Cheers!

lclevy commented 9 years ago

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

Hi,

it looks like it is a system problem: there is at least at one missing condition for booting, and not a filesystem problem (file integrity). you can hash the file (md5) before writing and after re-reading it using adflib and compare the 2 hashes. why are not closing the file ? I know flushing would be enough.

Laurent

Le 01/06/2015 13:29, Daniel Collin a écrit :

Hi,

I'm trying to use this lib to add a (executable) file to a floppy. The floppy is a standard DD with a standard Bootblock with S directory, Startup-Sequence file which has "test_exe" in it.

I use this code (roughly) to add the file:


0; struct Volume\* vol = 0; struct File\* file = 0; size_t fileSize =
0; void\* data = 0;

const char\* output = argv[1]; const char\* input = argv[2];

if (!(data = loadToMemory(input, &fileSize))) { printf("Unable to
load file \"%s\" to memory\n", input); return -1; }

if (!(flop = adfMountDev((char*)output, 0))) { printf("%s : unable
create newdev: %s\n", argv[0], output); return 1; }

if (!(vol = adfMount(flop, 0, 0))) { printf("%s : unable to mount
floppy\n", argv[0]); return 1; }

if (!(file = adfOpenFile(vol, "test_exe", "w"))) { printf("%s :
unable to create test_exe\n", argv[0]); return 1; }

printf("fileSize %d\n", (int)fileSize);

if (adfWriteFile(file, fileSize, (uint8_t*)data) != fileSize) { 
printf("%s : unable to write test_exe\n", argv[0]); return 1; }

adfFlushFile(file); adfUnMount(vol); adfUnMountDev(flop);

return 0; } ```

But when I try to boot it it simple will not boot the image. If run
the executable manually it works, also if do install df0: it will
work as expected. (Notice that I have already installed the floppy
before adding the executable to it)

If I take a copy of the floppy and run install df0 on it again it
works as expected. If I do a binary diff of the two images there
are some places where they differ which is likely the cause of why
the Amiga can't boot the image.

I compile this under MacOSX 64-bit (I have also made sure that
LITT_ENDIAN is set to 1)

Any ideas of the cause of this?

Cheers!

--- Reply to this email directly or view it on GitHub: 
https://github.com/lclevy/ADFlib/issues/3

-----BEGIN PGP SIGNATURE----- Version: GnuPG v2

iQIcBAEBAgAGBQJVbLh2AAoJENhRYAtXK3xN5D0QAKHd632IqY4Wd7WqyZixLCzO noWOsEvTucptEeOkfbdbPtegsiG6EAQxBxNns1CRmffP2FlcYWYFbSi+lgKB4wP6 /yiSkO60v+Fvif67xkuypWhvtRGzL0owYRuoiLTdaOmvqFZCM7CSU4lvo8d/4kDz WZtLlr/dzcjG1+0drDElgEQfrDIqpqNN+Ksl+5WPj8q1SR24JBBvWjhABXn+V/b/ 4XzCLiSRp8banap0AvapOZDbipjdpwBXp+MkpW8F4c7Wx0YZRxNphMc5GxXpTJS2 Xs9kn80uGjae0nkbgg2ExZRapSghL7t8snIyDOnQAio4ysw3kPxvDsj4wBRC9IDz d2w2Nkcm+/BQF4brildshQ7qr8oiJD7GL5fgCIG/Nco7/uy6GNhUX0Shb35OWbHJ +wu994fwp8Hg+VKT57qK5Qnex2v2a8du4CGYw1F+mZkLoy2NH6V97Fkz6ShTbfyk QxA4WwXRtuQrVNxtgjlJbIGYSYJ0bQpdnU0TULTjK0uwrsL9abYBwx/Dnz/IM7Q+ ygA9jYw8YJBG6IzayRikoZ/+g+isdrDc9br4cAaqIKo4QdwsEqKsZrQwK3Skju/z OlH0U+qKQVCO8Of4Bx6HKRp6aZXUMRuPDYX5jXoPX2p1UiCQ6iD9bwpYdNhiajt6 349VhpS7o6XlKDRdkAXX =Nrpz -----END PGP SIGNATURE-----

emoon commented 9 years ago

Hi,

Let me describe more in detail what I do.

  1. I have a floppy that boots fine. it has a startup-sequence that tries to boot load "test_exe" and it boots saying it can't find the exe (expected as I haven't written it)
  2. I open the adf (as of the code above) write the file to it and exit the program.
  3. I start the Amiga with the floppy that has the added file to it. It will not boot.
  4. Running the file manually works correct (df0:test_exe) so the data of the file is correct.

Now if i run "install df0:", reboot the machine the floppy will boot correct.

If I run a binary diff of the adf which I have written the file to and copy which I have done the same thing but also did "install df0" on. I see some spots like this:

diff_screen

So in some various spots of the filesystem some data is different. I'm not sure how exactly the file system is laid out but this data doesn't seem to be directly related to the data of the file but more of parts of the file system.

I'm trying to boot the floppy on a Amiga 500 (KS 1.3)

Cheers!

emoon commented 9 years ago

Bump,

Just wondering if you might have any idea on whats going on?

Cheers!

lclevy commented 9 years ago

Hi, on certain conditions when disk is almost full, yes this might happen. Any error message or details?

Le 25 juin 2015 11:53:33 GMT+02:00, Daniel Collin notifications@github.com a écrit :

Bump,

Just wondering if you might have any idea on whats going on?

Cheers!


Reply to this email directly or view it on GitHub: https://github.com/lclevy/ADFlib/issues/3#issuecomment-115194647

Envoyé de mon téléphone Android avec K-9 Mail. Excusez la brièveté.

emoon commented 9 years ago

Hi,

No the disk isn't full. I don't get any error messages from what I can see but I will double check when I get back home.

Cheers!

lclevy commented 9 years ago

I think this is more related to startup-sequence usage, not to adflib bug. did you try the same thing via an emulator ? WinUAE for example (running real AmigaOS) ? http://www.winuae.net/, kickstart 1.3 should be easy to find

emoon commented 9 years ago

Hi again,

Here is the code that I have written: https://gist.github.com/emoon/bed404b49bd6ed1cb392

The usage is ADFCreate

The tool will generate the following layout:

S/Startup-Sequence
T
test_exe

Startup-Sequence will contain: test_exe

I have tested this with FS-UAE http://fs-uae.net

What happens if I try to boot on this disk generated is that it will not boot correctly. Then if I run (as stated above)

dh0:c/install df0:

And reboot the emulator the Amiga boots correctly on the disk (runs the executable and the test executable (linked in the code) prints a text line.

lclevy commented 9 years ago

So, what was certainly needed is a valid bootblock, as explained here: http://lclevy.free.fr/adflib/adf_info.html#p52. Did you try "RETCODE adfInstallBootBlock(struct Volume* vol, unsigned char* code) " ?

emoon commented 9 years ago

Thanks! I will try that.

emoon commented 9 years ago

Hi,

I managed to get this to work with some issues. First I had a stupid bug in my file loader (that read the file to memory which read from byte 1 instead of zero (doh!) and after fixing that and adding the bootblock it worked fine when I tested on Windows but not on the Mac.

It turns out the the lib doesn't work correctly when compiled in 64-bit mode but when I forced the code to compile in 32-bit mode (on Mac also) it works fine.

I haven't investigated but I noticed when diffing the working adf with the non-working. there are several places on the disk that differs and one is being the checksum for the bootblock:

screen shot 2015-07-21 at 16 40 58

(correct version on the left side)

I haven't verified this but by looking at this code

/*
 * adfBootSum
 *
 */
    uint32_t
adfBootSum(uint8_t *buf)
{
    uint32_t d, newSum;
    int i;

    newSum=0L;
    for(i=0; i<256; i++) {
        if (i!=1) {
            d = Long(buf+i*4);
            if ( (ULONG_MAX-newSum)<d )
                newSum++;
            newSum+=d;
        }
    }
    newSum = ~newSum;   /* not */

    return(newSum);
}

I would assume in this case ULONG_MAX will not work here in 64-bit as described here:

http://northstar-www.dartmouth.edu/doc/ibmcxx/en_US/doc/complink/ref/rucl64mg.htm

It might be that I also have forgot to send so flags in when compiling the lib (I'm not using the autobuild tools for the lib)

david-c14 commented 9 years ago

I guess ULONG_MAX is not really what's intended here, since the actual size of d and newSum are fixed at 32bits. It would be better either to hard code 4294967295UL or for clarity to provide a distinct definition for UINT32_MAX.

This appears to be the only occurrence of ULONG_MAX in the code.

emoon commented 9 years ago

I agree.

I haven't verified that that is the only 64-bit issue but I can try to take a look at it (of course if anyone has some other ideas of other potential 64-bit issues that would be good to hear before I start digging around :)

rofl0r commented 9 years ago

this code seems to be copied from doc/FAQ/adf_info* based on an ancient newsgroup post of the DOS area... 0xffffffffU seems better to me than 4294967295UL since its more readable (and not of type long) the Long() and Short() macros also look kinda ugly and could be replaced by something less misleading...

rofl0r commented 9 years ago

further the code is broken for big endian systems, since it does an unconditional byteswap. to work there as well, i suppose the code could just use ntohl() on the 32bit value of the pointed-to-data so the swap is only done on little endian. this here seems to work as a replacement for Long():

#include <arpa/inet.h>
#define unsigned_bigendian(X) ntohl(X[0] | (X[1] << 8) | (X[2] << 16) | (X[3] << 24))
emoon commented 9 years ago

Created a fix as suggested by @rofl0r and it now works correct for me on Mac (64-bit) I haven't tested 64-bit on Windows but I would assume the issue would been behaving the same there.

lclevy commented 9 years ago

Thanks emoon !

matsp888 commented 5 years ago

You need to call adfEnvInitDefault() to initialize the library before doing anything else. I lesrned it the hard way, lol.