Don't Use DYLD_ROOT_PATH (In darlingserver, mldr, And vchroot)

[CuriousTommy]

Background

It turns out that dyld uses the DYLD_ROOT_PATH environment variable.

https://github.com/darlinghq/darling-dyld/blob/63f667cf06d7ed59553adebb0c8d70a117135ac9/src/dyld2.cpp#L6598-L6612

    // if this is host dyld, check to see if iOS simulator is being run
    const char* rootPath = _simple_getenv(envp, "DYLD_ROOT_PATH");
    if ( (rootPath != NULL) ) {
        // look to see if simulator has its own dyld
        char simDyldPath[PATH_MAX]; 
        strlcpy(simDyldPath, rootPath, PATH_MAX);
        strlcat(simDyldPath, "/usr/lib/dyld_sim", PATH_MAX);
        int fd = dyld3::open(simDyldPath, O_RDONLY, 0);
        if ( fd != -1 ) {
            const char* errMessage = useSimulatorDyld(fd, mainExecutableMH, simDyldPath, argc, argv, envp, apple, startGlue, &result);
            if ( errMessage != NULL )
                halt(errMessage);
            return result;
        }
    }

This is an issue since darlingserver, mldr, and vchroot also use the same environment variable.

https://github.com/darlinghq/darlingserver/blob/62a3321e98cbc9bc9d52c8f6f0d588e65c789954/src/darlingserver.cpp#L269

    setenv("DYLD_ROOT_PATH", LIBEXEC_PATH, 1);

https://github.com/darlinghq/darling/blob/25afbc76428c39c3909e9efcf5caef1140425211/src/startup/mldr/mldr.c#L522-L529 https://github.com/darlinghq/darling/blob/25afbc76428c39c3909e9efcf5caef1140425211/src/vchroot/vchroot.c#L49

Solution

A proper fix would be to reword the DYLD_ROOT_PATH environment variable name (that darlingserver, mldr, and vchroot uses) to something that is more specific to Darling (ex: DARLING_DYLD_ROOT_PATH).

[CuriousTommy]

@CKegel If you (or your team) want to work on this, I don't mind assigning this ticket to you.

[CKegel]

I wouldn't mind taking it up - I'd just like to get of clarity regarding the appropriate fix. It seems like a simple search and replace as I currently understand it, but that seems off.

[CuriousTommy]

It seems like a simple search and replace as I currently understand it, but that seems off.

Assuming that everything works fine (you're able to access the bash shell), that is pretty much what you have to do. With that being said, I would ask you to make sure that the code doesn't go into the simulator logic (inside the if ( (rootPath != NULL) ) scope). You can use of __simple_printf to verify the code path dyld takes.

However, if you run into the following issues:

• Darling is no longer able to bring up the bash shell.
• dyld still executes the code inside the if ( (rootPath != NULL) )

Then you will need to do some additional research/debugging to find the fix.

[luchsj]

This looks like something we can take care of. I'll start looking into it now.

[CKegel]

@CuriousTommy we have attempted changing the enviornment variable name to DARLING_DYLD_ROOT_PATH but are no longer able to bring up the shell. We've been trying to debug this for a bit but are stumped. The error we recieve is:

Bootstrapping the container with launchd... Cannot open /usr/lib/dyld: No such file or directory Error connecting to shellspawn in the container (/home/user/.darling/var/run/shellspawn.sock): No such file or directory

Any insight would be appreciated.

[CKegel]

I renenge my previous comment, we were not applying the change to all appropriate locations (darlingserver, mldr, and vchroot). We are still unable to launch bash, this time as a result of dyld failing to load libSystem. The output is included below:

Bootstrapping the container with launchd... dyld: Library not loaded: /usr/lib/libSystem.B.dylib Referenced from: /usr/libexec/darling/vchroot Reason: image not found abort_with_payload: reason: dyld: No shared cache present Library not loaded: /usr/lib/libSystem.B.dylib Referenced from: /usr/libexec/darling/vchroot Reason: image not found; code: 1 Cannot open mnt namespace file: No such file or directory

[luchsj]

This is the same issue I was encountering a while ago, so dyld is likely getting lost somewhere. Looking through binaries on the host confirms that libSystem.B is indeed there, so right now it's a matter of finding what else is referencing DYLD_ROOT_PATH. I couldn't find anything outside of external/dyld that references it (other than what we've already replaced), and print statements seem to indicate that dyld2.cpp isn't being reached before the error.

[CuriousTommy]

I wonder if something has gone horribly wrong with the vchroot expand method...

Now that I think about it, it's not too surprising that this is not working (since I ran into this issue when working on ARM64 support)

[CKegel]

What type of issue did you run into on ARM64?

[CuriousTommy]

What type of issue did you run into on ARM64?

It was the stat structure being different on ARM64: https://github.com/darlinghq/darling-xnu/commit/51f070bf4ba15a9d0384a81f118f6a43f0fa7c11

Edit: But this is probably not the reason why this is not working.