search

libgdx / packr

Packages your JAR, assets and a JVM for distribution on Windows, Linux and Mac OS X
Apache License 2.0
2.56k stars 171 forks source link

Creating symbolic link files on Windows for MacOs and Linux #197

Closed recepsahin closed 3 years ago

recepsahin commented 3 years ago

For version 3.0.3, in extractTarArchive method of ArchiveUtils class, while extracting symbolic link files it creates new symbolic files, but the created ones are system specific. If I create MacOs bundle on Windows it creates Windows specific symbolic link files and these are not compatible with MacOs. Symbolic link file structures are different for Windows and MacOs as far as I know. The same problem exists for Linux bundle created on Windows.

petoncle commented 3 years ago

Nice find. Does it prevent the app to run? It is not possible to create Linux/macOS symlinks on Windows, right? After a quick Google search I couldn't find a conclusive answer.

recepsahin commented 3 years ago

When I create MacOs bundle on Windows10 I could not copy/(zip and copy) to my flash drive to test on MacOs because symbolic link files causes file access errors. As far as I understand from my quick search it is not possible creating MacOs/Linux style symbolic link files on Windows.

karlsabo commented 3 years ago

Which files are still symbolic links? I've only seen symbolic links in JDKs not JREs.

Zips can handle symbolic links but it probably depends on the implementation, try transferring the files using a TAR archive so it keeps symbolic links.

recepsahin commented 3 years ago

Hi Karl, I've added a single line to extractTarArchive method to get a list of modified link files:

if (entry.isLink()) {
        Path linkTarget = Paths.get(entry.getLinkName());
        Files.deleteIfExists(entryExtractPath);
        Files.createLink(entryExtractPath, linkTarget);
} else if (entry.isSymbolicLink()) {
        Path linkTarget = Paths.get(entry.getLinkName());
        Files.deleteIfExists(entryExtractPath);
        Files.createSymbolicLink(entryExtractPath, linkTarget);
                System.out.println("SybomlicLink : " + linkTarget);
} else {
        if (entry.isDirectory()) {
                Files.createDirectories(entryExtractPath);
        } else {
                Files.createDirectories(entryExtractPath.getParent());
                Files.copy(archiveInputStream, entryExtractPath, StandardCopyOption.REPLACE_EXISTING);
        }
}

The output is :

java -jar Packr-3.0.3-all.jar packr_config.json
Copying executable ...
Copying classpath(s) ...
Unpacking JRE ...
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\cldr.md
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ..\java.base\LICENSE
SybomlicLink : ..\java.base\ADDITIONAL_LICENSE_INFO
SybomlicLink : ..\java.base\ASSEMBLY_EXCEPTION
SybomlicLink : ja_JP.UTF-8
Copying resources ...
Done!

I'm using Adopt OpenJDK: OpenJDK11U-jre_x64_mac_hotspot_11.0.10_9.tar.gz I've created tar archive and managed to test on MacOs, everything works fine but I think the files listed above are not any more MacOs compatible.

karlsabo commented 3 years ago

If you transfer files from one file system to another, eg NTFS to ext4, you've got to use apps that can handle links on both sides. 7-zip TAR should be able to do this for you from the Windows side https://superuser.com/a/1534363

petoncle commented 3 years ago

I think the problem @recepsahin has is that when he runs packr, packr extracts the JDK archive on his Windows NTFS file system. When packr does that, the portable symlinks from inside the zip are converted to Windows specific symlinks and written to disk. If he then creates a tar out of packr's output folder, the tar is going to contain Windows symlinks that don't work on Linux and macOS? I could be wrong though, maybe tar converts the Windows specific symlinks to portable symlinks (non-Windows specific)?

karlsabo commented 3 years ago

Yes, that's what I'm trying to explain, to transfer symbolic links you must use a filesystem and archive format which supports them.

Archives are like their own mini filesystems, if you need to transfer symbolic links you need to use a proper archive, such as TAR which supports symbolic links.

Download Adopt JRE, and use the latest 7-Zip to extract it and you'll see:

If you re-TAR that folder using 7-zip and preserve symbolic links, you can then open that tar in 7-zip and see that the file is still a symbolic link.

Thinking of it as "this is a Windows symbolic link" and it's not portable is not the correct mindset. Windows 10 runs on the NTFS filesystem, macOS runs on APFS filesystem, so of course they aren't "compatible".

The issue here is "How do I transfer files and symbolic links across filesystems?". The easiest way is to create an archive that supports symbolic links, which TAR does well.

Windows 10 -> flash drive -> macOS won't preserve symbolic links, why? Because most flash drives are formatted using the FAT filesystem which doesn't support link.

Windows 10 -> Zip -> macOS won't preserve symbolic links, why? Because most zip applications (e.g. 7-Zip) won't store symbolic links in the Zip archive format.

Windows 10 -> TAR -> macOS can preserve symbolic links, why? Because most tar applications (e.g. 7-Zip) have the option to store symbolic links in the TAR archive format.

recepsahin commented 3 years ago

Thank you so much @karlsabo for your explanation. I opened this issue because I thought re-creation of symbolic links is erroneous, but according to your explanation creating a proper tar archive and extracting it on the target system is safe.