search

eclipse-windowbuilder / windowbuilder

Eclipse Windowbuilder
https://projects.eclipse.org/projects/tools.windowbuilder
Eclipse Public License 1.0
79 stars 30 forks source link

Permissions in zipped file #771

Closed dhmartin closed 4 months ago

dhmartin commented 4 months ago

Today I noticed that WindowBuilder was missing in my eclipse under Ubuntu. Recently I made some changes to the scripts I use to automate unzipping of all eclipse files I use. I'm trying to use newer 7za instead of the traditional unzip 6.0 dated April 2009. Checking back with my old scripts I realized that now, WindowBuilder subfolders (inside eclipse/dropins) are hidden to other users but root.

I know that zip files are not ideal for preserving permissions in Unix environments. I see that i could go back to the old unzip, or even change permissions myself easily with chmod…

Even though, let me suggest taking a look at the tools or procedure used for zipping WindowBuilder, as it seems to be different to the other eclipse projects:

dmartin@alamo:~/eclipse2024-03$ zipinfo WindowBuilder/WindowBuilder-Updates-1.15.0.zip | tail -10
-rw----     2.0 fat    13168 bl defN 24-Mar-04 19:04 features/org.eclipse.wb.layout.group.feature.source_1.15.0.202402012120.jar
-rw----     2.0 fat    13153 bl defN 24-Mar-04 19:04 features/org.eclipse.wb.core.java.feature.source_1.15.0.202403040813.jar
-rw----     2.0 fat    20852 bl defN 24-Mar-04 19:04 features/org.eclipse.wb.layout.group.feature_1.15.0.202402012120.jar
-rw----     2.0 fat    13122 bl defN 24-Mar-04 19:04 features/org.eclipse.wb.core.ui.feature.source_1.15.0.202402012120.jar
-rw----     2.0 fat    20991 bl defN 24-Mar-04 19:04 features/org.eclipse.wb.swing.feature_1.15.0.202402200547.jar
-rw----     2.0 fat    20906 bl defN 24-Mar-04 19:04 features/org.eclipse.wb.core.feature_1.15.0.202402200547.jar
-rw----     2.0 fat    24616 bl defN 24-Mar-04 19:04 content.xml.xz
-rw----     2.0 fat    43024 bl defN 24-Mar-04 19:04 artifacts.xml.xz
-rw----     2.0 fat      172 bl defN 24-Mar-04 19:04 p2.index
130 files, 66772301 bytes uncompressed, 61643650 bytes compressed:  7.7%
dmartin@alamo:~/eclipse2024-03$ zipinfo eclipse-jee-2024-03-R-win32-x86_64.zip | tail -10
-rw-r--r--  2.0 unx    11164 b- defN 24-Mar-04 15:50 eclipse/features/org.eclipse.wst.xsl.feature_1.3.1500.v202307260701/feature.properties
-rw-r--r--  2.0 unx       61 b- defN 24-Feb-29 10:28 eclipse/.eclipseproduct
-rw-r--r--  2.0 unx    77458 b- defN 24-Feb-29 10:28 eclipse/readme/readme_eclipse.html
-rw-r--r--  2.0 unx      907 b- defN 24-Mar-07 16:28 eclipse/eclipse.ini
-rw-r--r--  2.0 unx   121709 b- defN 24-Mar-07 16:28 eclipse/configuration/org.eclipse.equinox.simpleconfigurator/bundles.info
-rw-r--r--  2.0 unx      905 b- defN 24-Mar-07 16:28 eclipse/configuration/config.ini
-rw-r--r--  2.0 unx    32509 b- defN 24-Mar-07 16:28 eclipse/configuration/org.eclipse.update/platform.xml
-rw-r--r--  2.0 unx   533384 b- defN 24-Mar-07 16:34 eclipse/eclipse.exe
-rw-r--r--  2.0 unx   238472 b- defN 24-Mar-07 16:34 eclipse/eclipsec.exe
11975 files, 722242106 bytes uncompressed, 541899069 bytes compressed:  25.0%
dmartin@alamo:~/eclipse2024-03$ zipinfo Babel/BabelLanguagePack-eclipse-es_4.26.0.v20230220105658.zip | tail -10
-rw-rw-r--  3.0 unx     5156 bx defN 23-Feb-21 10:05 eclipse/plugins/org.eclipse.pde.ui.templates.nl_es_4.26.0.v20230220105658.jar
-rw-rw-r--  3.0 unx     3755 bx defN 23-Feb-21 10:05 eclipse/plugins/org.eclipse.equinox.p2.publisher.eclipse.nl_es_4.26.0.v20230220105658.jar
-rw-rw-r--  3.0 unx     3163 bx defN 23-Feb-21 10:05 eclipse/plugins/org.eclipse.e4.ui.workbench.renderers.swt.cocoa.nl_es_4.26.0.v20230220105658.jar
-rw-rw-r--  3.0 unx     1445 bx defN 23-Feb-21 10:05 eclipse/plugins/org.eclipse.ui.intro.quicklinks.nl_es_4.26.0.v20230220105658.jar
-rw-rw-r--  3.0 unx     1448 bx defN 23-Feb-21 10:05 eclipse/plugins/org.eclipse.jface.tests.databinding.nl_es_4.26.0.v20230220105658.jar
-rw-rw-r--  3.0 unx     1442 bx defN 23-Feb-21 10:05 eclipse/plugins/org.eclipse.team.tests.core.nl_es_4.26.0.v20230220105658.jar
-rw-rw-r--  3.0 unx     3596 bx defN 23-Feb-21 10:05 eclipse/plugins/org.eclipse.equinox.p2.installer.nl_es_4.26.0.v20230220105658.jar
-rw-rw-r--  3.0 unx     1490 bx defN 23-Feb-21 10:05 eclipse/plugins/org.eclipse.jdt.doc.isv.nl_es_4.26.0.v20230220105658.jar
-rw-rw-r--  3.0 unx     1578 bx defN 23-Feb-21 10:05 eclipse/plugins/org.eclipse.ui.tests.navigator.nl_es_4.26.0.v20230220105658.jar
284 files, 1431607 bytes uncompressed, 1066028 bytes compressed:  25.5%

zipinfo -v says that the origin was MS-DOS, OS/2 or NT FAT and recreating proper UNIX permissions there may not be straight… Anyway, it might be worth reviewing this.

Thanks, David

merks commented 4 months ago

That content is produced by JustJ's p2 manager:

I think it's a know issue that this does not preserve posix permissions.

Perhaps if it a zip file system it would do a better job.

https://docs.oracle.com/javase/8/docs/technotes/guides/io/fsp/zipfilesystemprovider.html

I'm not sure if I can know if does a better job developing on Windows...

merks commented 4 months ago

An issue would need to be opened here:

https://github.com/eclipse-justj/justj.tools/issues

merks commented 4 months ago

I committed these changes:

https://github.com/eclipse-justj/justj.tools/commit/8e98d828aa9b0d2fb10eef0c36152a081f840518

It uses Files.copy(file, targetPath, StandardCopyOption.COPY_ATTRIBUTES) when creating the zip content.

I think that's the best I can do to try to preserve attributes.

merks commented 4 months ago

FYI, I'm not sure if this version is better:

https://download.eclipse.org/windowbuilder/updates/nightly/N202405211026/WindowBuilder-Updates-N202405211026.zip

If not, I don't think I can do better from within a Java application.

dhmartin commented 4 months ago

No apparent changes…

chapas@ID19:~/tmp$ zipinfo WindowBuilder-Updates-N202405211026.zip | tail -10
-rw----     2.0 fat    20832 bX defN 24-May-21 13:00 features/org.eclipse.wb.rcp.doc.user.feature_1.16.0.202403132032.jar
-rw----     2.0 fat    20855 bX defN 24-May-21 13:00 features/org.eclipse.wb.rcp.SWT_AWT_support_1.16.0.202403201915.jar
-rw----     2.0 fat    20840 bX defN 24-May-21 13:00 features/org.eclipse.wb.swt.feature_1.16.0.202405081620.jar
-rw----     2.0 fat    13142 bX defN 24-May-21 13:00 features/org.eclipse.wb.swing.feature.source_1.16.0.202405151946.jar
-rw----     2.0 fat    13180 bX defN 24-May-21 13:00 features/org.eclipse.wb.core.feature.source_1.16.0.202405151946.jar
-rw----     2.0 fat    13119 bX defN 24-May-21 13:00 features/org.eclipse.wb.core.ui.feature.source_1.16.0.202403132032.jar
-rw----     2.0 fat    25640 bX defN 24-May-21 13:00 content.xml.xz
-rw----     2.0 fat    44612 bX defN 24-May-21 13:00 artifacts.xml.xz
-rw----     2.0 fat      172 bX defN 24-May-21 13:00 p2.index
137 files, 66883166 bytes uncompressed, 61705110 bytes compressed:  7.7%

Compared to the main JEE IDE zip file, as shown above, permissions for plain files should be -rw-r--r-- instead of -rw----, and -rwxr-xr-x for directories.

Folders are now included in this new zip:

chapas@ID19:~/tmp$ zipinfo WindowBuilder-Updates-1.15.0.zip | grep /$
chapas@ID19:~/tmp$ zipinfo WindowBuilder-Updates-N202405211026.zip | grep /$
-rw----     1.0 fat        0 bx stor 24-May-21 13:00 plugins/
-rw----     1.0 fat        0 bx stor 24-May-21 13:00 features/
merks commented 4 months ago

Yes folder are included so that I could copy their attributes, but that doesn't actually do any actual good.

All the logic is here:

https://github.com/eclipse-justj/justj.tools/blob/8e98d828aa9b0d2fb10eef0c36152a081f840518/plugins/org.eclipse.justj.p2/src/org/eclipse/justj/p2/UpdateSiteGenerator.java#L1601-L1613

Maybe there are some clues here one could try:

https://bugs.openjdk.org/browse/JDK-8213082

Maybe if you care enough, you could contribute something the works?

dhmartin commented 4 months ago

If we share the same p2 tools ¿why other projects produce different zip profiles?

I thought that creating the update site archive was something of a Maven task. Looking at this Java code, I see the Files.copy() option set to keep the original file attributes. How were those original attributes set?

What's the hosting OS? Is it naive to try setting them on purpose for each file or directory copied?

private static final Set<PosixFilePermission> PERMS_FILE, PERMS_EXECUTABLE_DIRECTORY;

static {
    PERMS_FILE = EnumSet.of(OWNER_READ, OWNER_WRITE, GROUP_READ, OTHERS_READ);
    PERMS_EXECUTABLE_DIRECTORY = EnumSet.of(OWNER_READ, OWNER_WRITE, OWNER_EXECUTE, GROUP_READ, GROUP_EXECUTE,
            OTHERS_READ, OTHERS_EXECUTE);
}

public static void main(String[] args) throws IOException {
    Path file1Ref = Path.of("ejemplo.txt");
    Path file2Ref = Path.of("ejemplo2.txt");

    Files.copy(file1Ref, file2Ref, StandardCopyOption.REPLACE_EXISTING);
    if (Files.isExecutable(file2Ref) || Files.isDirectory(file2Ref)) {
        Files.setPosixFilePermissions(file2Ref, PERMS_EXECUTABLE_DIRECTORY);
    } else if (Files.isRegularFile(file2Ref)) {
        Files.setPosixFilePermissions(file2Ref, PERMS_FILE);
    }
}
merks commented 4 months ago

Most other projects use shell scripts for lots of stuff such as this. The zip file for the update site as a whole is generally intended for folks who are behind a firewall and can't use the download.eclipse.org server for updates. That folks use it to unzip into a dropins folder in an installation is interesting, but of course in that case, it's each to use chmod...

For JustJ's p2 manager I need something that will run on any OS nad can do what this is doing:

  public Path createArchive(Path repository) throws IOException
  {
    Path archiveFile = getArchiveFile(repository);
    Path folder = archiveFile.getParent();
    if (!Files.isDirectory(folder) || !Files.isRegularFile(folder.resolve("content.jar")))
    {
      throw new IllegalStateException(repository + "is not a valid p2 repository");
    }

    // Delete it if it already exists.
    if (Files.isRegularFile(archiveFile))
    {
      Files.delete(archiveFile);
    }

    URI archiveURI = URI.create("jar:" + archiveFile.toUri());
    boolean delete = false;
    try (FileSystem fileSystem = FileSystems.newFileSystem(archiveURI, Map.of("create", true)))
    {
      for (Path file : Files.list(folder).collect(Collectors.toList()))
      {
        String name = file.getFileName().toString();
        if (UPDATE_SITE_CONTENT.contains(name))
        {
          visit(fileSystem, folder, name);
        }
      }
    }
    catch (IOException exception)
    {
      delete = true;
    }
    finally
    {
      if (delete)
      {
        Files.delete(archiveFile);
      }
    }

    return archiveFile;
  }

  private static void visit(FileSystem fileSystem, Path root, String path) throws IOException
  {
    Path file = root.resolve(path);
    Path targetPath = fileSystem.getPath(path);
    Files.copy(file, targetPath, StandardCopyOption.COPY_ATTRIBUTES);
    if (Files.isDirectory(file))
    {
      for (Path child : Files.list(file).collect(Collectors.toList()))
      {
        visit(fileSystem, root, path + "/" + child.getFileName());
      }
    }
  }

I.e., copy some file system folders into a zip file system and produce the thing that you personally want to see...

dhmartin commented 4 months ago

The weak point to me seems that Files.copy(file, targetPath, StandardCopyOption.COPY_ATTRIBUTES); in the visit(…) method. Setting Posix permissions worked for me, but I supppose it's because I'm using Ubuntu.

Files.copy(file1Ref, file2Ref, StandardCopyOption.REPLACE_EXISTING);
if (Files.isExecutable(file2Ref) || Files.isDirectory(file2Ref)) {
    Files.setPosixFilePermissions(file2Ref, PERMS_EXECUTABLE_DIRECTORY);
} else if (Files.isRegularFile(file2Ref)) {
    Files.setPosixFilePermissions(file2Ref, PERMS_FILE);
}

I have been doing some testing about creating a ZIPfs from Java. It mostly worked. I just had to provide an option to include support for Posix attributes.

Path file1Ref = Path.of("ejemplo.txt");
Path file2Ref = Path.of("ejemplo2.txt");
Path zipfileRef = Path.of("ejemplo.zip");

URI zipUri = new URI("jar:file", zipfileRef.toUri().getPath(), null);

Map<String, String> env = new HashMap<>();
env.put("enablePosixFileAttributes", "true");
env.put("create", "true");
FileSystem zipfs = FileSystems.newFileSystem(zipfileRef, env);

Path root = zipfs.getPath("/");
file2Ref = zipfs.getPath(root.toString(), file2Ref.toString());

Files.createDirectories(zipfs.getPath("lib"), PosixFilePermissions.asFileAttribute(PERMS_EXECUTABLE_DIRECTORY));
Files.copy(file1Ref, file2Ref, StandardCopyOption.REPLACE_EXISTING);
if (Files.isExecutable(file2Ref) || Files.isDirectory(file2Ref)) {
    Files.setPosixFilePermissions(file2Ref, PERMS_EXECUTABLE_DIRECTORY);
} else if (Files.isRegularFile(file2Ref)) {
    Files.setPosixFilePermissions(file2Ref, PERMS_FILE);
}
zipfs.close();

So far, I got:

dmartin@alamo:~/eclipse/05.20 EjemplosArchivos$ zipinfo ejemplo.zip 
Archive:  ejemplo.zip
Zip file size: 324 bytes, number of entries: 2
?rwxr-xr-x  1.0 unx        0 bx stor 24-May-21 18:56 lib/
?rw-r--r--  2.0 unx       68 bX defN 24-May-21 18:56 ejemplo2.txt
2 files, 68 bytes uncompressed, 50 bytes compressed:  26.5%

…which looks fine, except for those suspicious interrogation marks next to the permission strings. Unzipping worked fine as expected.

You may take a look at the options you set in the try-with-resources statement Map.of("create", true), by the middle of createArchive(…), as adding there "enablePosixFileAttributes", "true" might as well help.

About my former scripts, I managed to reset permissions easily by fixing my chmod.

David

merks commented 4 months ago

@dhmartin

FYI, it's working like this now:

image

ptziegler commented 4 months ago

I was wondering whether this had something to do with justj but I never would've thought that this is because of Java 17/21. Nice find!

merks commented 4 months ago

I did add "enablePosixFileAttributes" to the env, but testing on Linux it still didn't work. Even setting the posix attributes as @dhmartin showed it still did not work. That made me suspicious enough to try a new Adoptium JDK. Jeesh!!