Cannot import Apache Iceberg project

laurentgo commented 3 days ago

Visual Studio fails to import the Apache Iceberg project correctly with classes not being referenced properly. Workspace logs show the following message:

!ENTRY org.eclipse.jdt.core 4 4 2024-09-23 15:23:43.332
!MESSAGE Invalid ZIP archive: /Users/laurent/.gradle/caches/modules-2/files-2.1/org.apache.curator/apache-curator/2.7.1/b223ec624c73f9cf9277287bb71d32cb9b1fcd3f/apache-curator-2.7.1.pom
!STACK 0
java.util.zip.ZipException: zip END header not found
    at java.base/java.util.zip.ZipFile$Source.findEND(ZipFile.java:1649)
    at java.base/java.util.zip.ZipFile$Source.initCEN(ZipFile.java:1657)
    at java.base/java.util.zip.ZipFile$Source.<init>(ZipFile.java:1495)
    at java.base/java.util.zip.ZipFile$Source.get(ZipFile.java:1458)
    at java.base/java.util.zip.ZipFile$CleanableResource.<init>(ZipFile.java:724)
    at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:251)
    at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:180)
    at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:194)
    at org.eclipse.jdt.internal.core.JavaModelManager.getZipFile(JavaModelManager.java:3043)
    at org.eclipse.jdt.internal.core.JavaModelManager.getZipFile(JavaModelManager.java:2977)
    at org.eclipse.jdt.internal.core.JarPackageFragmentRoot.getJar(JarPackageFragmentRoot.java:262)
    at org.eclipse.jdt.internal.core.JarPackageFragmentRoot.computeChildren(JarPackageFragmentRoot.java:130)
    at org.eclipse.jdt.internal.core.JavaProjectElementInfo.initializePackageNames(JavaProjectElementInfo.java:276)
    at org.eclipse.jdt.internal.core.JavaProjectElementInfo.getProjectCache(JavaProjectElementInfo.java:244)
    at org.eclipse.jdt.internal.core.JavaProject.getProjectCache(JavaProject.java:2387)
    at org.eclipse.jdt.internal.core.JavaProjectElementInfo.newNameLookup(JavaProjectElementInfo.java:332)
    at org.eclipse.jdt.internal.core.JavaProject.newNameLookup(JavaProject.java:2798)
    at org.eclipse.jdt.internal.core.SearchableEnvironment.<init>(SearchableEnvironment.java:107)
    at org.eclipse.jdt.internal.core.SearchableEnvironment.<init>(SearchableEnvironment.java:148)
    at com.microsoft.java.builder.JavaNameEnvironment.<init>(JavaNameEnvironment.java:42)
    at com.microsoft.java.builder.JavaProblemChecker.createNameEnvironment(JavaProblemChecker.java:51)
    at com.microsoft.java.builder.jdtbuilder.JavaBuilder.initializeBuilder(JavaBuilder.java:653)
    at com.microsoft.java.builder.jdtbuilder.JavaBuilder.build(JavaBuilder.java:192)
    at org.eclipse.core.internal.events.BuildManager$2.run(BuildManager.java:1077)
    at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:47)
    at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:296)
    at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:352)
    at org.eclipse.core.internal.events.BuildManager$1.run(BuildManager.java:441)
    at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:47)
    at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:444)
    at org.eclipse.core.internal.events.BuildManager.basicBuildLoop(BuildManager.java:555)
    at org.eclipse.core.internal.events.BuildManager.basicBuildLoop(BuildManager.java:503)
    at org.eclipse.core.internal.events.BuildManager.build(BuildManager.java:585)
    at org.eclipse.core.internal.events.AutoBuildJob.doBuild(AutoBuildJob.java:207)
    at org.eclipse.core.internal.events.AutoBuildJob.run(AutoBuildJob.java:300)
    at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)

The local file is not a zip file but an actual pom file (as hinted by the file extension)

Same project opens find in other editors like Intellij

Environment

Operating System: MacOS
JDK version: 21.0.4
Visual Studio Code version: 1.93.1 (commit: 38c31bc77e0dd6ae88a4e9cc93428cc27a56ba40)
Java extension version: 1.34.0

Steps To Reproduce

Clone apache/iceberg project and import it in visual studio
Use JDK 21 for java.jdt.ls.java.home and java.import.gradle.java.home settings, and reload

[Please attach a sample project reproducing the error] Please attach logs

Current Result

Project is not imported correctly, some org.apache.iceberg classes are not discovered properly.

Expected Result

Project is imported successfully.

Additional Informations

laurentgo commented 3 days ago

I forgot that there's another issue with the project defining two resources on top of each other, which can be worked around by commenting the following lines in build.gradle:

  sourceSets {
    test {
      resources {
        srcDir "${rootDir}/build"
      }
    }
  }

snjeza commented 8 hours ago

@laurentgo Could you try the following:

build the iceberg project from the command line.
```
$ cd <iceberg_root>
$ ./gradlew build -x test
```

apply the following patch


index 02758e2a7..1372cd9f8 100644
--- a/build.gradle
+++ b/build.gradle
@@ -163,6 +163,28 @@ subprojects {
 }
}

apply plugin: 'eclipse'
if (project.name != 'iceberg-bundled-guava') {
eclipse {
classpath {
containers 'org.eclipse.buildship.core.gradleclasspathcontainer'
file {
whenMerged {
entries.findAll { entry -> entry instanceof org.gradle.plugins.ide.eclipse.model.ProjectDependency }
.each {
it.entryAttributes['without_test_code'] = 'false'
}
def fileName = rootProject.projectDir.getAbsolutePath() + '/bundled-guava/build/libs/iceberg-bundled-guava-' + rootProject.version + '.jar'
def file = new File(fileName)
def lib = new org.gradle.plugins.ide.eclipse.model.Library(fileReference(file))
lib.exported = true;
entries += lib
}
}
}
}
}
configurations { testImplementation.extendsFrom compileOnly


3. start VS Code

laurentgo commented 6 hours ago

Sadly, it doesn't seem to work/the error is still present. I don't see new error messages either, so I am not sure if the code above is invoked or not

snjeza commented 6 hours ago

...so I am not sure if the code above is invoked or not

You can check if there is iceberg-aliyun>Project and External Dependencies>iceberg-bundled-guava-1.7.0-SNAPSHOT.jar in the Java Projects view.

Could you also try the following:

exit VS Code

$ cd <iceberg>
$ git clean -fd
$ ./gradlew build -x test
$ ls bundled-guava/build/libs/iceberg-bundled-guava-1.7.0-SNAPSHOT.jar 
bundled-guava/build/libs/iceberg-bundled-guava-1.7.0-SNAPSHOT.jar
$ code .

laurentgo commented 5 hours ago

I do not see see iceberg-bundled-guava-1.7.0-SNAPSHOT.jar in the list of referenced libraries:

I do see the bundled libray at bundled-guava/build/libs/iceberg-bundled-guava-1.7.0-SNAPSHOT.jar though

laurentgo commented 5 hours ago

if I look at the .classpath file, I only see this:

     <classpathentry kind="src" path="/bundled-guava">
         <attributes>
             <attribute name="gradle.buildServer" value="true"/>
         </attributes>
     </classpathentry>

snjeza commented 4 hours ago

@laurentgo Could you add

"gradle.autoDetect": "off",
"java.gradle.buildServer.enabled": "off",

to settings.json?

redhat-developer / vscode-java