Open peterzhuamazon opened 2 years ago
reta
commented
2 years ago @peterzhuamazon AFAIK it was intentional, one of the problems we run into was symbolic links (used by JDK distributions), the build tooling was failing trying to package tar + jdk bundle.
dblock
commented
2 years ago @reta That symlink problem was a JDK11 thing, wasn't it? Is it still an issue with 17.
@peterzhuamazon Feel free to fix this.
reta
commented
2 years ago @reta That symlink problem was a JDK11 thing, wasn't it? Is it still an issue with 17.
I think JDK-17 as well (it is mostly because of the JDK packaging), but I could recheck
peterzhuamazon
commented
2 years ago @reta That symlink problem was a JDK11 thing, wasn't it? Is it still an issue with 17.
@peterzhuamazon Feel free to fix this.
Hi,
I think these are two different issues.
JDK11 has a ja link which prevent cpio to extract.
As for this JDK linking mentioned by @reta I am not familiar with it.
Do you have any previous issue I can take a look?
Thanks.
reta
commented
2 years ago @peterzhuamazon it was this one https://github.com/opensearch-project/OpenSearch/pull/1433
Yury-Fridlyand
commented
2 years ago I can build linux-tar on WSL2 (Linux Subsystem for Windows). Tested on Windows 10 and 11.
dblock
commented
2 years ago @Yury-Fridlyand something you want to add to https://github.com/opensearch-project/OpenSearch/blob/main/DEVELOPER_GUIDE.md#windows ?
Yury-Fridlyand
commented
2 years ago Maybe, thanks for the idea. I used only ./gradlew :distribution:archives:linux-tar:assemble. I need to test other workflows and gradlew tasks before publishing any instructions.
[BUG] Cannot compile linux-tar on windows through cygwin/mingw. Using 2.0 branch, jdk11, on win10.
Windows Build Log
``` $ ./gradlew :distribution:archives:linux-tar:assemble -Dbuild.snapshot=false -Dbuild.version_qualifier=rc1 --console=plain > Task :buildSrc:reaper:compileJava UP-TO-DATE > Task :buildSrc:reaper:processResources NO-SOURCE > Task :buildSrc:reaper:classes UP-TO-DATE > Task :buildSrc:reaper:jar UP-TO-DATE > Task :buildSrc:reaper:assemble UP-TO-DATE > Task :buildSrc:reaper:compileTestJava NO-SOURCE > Task :buildSrc:reaper:processTestResources NO-SOURCE > Task :buildSrc:reaper:testClasses UP-TO-DATE > Task :buildSrc:reaper:test NO-SOURCE > Task :buildSrc:reaper:check UP-TO-DATE > Task :buildSrc:reaper:build UP-TO-DATE > Task :buildSrc:compileJava UP-TO-DATE > Task :buildSrc:compileGroovy UP-TO-DATE > Task :buildSrc:generateVersionProperties UP-TO-DATE > Task :buildSrc:pluginDescriptors UP-TO-DATE > Task :buildSrc:processResources UP-TO-DATE > Task :buildSrc:classes UP-TO-DATE > Task :buildSrc:jar UP-TO-DATE > Task :buildSrc:assemble UP-TO-DATE > Task :buildSrc:pluginUnderTestMetadata UP-TO-DATE > Task :buildSrc:compileTestFixturesJava UP-TO-DATE > Task :buildSrc:compileTestFixturesGroovy NO-SOURCE > Task :buildSrc:processTestFixturesResources NO-SOURCE > Task :buildSrc:testFixturesClasses UP-TO-DATE > Task :buildSrc:testFixturesJar UP-TO-DATE > Task :buildSrc:compileTestJava UP-TO-DATE > Task :buildSrc:compileTestGroovy NO-SOURCE > Task :buildSrc:processTestResources UP-TO-DATE > Task :buildSrc:testClasses UP-TO-DATE > Task :buildSrc:test SKIPPED > Task :buildSrc:validatePlugins UP-TO-DATE > Task :buildSrc:check UP-TO-DATE > Task :buildSrc:build UP-TO-DATE > Configure project :plugins:repository-hdfs hdfsFixture unsupported, please set HADOOP_HOME and put HADOOP_HOME\bin in PATH > Configure project :qa:os Cannot add task 'destructiveDistroTest.docker' as a task with that name already exists. ======================================= OpenSearch Build Hamster says Hello! Gradle Version : 7.3.3 OS Info : Windows 10 10.0 (amd64) JDK Version : 11 (Eclipse Adoptium JDK) JAVA_HOME : H:\Programs\msys\opt\jdk-11 Random Testing Seed : 93E290DD815CBAB4 In FIPS 140 mode : false ======================================= > Task :modules:geo:processResources NO-SOURCE > Task :modules:geo:copyPluginPropertiesTemplate UP-TO-DATE > Task :modules:geo:pluginProperties UP-TO-DATE > Task :modules:ingest-user-agent:processResources UP-TO-DATE > Task :modules:ingest-geoip:copyDefaultGeoIp2DatabaseFiles UP-TO-DATE > Task :modules:ingest-user-agent:copyPluginPropertiesTemplate UP-TO-DATE > Task :modules:analysis-common:processResources UP-TO-DATE > Task :modules:ingest-geoip:processResources NO-SOURCE > Task :libs:opensearch-dissect:compileJava UP-TO-DATE > Task :modules:analysis-common:copyPluginPropertiesTemplate UP-TO-DATE > Task :modules:lang-expression:processResources NO-SOURCE > Task :modules:ingest-user-agent:pluginProperties UP-TO-DATE > Task :modules:ingest-geoip:copyPluginPropertiesTemplate UP-TO-DATE > Task :libs:opensearch-dissect:processResources NO-SOURCE > Task :libs:opensearch-dissect:classes UP-TO-DATE > Task :libs:opensearch-plugin-classloader:compileJava UP-TO-DATE > Task :modules:analysis-common:pluginProperties UP-TO-DATE > Task :modules:ingest-geoip:pluginProperties UP-TO-DATE > Task :libs:opensearch-geo:compileJava UP-TO-DATE > Task :modules:aggs-matrix-stats:processResources UP-TO-DATE > Task :libs:opensearch-core:compileJava UP-TO-DATE > Task :modules:ingest-common:processResources UP-TO-DATE > Task :modules:aggs-matrix-stats:copyPluginPropertiesTemplate UP-TO-DATE > Task :server:generateModulesList UP-TO-DATE > Task :modules:lang-mustache:processResources NO-SOURCE > Task :libs:opensearch-secure-sm:compileJava UP-TO-DATE > Task :modules:lang-mustache:copyPluginPropertiesTemplate UP-TO-DATE > Task :modules:lang-expression:copyPluginPropertiesTemplate UP-TO-DATE > Task :modules:ingest-common:copyPluginPropertiesTemplate UP-TO-DATE > Task :modules:aggs-matrix-stats:pluginProperties UP-TO-DATE > Task :modules:lang-mustache:pluginProperties UP-TO-DATE > Task :libs:opensearch-grok:compileJava UP-TO-DATE > Task :server:generatePluginsList UP-TO-DATE > Task :modules:lang-expression:pluginProperties UP-TO-DATE > Task :libs:opensearch-cli:compileJava UP-TO-DATE > Task :libs:opensearch-core:processResources NO-SOURCE > Task :libs:opensearch-geo:processResources NO-SOURCE > Task :libs:opensearch-cli:processResources NO-SOURCE > Task :libs:opensearch-core:classes UP-TO-DATE > Task :modules:lang-painless:copyPluginPropertiesTemplate UP-TO-DATE > Task :modules:lang-painless:spi:processResources NO-SOURCE > Task :libs:opensearch-x-content:compileJava UP-TO-DATE > Task :modules:lang-painless:pluginProperties UP-TO-DATE > Task :libs:opensearch-x-content:processResources NO-SOURCE > Task :modules:mapper-extras:processResources NO-SOURCE > Task :modules:ingest-common:pluginProperties UP-TO-DATE > Task :server:processResources UP-TO-DATE > Task :libs:opensearch-grok:processResources UP-TO-DATE > Task :libs:opensearch-secure-sm:processResources NO-SOURCE > Task :modules:lang-painless:processResources UP-TO-DATE > Task :libs:opensearch-core:compileJava11Java NO-SOURCE > Task :libs:opensearch-cli:classes UP-TO-DATE > Task :libs:opensearch-geo:classes UP-TO-DATE > Task :libs:opensearch-grok:classes UP-TO-DATE > Task :libs:opensearch-core:processJava11Resources NO-SOURCE > Task :libs:opensearch-secure-sm:classes UP-TO-DATE > Task :libs:opensearch-x-content:classes UP-TO-DATE > Task :client:rest:compileJava UP-TO-DATE > Task :modules:mapper-extras:copyPluginPropertiesTemplate UP-TO-DATE > Task :modules:mapper-extras:pluginProperties UP-TO-DATE > Task :libs:opensearch-core:java11Classes UP-TO-DATE > Task :client:rest:processResources NO-SOURCE > Task :libs:opensearch-ssl-config:compileJava UP-TO-DATE > Task :client:rest:classes UP-TO-DATE > Task :modules:opensearch-dashboards:processResources NO-SOURCE > Task :libs:opensearch-ssl-config:processResources NO-SOURCE > Task :libs:opensearch-ssl-config:classes UP-TO-DATE > Task :modules:opensearch-dashboards:copyPluginPropertiesTemplate UP-TO-DATE > Task :libs:opensearch-dissect:jar UP-TO-DATE > Task :modules:reindex:processResources NO-SOURCE > Task :modules:opensearch-dashboards:pluginProperties UP-TO-DATE > Task :modules:parent-join:processResources UP-TO-DATE > Task :modules:percolator:processResources NO-SOURCE > Task :modules:parent-join:copyPluginPropertiesTemplate UP-TO-DATE > Task :modules:percolator:copyPluginPropertiesTemplate UP-TO-DATE > Task :modules:percolator:pluginProperties UP-TO-DATE > Task :modules:parent-join:pluginProperties UP-TO-DATE > Task :modules:rank-eval:processResources UP-TO-DATE > Task :modules:rank-eval:copyPluginPropertiesTemplate UP-TO-DATE > Task :modules:transport-netty4:copyPluginPropertiesTemplate UP-TO-DATE > Task :modules:reindex:copyParentJoinMetadata UP-TO-DATE > Task :modules:rank-eval:pluginProperties UP-TO-DATE > Task :modules:reindex:copyPluginPropertiesTemplate UP-TO-DATE > Task :modules:transport-netty4:pluginProperties UP-TO-DATE > Task :modules:transport-netty4:processResources NO-SOURCE > Task :modules:repository-url:processResources NO-SOURCE > Task :libs:opensearch-x-content:jar UP-TO-DATE > Task :modules:reindex:copyTransportNetty4Metadata UP-TO-DATE > Task :modules:repository-url:copyPluginPropertiesTemplate UP-TO-DATE > Task :modules:repository-url:pluginProperties UP-TO-DATE > Task :distribution:archives:createJvmOptionsDir > Task :libs:opensearch-secure-sm:jar UP-TO-DATE > Task :libs:opensearch-plugin-classloader:processResources NO-SOURCE > Task :distribution:archives:createLogsDir > Task :modules:reindex:pluginProperties UP-TO-DATE > Task :libs:opensearch-grok:jar UP-TO-DATE > Task :libs:opensearch-plugin-classloader:classes UP-TO-DATE > Task :distribution:tools:keystore-cli:processResources NO-SOURCE > Task :libs:opensearch-geo:jar UP-TO-DATE > Task :distribution:tools:plugin-cli:processResources UP-TO-DATE > Task :distribution:tools:java-version-checker:compileJava UP-TO-DATE > Task :distribution:tools:launchers:processResources NO-SOURCE > Task :distribution:archives:createPluginsDir > Task :distribution:tools:upgrade-cli:processResources NO-SOURCE > Task :distribution:tools:java-version-checker:processResources NO-SOURCE > Task :distribution:archives:linux-tar:distTar NO-SOURCE > Task :libs:opensearch-core:jar UP-TO-DATE > Task :distribution:archives:linux-tar:distZip NO-SOURCE > Task :libs:opensearch-cli:jar UP-TO-DATE > Task :server:compileJava UP-TO-DATE > Task :distribution:buildNotice UP-TO-DATE > Task :distribution:tools:launchers:compileJava UP-TO-DATE > Task :client:rest:jar UP-TO-DATE > Task :libs:opensearch-plugin-classloader:jar UP-TO-DATE > Task :libs:opensearch-ssl-config:jar UP-TO-DATE > Task :server:classes UP-TO-DATE > Task :modules:aggs-matrix-stats:compileJava UP-TO-DATE > Task :modules:geo:compileJava UP-TO-DATE > Task :modules:lang-painless:spi:compileJava UP-TO-DATE > Task :server:compileJava11Java NO-SOURCE > Task :server:processJava11Resources NO-SOURCE > Task :modules:ingest-geoip:compileJava UP-TO-DATE > Task :modules:aggs-matrix-stats:classes UP-TO-DATE > Task :modules:geo:classes UP-TO-DATE > Task :modules:lang-painless:spi:classes UP-TO-DATE > Task :modules:ingest-user-agent:compileJava UP-TO-DATE > Task :modules:ingest-geoip:classes UP-TO-DATE > Task :modules:ingest-user-agent:classes UP-TO-DATE > Task :server:java11Classes UP-TO-DATE > Task :modules:mapper-extras:compileJava UP-TO-DATE > Task :modules:reindex:compileJava UP-TO-DATE > Task :modules:lang-mustache:compileJava UP-TO-DATE > Task :modules:lang-painless:compileJava UP-TO-DATE > Task :modules:parent-join:compileJava UP-TO-DATE > Task :modules:mapper-extras:classes UP-TO-DATE > Task :modules:reindex:classes UP-TO-DATE > Task :modules:lang-expression:compileJava UP-TO-DATE > Task :modules:lang-expression:classes UP-TO-DATE > Task :modules:lang-mustache:classes UP-TO-DATE > Task :modules:opensearch-dashboards:compileJava UP-TO-DATE > Task :modules:geo:jar UP-TO-DATE > Task :modules:lang-painless:classes UP-TO-DATE > Task :modules:opensearch-dashboards:classes UP-TO-DATE > Task :modules:ingest-common:compileJava UP-TO-DATE > Task :modules:ingest-common:classes UP-TO-DATE > Task :modules:analysis-common:compileJava UP-TO-DATE > Task :modules:analysis-common:classes UP-TO-DATE > Task :modules:ingest-geoip:jar UP-TO-DATE > Task :modules:aggs-matrix-stats:jar UP-TO-DATE > Task :modules:lang-painless:spi:jar UP-TO-DATE > Task :modules:geo:bundlePlugin UP-TO-DATE > Task :modules:parent-join:classes UP-TO-DATE > Task :modules:ingest-user-agent:jar UP-TO-DATE > Task :modules:mapper-extras:jar UP-TO-DATE > Task :server:jar UP-TO-DATE > Task :modules:lang-mustache:jar UP-TO-DATE > Task :modules:ingest-geoip:bundlePlugin UP-TO-DATE > Task :modules:aggs-matrix-stats:bundlePlugin UP-TO-DATE > Task :modules:percolator:compileJava UP-TO-DATE > Task :modules:percolator:classes UP-TO-DATE > Task :modules:lang-expression:jar UP-TO-DATE > Task :modules:analysis-common:jar UP-TO-DATE > Task :modules:opensearch-dashboards:jar UP-TO-DATE > Task :modules:ingest-common:jar UP-TO-DATE > Task :modules:ingest-user-agent:bundlePlugin UP-TO-DATE > Task :modules:rank-eval:compileJava UP-TO-DATE > Task :modules:parent-join:jar UP-TO-DATE > Task :modules:rank-eval:classes UP-TO-DATE > Task :modules:lang-mustache:bundlePlugin UP-TO-DATE > Task :distribution:tools:java-version-checker:classes UP-TO-DATE > Task :modules:mapper-extras:bundlePlugin UP-TO-DATE > Task :modules:reindex:jar UP-TO-DATE > Task :modules:parent-join:bundlePlugin UP-TO-DATE > Task :modules:repository-url:compileJava UP-TO-DATE > Task :modules:repository-url:classes UP-TO-DATE > Task :modules:lang-painless:shadowJar UP-TO-DATE > Task :modules:opensearch-dashboards:bundlePlugin UP-TO-DATE > Task :distribution:tools:launchers:classes UP-TO-DATE > Task :distribution:tools:keystore-cli:compileJava UP-TO-DATE > Task :distribution:tools:keystore-cli:classes UP-TO-DATE > Task :modules:percolator:jar UP-TO-DATE > Task :modules:transport-netty4:compileJava UP-TO-DATE > Task :modules:transport-netty4:classes UP-TO-DATE > Task :distribution:tools:java-version-checker:jar UP-TO-DATE > Task :modules:rank-eval:jar UP-TO-DATE > Task :modules:percolator:bundlePlugin UP-TO-DATE > Task :modules:reindex:bundlePlugin UP-TO-DATE > Task :modules:lang-painless:bundlePlugin UP-TO-DATE > Task :modules:lang-expression:bundlePlugin UP-TO-DATE > Task :modules:repository-url:jar UP-TO-DATE > Task :modules:rank-eval:bundlePlugin UP-TO-DATE > Task :distribution:tools:keystore-cli:jar UP-TO-DATE > Task :distribution:tools:plugin-cli:compileJava UP-TO-DATE > Task :distribution:tools:upgrade-cli:compileJava UP-TO-DATE > Task :distribution:tools:plugin-cli:classes UP-TO-DATE > Task :distribution:tools:upgrade-cli:classes UP-TO-DATE > Task :modules:analysis-common:bundlePlugin UP-TO-DATE > Task :modules:repository-url:bundlePlugin UP-TO-DATE > Task :modules:transport-netty4:jar UP-TO-DATE > Task :modules:ingest-common:bundlePlugin UP-TO-DATE > Task :distribution:tools:launchers:jar UP-TO-DATE > Task :modules:transport-netty4:bundlePlugin UP-TO-DATE > Task :distribution:tools:upgrade-cli:jar UP-TO-DATE > Task :distribution:processOutputs UP-TO-DATE > Task :distribution:buildBin UP-TO-DATE > Task :distribution:buildConfig UP-TO-DATE > Task :distribution:tools:plugin-cli:jar UP-TO-DATE > Task :distribution:buildLog4jConfig UP-TO-DATE > Task :distribution:buildModules UP-TO-DATE > Task :distribution:archives:buildLinux SKIPPED > Task :distribution:archives:buildLinuxTar SKIPPED > Task :distribution:archives:linux-tar:assemble UP-TO-DATE BUILD SUCCESSFUL in 2s 154 actionable tasks: 3 executed, 151 up-to-date $ ll distribution/archives/linux-tar/ total 1 -rw-r--r-- 1 Peter None 404 May 1 21:32 build.gradle ```As you can see the task of buildlinuxtar skipped for some reason.
This means I have to start a VM or docker on Windows just to develop.
Thanks.