search

vaadin / base-starter-flow-quarkus

A project base/example for using Vaadin with Quarkus
The Unlicense
37 stars 20 forks source link

Missing pnpm dependencies when building Quarkus starter on Jenkins using Gradle #24

Open luc-rovio opened 2 years ago

luc-rovio commented 2 years ago

I created an app based on base-starter-flow-quarkus (very few changes, I only added a logo at this time). I'm building on Jenkins using Gradle, with the vaadinBuildFrontend target.

I get the following error:

13:00:07  > Task :compileJava
13:00:17  > Task :vaadinPrepareFrontend
13:00:17  > Task :processResources
13:00:17  > Task :classes
13:00:23  
13:00:23  > Task :vaadinBuildFrontend FAILED
13:00:23  Command `/root/.vaadin/node/node /root/.vaadin/node/node_modules/npm/bin/npx-cli.js --yes --quiet pnpm@5.18.10 --shamefully-hoist=true install` failed:
13:00:23  
13:00:23  >>> Dependency ERROR. Check that all required dependencies are deployed in pnpm repositories.
13:00:23  
13:00:23  FAILURE: Build failed with an exception.
13:00:23  
13:00:23  * What went wrong:
13:00:23  Execution failed for task ':vaadinBuildFrontend'.
13:00:23  > com.vaadin.flow.server.ExecutionFailedException: Pnpm install has exited with non zero status. Some dependencies are not installed. Check pnpm command output

The build runs on a Linux agent based on a amazoncorretto:17.0.0 (Java 17) with Node 14.17.1 The build was successful with the build target but this cannot be deployed to the cloud since it would be running in dev mode.

mvysny commented 2 years ago

Hi, which Vaadin version are you using please? I think that at some point Vaadin reverted the default package manager from pnpm to npm, so maybe it's better to use npm. Could you please try running ./gradlew vaadinDance and switch to npm, then try again, whether that would help? Alternatively, could you run ./gradlew --info --stacktrace to obtain more info?

luc-rovio commented 2 years ago

My project is almost identical to the project base at https://github.com/vaadin/base-starter-flow-quarkus/tree/gradle so the Vaadin version is 22.0.2. It also uses vaadinQuarkusVersion=1.0.0. I tried ./gradlew vaadinDance but this target was not recognized. Here is the output of ./gradlew --info --stacktrace (I removed the massive dependency download and Java compilation) :

11:01:40  file or directory '/home/jenkins/agent/workspace/fika-admin-tool_master/build/classes/java/quarkus-generated-sources/grpc', not found
11:01:40  file or directory '/home/jenkins/agent/workspace/fika-admin-tool_master/build/classes/java/quarkus-generated-sources/avdl', not found
11:01:40  file or directory '/home/jenkins/agent/workspace/fika-admin-tool_master/build/classes/java/quarkus-generated-sources/avpr', not found
11:01:40  file or directory '/home/jenkins/agent/workspace/fika-admin-tool_master/build/classes/java/quarkus-generated-sources/avsc', not found
11:01:40  Compiling with JDK Java compiler API.
11:01:40  Class dependency analysis for incremental compilation took 0.038 secs.
11:02:19  Created classpath snapshot for incremental compilation in 36.463 secs.
11:02:19  :compileJava (Thread[Daemon worker,5,main]) completed. Took 37.71 secs.
11:02:19  :classes (Thread[Execution worker for ':',5,main]) started.
11:02:19  
11:02:19  > Task :classes
11:02:19  Skipping task ':classes' as it has no actions.
11:02:19  :classes (Thread[Execution worker for ':',5,main]) completed. Took 0.0 secs.
11:02:19  :vaadinBuildFrontend (Thread[Execution worker for ':',5,main]) started.
11:02:19  
11:02:19  > Task :vaadinBuildFrontend
11:02:19  Caching disabled for task ':vaadinBuildFrontend' because:
11:02:19    Build cache is disabled
11:02:19  Task ':vaadinBuildFrontend' is not up-to-date because:
11:02:19    Task has not declared any outputs despite executing actions.
11:02:19  Running the vaadinBuildFrontend task with effective configuration VaadinFlowPluginExtension(productionMode=false, webpackOutputDirectory=/home/jenkins/agent/workspace/fika-admin-tool_master/build/resources/main/META-INF/VAADIN/webapp, npmFolder=/home/jenkins/agent/workspace/fika-admin-tool_master, webpackTemplate='webpack.config.js', webpackGeneratedTemplate='webpack.generated.js', generatedFolder=/home/jenkins/agent/workspace/fika-admin-tool_master/build/frontend, frontendDirectory=/home/jenkins/agent/workspace/fika-admin-tool_master/frontend, generateBundle=true, runNpmInstall=true, generateEmbeddableWebComponents=true, frontendResourcesDirectory=/home/jenkins/agent/workspace/fika-admin-tool_master/src/main/resources/META-INF/resources/frontend, optimizeBundle=true, pnpmEnable=true, useGlobalPnpm=false, requireHomeNodeExec=false, useDeprecatedV14Bootstrapping=false, eagerServerLoad=false, applicationProperties=/home/jenkins/agent/workspace/fika-admin-tool_master/src/main/resources/application.properties, openApiJsonFile=/home/jenkins/agent/workspace/fika-admin-tool_master/build/generated-resources/openapi.json, javaSourceFolder=/home/jenkins/agent/workspace/fika-admin-tool_master/src/main/java, javaResourceFolder=/home/jenkins/agent/workspace/fika-admin-tool_master/src/main/resources, generatedTsFolder=/home/jenkins/agent/workspace/fika-admin-tool_master/frontend/generated, nodeVersion=v16.13.0, nodeDownloadRoot=https://nodejs.org/dist/, nodeAutoUpdate=falseresourceOutputDirectory=/home/jenkins/agent/workspace/fika-admin-tool_master/build/vaadin-generated)
11:02:19  Reflections took 2090 ms to scan 210 urls, producing 5956 keys and 31627 values
11:02:19  Passing this classpath to NodeTasks.Builder: [vaadin-core-jandex-22.0.2.jar, vaadin-core-22.0.2.jar, vaadin-quarkus-1.0.0.jar, quarkus-undertow-websockets-2.5.4.Final.jar, slf4j-simple-1.7.32.jar, fusion-endpoint-9.0.2.jar, vaadin-dev-server-9.0.2.jar, flow-lit-template-9.0.2.jar, flow-polymer-template-9.0.2.jar, flow-html-components-9.0.2.jar, flow-data-9.0.2.jar, flow-dnd-9.0.2.jar, flow-server-9.0.2.jar, flow-push-9.0.2.jar, flow-client-9.0.2.jar, vaadin-lumo-theme-22.0.2.jar, vaadin-material-theme-22.0.2.jar, vaadin-accordion-flow-22.0.2.jar, vaadin-avatar-flow-22.0.2.jar, vaadin-app-layout-flow-22.0.2.jar, vaadin-button-flow-22.0.2.jar, vaadin-checkbox-flow-22.0.2.jar, vaadin-combo-box-flow-22.0.2.jar, vaadin-date-time-picker-flow-22.0.2.jar, vaadin-date-picker-flow-22.0.2.jar, vaadin-time-picker-flow-22.0.2.jar, vaadin-select-flow-22.0.2.jar, vaadin-dialog-flow-22.0.2.jar, vaadin-details-flow-22.0.2.jar, vaadin-form-layout-flow-22.0.2.jar, vaadin-grid-flow-22.0.2.jar, vaadin-icons-flow-22.0.2.jar, vaadin-iron-list-flow-22.0.2.jar, vaadin-virtual-list-flow-22.0.2.jar, vaadin-list-box-flow-22.0.2.jar, vaadin-messages-flow-22.0.2.jar, vaadin-notification-flow-22.0.2.jar, vaadin-ordered-layout-flow-22.0.2.jar, vaadin-progress-bar-flow-22.0.2.jar, vaadin-radio-button-flow-22.0.2.jar, vaadin-menu-bar-flow-22.0.2.jar, vaadin-context-menu-flow-22.0.2.jar, vaadin-renderer-flow-22.0.2.jar, vaadin-split-layout-flow-22.0.2.jar, vaadin-tabs-flow-22.0.2.jar, vaadin-text-field-flow-22.0.2.jar, vaadin-upload-flow-22.0.2.jar, vaadin-login-flow-22.0.2.jar, vaadin-custom-field-flow-22.0.2.jar, ph-css-6.3.4.jar, quarkus-undertow-2.5.4.Final.jar, quarkus-websockets-2.5.4.Final.jar, quarkus-vertx-http-2.5.4.Final.jar, quarkus-websockets-client-2.5.4.Final.jar, quarkus-vertx-2.5.4.Final.jar, quarkus-mutiny-2.5.4.Final.jar, quarkus-smallrye-context-propagation-2.5.4.Final.jar, quarkus-netty-2.5.4.Final.jar, quarkus-arc-2.5.4.Final.jar, quarkus-security-runtime-spi-2.5.4.Final.jar, quarkus-vertx-http-dev-console-runtime-spi-2.5.4.Final.jar, quarkus-core-2.5.4.Final.jar, spring-data-commons-2.5.6.jar, swagger-codegen-generators-1.0.20.jar, swagger-codegen-3.0.20.jar, swagger-codegen-2.4.14.jar, swagger-parser-2.0.20.jar, swagger-parser-v2-converter-2.0.20.jar, swagger-compat-spec-parser-1.0.51.jar, swagger-parser-1.0.51.jar, slf4j-ext-1.7.32.jar, atmosphere-runtime-2.4.30.slf4jvaadin1.jar, ph-commons-10.1.2.jar, handlebars-4.1.2.jar, swagger-core-1.6.1.jar, swagger-models-1.6.1.jar, swagger-parser-v3-2.0.20.jar, swagger-core-2.1.2.jar, slf4j-api-1.7.32.jar, jackson-dataformat-yaml-2.12.6.jar, quarkus-http-vertx-backend-4.1.4.jar, quarkus-http-websocket-vertx-4.1.4.jar, smallrye-mutiny-vertx-web-2.15.1.jar, vertx-web-4.2.2.jar, smallrye-mutiny-vertx-web-common-2.15.1.jar, smallrye-mutiny-vertx-auth-common-2.15.1.jar, smallrye-mutiny-vertx-bridge-common-2.15.1.jar, smallrye-mutiny-vertx-core-2.15.1.jar, vertx-web-common-4.2.2.jar, vertx-auth-common-4.2.2.jar, vertx-bridge-common-4.2.2.jar, vertx-mutiny-generator-2.15.1.jar, smallrye-mutiny-vertx-runtime-2.15.1.jar, vertx-core-4.2.2.jar, vertx-codegen-4.2.2.jar, jackson-core-2.12.6.jar, jackson-datatype-jsr310-2.12.6.jar, json-patch-1.6.jar, json-schema-validator-2.2.8.jar, json-schema-core-1.2.8.jar, jackson-coreutils-1.8.jar, jackson-databind-2.12.6.jar, swagger-parser-core-2.0.20.jar, swagger-models-2.1.2.jar, jackson-annotations-2.12.6.jar, javaparser-symbol-solver-core-3.23.1.jar, uri-template-0.9.jar, guava-30.1.1-jre.jar, error_prone_annotations-2.2.0.jar, httpclient-4.5.13.jar, commons-codec-1.15.jar, commons-io-2.11.0.jar, quarkus-http-servlet-4.1.4.jar, quarkus-http-core-4.1.4.jar, quarkus-http-http-core-4.1.4.jar, quarkus-http-websocket-core-4.1.4.jar, netty-handler-proxy-4.1.72.Final.jar, netty-codec-http2-4.1.72.Final.jar, netty-codec-http-4.1.72.Final.jar, netty-codec-haproxy-4.1.72.Final.jar, netty-resolver-dns-4.1.72.Final.jar, netty-handler-4.1.72.Final.jar, netty-codec-socks-4.1.72.Final.jar, netty-codec-dns-4.1.72.Final.jar, netty-codec-4.1.72.Final.jar, netty-transport-4.1.72.Final.jar, netty-buffer-4.1.72.Final.jar, netty-resolver-4.1.72.Final.jar, netty-common-4.1.72.Final.jar, netty-tcnative-classes-2.0.46.Final.jar, arc-2.5.4.Final.jar, quarkus-security-1.1.4.Final.jar, quarkus-bootstrap-runner-2.5.4.Final.jar, quarkus-development-mode-spi-2.5.4.Final.jar, quarkus-ide-launcher-2.5.4.Final.jar, mutiny-smallrye-context-propagation-1.1.2.jar, mutiny-1.1.2.jar, smallrye-config-2.7.0.jar, smallrye-config-core-2.7.0.jar, smallrye-common-annotation-1.8.0.jar, smallrye-config-common-2.7.0.jar, smallrye-common-classloader-1.8.0.jar, smallrye-common-expression-1.8.0.jar, smallrye-common-function-1.8.0.jar, smallrye-common-constraint-1.8.0.jar, smallrye-common-io-1.8.0.jar, smallrye-context-propagation-1.2.2.jar, smallrye-context-propagation-api-1.2.2.jar, smallrye-fault-tolerance-vertx-5.2.1.jar, jakarta.xml.bind-api-2.3.3.jar, jakarta.activation-api-1.2.1.jar, jakarta.enterprise.cdi-api-2.0.2.jar, jakarta.interceptor-api-1.2.5.jar, jakarta.annotation-api-1.3.5.jar, jakarta.inject-api-1.0.jar, jakarta.servlet-api-4.0.3.jar, jakarta.transaction-api-1.3.3.jar, jakarta.validation-api-2.0.2.jar, jakarta.websocket-api-1.1.2.jar, commons-compress-1.21.jar, commons-lang3-3.12.0.jar, httpcore-4.4.14.jar, checker-qual-2.5.2.jar, microprofile-config-api-2.0.jar, microprofile-context-propagation-api-1.2.jar, graal-sdk-21.3.0.jar, jboss-logging-annotations-2.2.1.Final.jar, jboss-threads-3.4.2.Final.jar, jboss-logging-3.4.2.Final.jar, jboss-logmanager-embedded-1.0.9.jar, slf4j-jboss-logmanager-1.1.0.Final.jar, jsoup-1.14.2.jar, reactive-streams-1.0.3.jar, wildfly-common-1.5.4.Final-format-001.jar, snakeyaml-1.29.jar, javax.annotation-api-1.3.2.jar, gwt-elemental-2.8.2.vaadin2.jar, commons-fileupload-1.3.3.jar, byte-buddy-1.12.1.jar, gentyref-1.2.0.vaadin1.jar, validation-api-2.0.1.Final.jar, vaadin__vaadin-mobile-drag-drop-1.0.1.jar, mobile-drag-drop-2.3.0-rc.2.jar, commons-logging-1.2.jar, jakarta.el-api-3.0.3.jar, spring-beans-5.3.11.jar, spring-core-5.3.11.jar, javaparser-core-3.23.1.jar, javassist-3.28.0-GA.jar, jmustache-1.12.jar, commons-cli-1.2.jar, commonmark-0.9.0.jar, msg-simple-1.1.jar, btf-1.2.jar, jsr305-3.0.2.jar, spring-jcl-5.3.11.jar, failureaccess-1.0.1.jar, listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar, j2objc-annotations-1.3.jar, jboss-websocket-api_1.1_spec-1.1.4.Final.jar, smallrye-context-propagation-storage-1.2.2.jar, swagger-annotations-1.6.1.jar, swagger-annotations-2.1.2.jar, joda-time-2.9.7.jar, libphonenumber-8.0.0.jar, jopt-simple-5.0.3.jar, rhino-1.7R4.jar, mailapi-1.4.3.jar, activation-1.1.jar, /home/jenkins/agent/workspace/fika-admin-tool_master/build/classes/java/main]
11:02:19  Scanning classes to find frontend configurations and dependencies...
11:02:21  Visited 1938 classes. Took 3083 ms.
11:02:21  Visited 104 classes. Took 64 ms.
11:02:21  Running `pnpm install` to resolve and optionally download frontend dependencies. This may take a moment, please stand by...
11:02:22  using '/root/.vaadin/node/node /root/.vaadin/node/node_modules/npm/bin/npx-cli.js --yes --quiet pnpm@5.18.10' for frontend package installation
11:02:23  Command `/root/.vaadin/node/node /root/.vaadin/node/node_modules/npm/bin/npx-cli.js --yes --quiet pnpm@5.18.10 --shamefully-hoist=true install` failed:
11:02:23  
11:02:23  >>> Dependency ERROR. Check that all required dependencies are deployed in pnpm repositories.
11:02:23  
11:02:23  > Task :vaadinBuildFrontend FAILED
11:02:23  :vaadinBuildFrontend (Thread[Execution worker for ':',5,main]) completed. Took 7.32 secs.
11:02:23  
11:02:23  FAILURE: Build failed with an exception.

Even though the build machine provides nodes+npm, the Gradle plugin appears to install its own copes in the .vaadin directory. I did a successful build on my Windows machine so the issue appears to be specific to our build agent (a Docker agent running on AWS EKS).

luc-rovio commented 2 years ago

I logged onto the build agent. The build was able to create /root/.vaadin, and disk space appears to be sufficient (15 GB for the root partition): Filesystem 1K-blocks Used Available Use% Mounted on overlay 20959212 5703088 15256124 28% / tmpfs 65536 0 65536 0% /dev tmpfs 3967232 0 3967232 0% /sys/fs/cgroup /dev/nvme0n1p1 20959212 5703088 15256124 28% /etc/hosts shm 65536 0 65536 0% /dev/shm tmpfs 3967232 12 3967220 1% /run/secrets/kubernetes.io/serviceaccount tmpfs 3967232 0 3967232 0% /proc/acpi tmpfs 3967232 0 3967232 0% /sys/firmware

luc-rovio commented 2 years ago

This is not new: https://vaadin.com/forum/thread/18159870/jenkins-build-fails-with-vaadin-gradle-plugin-due-to-missing-dependency

mvysny commented 2 years ago

Can you please try to run the command /root/.vaadin/node/node /root/.vaadin/node/node_modules/npm/bin/npx-cli.js --yes --quiet pnpm@5.18.10 --shamefully-hoist=true install manually in your environment? Looks like Gradle plugin doesn't properly read stderr of the process and therefore doesn't print the error message (that's definitely a bug in the Vaadin Gradle plugin and should be filed at https://github.com/vaadin/flow/issues )

luc-rovio commented 2 years ago 
sh-4.2# /root/.vaadin/node/node /root/.vaadin/node/node_modules/npm/bin/npx-cli.js --yes --quiet pnpm@5.18.10 --shamefully-hoist=true install
Already up-to-date

   ╭──────────────────────────────────────────────────────────────────╮
   │                                                                  │
   │                Update available! 6.11.0 → 6.32.3.                │
   │   Changelog: https://github.com/pnpm/pnpm/releases/tag/v6.32.3   │
   │                 Run pnpm add -g pnpm to update.                  │
   │                                                                  │
   │      Follow @pnpmjs for updates: https://twitter.com/pnpmjs      │
   │                                                                  │
   ╰──────────────────────────────────────────────────────────────────╯
luc-rovio commented 2 years ago

I also tried to set HOME as a workaround suggested in https://github.com/vaadin/vaadin-gradle-plugin/issues/54 but it didn't work

mvysny commented 2 years ago

The command must be run in the folder of The vaadin project (or where the package.json file is)

luc-rovio commented 2 years ago

@mvysny sorry, here it is:

14:07:18  [Pipeline] sh
14:07:18  + /root/.vaadin/node/node /root/.vaadin/node/node_modules/npm/bin/npx-cli.js --yes --quiet pnpm@5.18.10 --shamefully-hoist=true install
14:07:19  sh: pnpm: command not found

it is starting to look like a pnpm/npx issue such as this: https://github.com/pnpm/pnpm/issues/2873

luc-rovio commented 2 years ago

I installed pnpm on the docker agent and set pnpm.global to true, but the build fails to detect the installation:

10:37:52    ======================================================================================================
10:37:52    Vaadin application is configured to use globally installed pnpm ('pnpm.global=true'), but no pnpm tool has been found on your system.
10:37:52    Please install pnpm tool following the instruction given here https://pnpm.io/installation, 
10:37:52    or let Vaadin use the default pnpm version by excluding 'pnpm.global' (or setting it to false) from configuration.
10:37:52    ======================================================================================================

When logged in on the agent, I'm able to launch pnpm successfully.

luc-rovio commented 2 years ago

It turns out the plugin is using the which command (https://github.com/vaadin/flow/blob/77d281cce70085a362d238ea413f99f967c69d60/flow-server/src/main/java/com/vaadin/flow/server/frontend/FrontendToolsLocator.java#L78) and the Amazon Coretto Docker image does not include this. After installing which I was able to build successfully.

luc-rovio commented 2 years ago

In conclusion: