quarkus-bot[bot]
commented
11 months ago /cc @geoand (openshift), @iocanel (openshift)
Open l0rd opened 11 months ago
quarkus-bot[bot]
commented
11 months ago /cc @geoand (openshift), @iocanel (openshift)
gsmet
commented
11 months ago @radcortez I suspect it has something to do with config changes and maybe some specific environment variables set in the OpenShift developer sandbox? Maybe some additional logging would help, especially the current variable being analyzed?
l0rd
commented
11 months ago quarkus-dev-services-ide (main) $ env
NVM_DIR=/home/user/.nvm
SECONDS_OF_DW_RUN_BEFORE_IDLING=-1
HTTPD_DATA_ORIG_PATH=/var/www
PHP_SYSCONF_PATH=/etc
WORKSPACEFDBC628CB53F4750_SERVICE_PORT_13133_TCP_PORT=13133
WORKSPACEFDBC628CB53F4750_SERVICE_PORT_13132_TCP_PROTO=tcp
MODULES_RUN_QUARANTINE=LD_LIBRARY_PATH LD_PRELOAD
LANG=C.utf8
NODEJS_HOME_18=/home/user/.nvm/versions/node/v18.16.1
WORKSPACEFDBC628CB53F4750_SERVICE_PORT_13131_TCP_ADDR=172.30.73.12
DEVWORKSPACE_CREATOR=c7b5315e-0f91-4586-b5e6-1b6a7e353f88
HISTCONTROL=ignoredups
HTTPD_DATA_PATH=/var/www
HOSTNAME=workspacefdbc628cb53f4750-8567df4b8b-jpf44
OLDPWD=/projects
SDKMAN_CANDIDATES_API=https://api.sdkman.io/2
KUBECONFIG=/home/user/.kube/config
MAVEN_HOME=/home/user/.sdkman/candidates/maven/current
RUSTUP_HOME=/home/user/.rustup
CHE_PLUGIN_REGISTRY_INTERNAL_URL=http://plugin-registry.crw.svc:8080/v3
COLORTERM=truecolor
NVM_CD_FLAGS=
WORKSPACEFDBC628CB53F4750_SERVICE_PORT_13131_TCP_PORT=13131
JAVA_HOME=/home/user/.sdkman/candidates/java/17.0.3-tem
KUBEDOCK_VERSION=0.13.0
DEVWORKSPACE_NAME=quarkus-awesome
VSCODE_GIT_ASKPASS_EXTRA_ARGS=
PHP_HTTPD_CONF_FILE=php.conf
DEVWORKSPACE_TELEMETRY_BACKEND_PORT=4167
WORKSPACEFDBC628CB53F4750_SERVICE_PORT_13131_TCP_PROTO=tcp
KAMEL_VERSION=1.11.0
HTTPD_VAR_RUN=/var/run/httpd
DOTNET_ROOT=/usr/lib64/dotnet
CHE_PLUGIN_REGISTRY_URL=https://devspaces.apps.sandbox-m2.ll9k.p1.openshiftapps.com/plugin-registry/v3
which_declare=declare -f
KUBERNETES_PORT_443_TCP_PROTO=tcp
KUBERNETES_PORT_443_TCP_ADDR=172.30.0.1
container=oci
CHE_DASHBOARD_URL=https://devspaces.apps.sandbox-m2.ll9k.p1.openshiftapps.com
MODULES_CMD=/usr/share/Modules/libexec/modulecmd.tcl
DOTNET_BUNDLE_EXTRACT_BASE_DIR=/home/user/.cache/dotnet_bundle_extract
USER=user
MACHINE_EXEC_PORT=3333
JBANG_HOME=/home/user/.sdkman/candidates/jbang/current
KUBERNETES_PORT=tcp://172.30.0.1:443
GRADLE_HOME=/home/user/.sdkman/candidates/gradle/current
DEVWORKSPACE_METADATA=/devworkspace-metadata
PWD=/projects/quarkus-dev-services-ide
SECONDS_OF_DW_INACTIVITY_BEFORE_IDLING=1800
HOME=/home/user
WORKSPACEFDBC628CB53F4750_SERVICE_PORT_13133_TCP_ADDR=172.30.73.12
BROWSER=/checode/checode-linux-libc/bin/helpers/browser.sh
VSCODE_GIT_ASKPASS_NODE=/checode/checode-linux-libc/node
OPENVSX_REGISTRY_URL=https://open-vsx.org
NODEJS_HOME_20=/home/user/.nvm/versions/node/v20.7.0
TERM_PROGRAM=vscode
TERM_PROGRAM_VERSION=1.81.0
HTTPD_VAR_PATH=/var
JAVA_HOME_8=/home/user/.sdkman/candidates/java/8.0.332-tem
KUBERNETES_SERVICE_PORT_HTTPS=443
WORKSPACEFDBC628CB53F4750_SERVICE_SERVICE_PORT_CODE_REDIRECT_1=13131
WORKSPACEFDBC628CB53F4750_SERVICE_SERVICE_PORT_CODE_REDIRECT_3=13133
WORKSPACEFDBC628CB53F4750_SERVICE_SERVICE_PORT_CODE_REDIRECT_2=13132
DEVWORKSPACE_NAMESPACE=mloriedo-dev
LOMBOK_VERSION=1.18.18
OC_VERSION=4.6
WORKSPACEFDBC628CB53F4750_SERVICE_PORT_3030_TCP_PROTO=tcp
KUBERNETES_PORT_443_TCP_PORT=443
TESTCONTAINERS_RYUK_DISABLED=true
QUARKUS_DATASOURCE_DEVSERVICES_VOLUMES_=/var/lib/postgresql/
PROJECT_SOURCE=/projects/quarkus-dev-services-ide
BUILDAH_ISOLATION=chroot
VSCODE_IPC_HOOK_CLI=/tmp/vscode-ipc-7f838a16-b1c0-4ee1-841a-6917040ea778.sock
NODEJS_18_VERSION=18.16.1
CARGO_HOME=/home/user/.cargo
WORKSPACEFDBC628CB53F4750_SERVICE_PORT=tcp://172.30.73.12:3030
WORKSPACEFDBC628CB53F4750_SERVICE_SERVICE_PORT=3030
LOADEDMODULES=
SDKMAN_DIR=/home/user/.sdkman
KUBERNETES_PORT_443_TCP=tcp://172.30.0.1:443
WORKSPACEFDBC628CB53F4750_SERVICE_PORT_13132_TCP_ADDR=172.30.73.12
HTTPD_MODULES_CONF_D_PATH=/etc/httpd/conf.modules.d
MAIL=/var/spool/mail/user
SDKMAN_OLD_PWD=/projects/quarkus-dev-services-ide
NODEJS_20_VERSION=20.7.0
VSCODE_GIT_ASKPASS_MAIN=/checode/checode-linux-libc/extensions/git/dist/askpass-main.js
CLUSTER_CONSOLE_URL=https://console-openshift-console.apps.sandbox-m2.ll9k.p1.openshiftapps.com
WORKSPACEFDBC628CB53F4750_SERVICE_PORT_13132_TCP=tcp://172.30.73.12:13132
TERM=xterm-256color
DOCKER_HOST=tcp://127.0.0.1:2475
GRAALVM_HOME=/home/user/.sdkman/candidates/java/22.1.0.0.r17-mandrel
WORKSPACEFDBC628CB53F4750_SERVICE_PORT_13132_TCP_PORT=13132
WORKSPACEFDBC628CB53F4750_SERVICE_PORT_13133_TCP_PROTO=tcp
NSS_SDB_USE_CACHE=no
NVM_BIN=/home/user/.nvm/versions/node/v18.16.1/bin
SDKMAN_CANDIDATES_DIR=/home/user/.sdkman/candidates
_BUILDAH_STARTED_IN_USERNS=
PHP_VERSION=7.4
WORKSPACE_NAME=quarkus-awesome
CLUSTER_CONSOLE_TITLE=OpenShift console
WORKSPACEFDBC628CB53F4750_SERVICE_SERVICE_HOST=172.30.73.12
KUBEDOCK_ENABLED=true
SHLVL=4
VSCODE_GIT_IPC_HANDLE=/tmp/vscode-git-524c43a6db.sock
DASHBOARD_URL=https://devspaces.apps.sandbox-m2.ll9k.p1.openshiftapps.com
MANPATH=/home/user/.nvm/versions/node/v18.16.1/share/man::
KUBERNETES_SERVICE_PORT=443
TESTCONTAINERS_CHECKS_DISABLE=true
MODULEPATH=/etc/scl/modulefiles:/etc/scl/modulefiles:/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/share/modulefiles
WORKSPACE_NAMESPACE=mloriedo-dev
JAVA_HOME_17=/home/user/.sdkman/candidates/java/17.0.3-tem
JAVA_HOME_11=/home/user/.sdkman/candidates/java/11.0.15-tem
DEVWORKSPACE_FLATTENED_DEVFILE=/devworkspace-metadata/flattened.devworkspace.yaml
LOGNAME=user
DEVWORKSPACE_ORIGINAL_DEVFILE=/devworkspace-metadata/original.devworkspace.yaml
GIT_ASKPASS=/checode/checode-linux-libc/extensions/git/dist/askpass.sh
DOTNET_RPM_VERSION=6.0
MODULEPATH_modshare=/usr/share/Modules/modulefiles:2:/etc/modulefiles:2:/usr/share/modulefiles:2
WORKSPACEFDBC628CB53F4750_SERVICE_PORT_13133_TCP=tcp://172.30.73.12:13133
GOBIN=/home/user/go/bin/
PROJECTS_ROOT=/projects
NODEJS_DEFAULT_VERSION=18.16.1
DEVWORKSPACE_ID=workspacefdbc628cb53f4750
WORKSPACEFDBC628CB53F4750_SERVICE_PORT_3030_TCP_PORT=3030
PATH=/home/user/.krew/bin:/checode/checode-linux-libc/bin/remote-cli:/home/user/.krew/bin:/home/user/.sdkman/candidates/jbang/current/bin:/home/user/.local/bin:/home/user/bin:/usr/share/Modules/bin:/home/user/.cargo/bin:/home/user/go/bin/:/home/user/.local/bin:/home/user/.nvm/versions/node/v18.16.1/bin:/home/user/.local/share/coursier/bin:/home/user/.sdkman/candidates/gradle/current/bin:/home/user/.sdkman/candidates/java/17.0.3-tem/bin:/home/user/.sdkman/candidates/maven/current/bin:/home/user/.krew/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/user/.dotnet/tools
HTTPD_MAIN_CONF_PATH=/etc/httpd/conf
SDKMAN_VERSION=5.13.0
PS1=\[\]\W `git branch --show-current 2>/dev/null | sed -r -e "s@^(.+)@\(\1\) @"`$ \[\]
WORKSPACEFDBC628CB53F4750_SERVICE_PORT_3030_TCP_ADDR=172.30.73.12
DEVWORKSPACE_IDLE_TIMEOUT=15m
WORKSPACEFDBC628CB53F4750_SERVICE_PORT_3030_TCP=tcp://172.30.73.12:3030
MODULESHOME=/usr/share/Modules
NODE_EXTRA_CA_CERTS=/tmp/node-extra-certificates/ca.crt
NVM_INC=/home/user/.nvm/versions/node/v18.16.1/include/node
PHP_DEFAULT_INCLUDE_PATH=/usr/share/pear
HISTSIZE=1000
KUBERNETES_SERVICE_HOST=172.30.0.1
SDKMAN_PLATFORM=linuxx64
WORKSPACEFDBC628CB53F4750_SERVICE_SERVICE_PORT_WS_ROUTE=3030
DEVWORKSPACE_COMPONENT_NAME=tooling-container
WORKSPACEFDBC628CB53F4750_SERVICE_PORT_13131_TCP=tcp://172.30.73.12:13131
LESSOPEN=||/usr/bin/lesspipe.sh %s
HTTPD_MAIN_CONF_D_PATH=/etc/httpd/conf.d
BASH_FUNC_which%%=() { ( alias;
eval ${which_declare} ) | /usr/bin/which --tty-only --read-alias --read-functions --show-tilde --show-dot $@
}
BASH_FUNC_module%%=() { _module_raw "$@" 2>&1
}
BASH_FUNC__module_raw%%=() { unset _mlshdbg;
if [ "${MODULES_SILENT_SHELL_DEBUG:-0}" = '1' ]; then
case "$-" in
*v*x*)
set +vx;
_mlshdbg='vx'
;;
*v*)
set +v;
_mlshdbg='v'
;;
*x*)
set +x;
_mlshdbg='x'
;;
*)
_mlshdbg=''
;;
esac;
fi;
unset _mlre _mlIFS;
if [ -n "${IFS+x}" ]; then
_mlIFS=$IFS;
fi;
IFS=' ';
for _mlv in ${MODULES_RUN_QUARANTINE:-};
do
if [ "${_mlv}" = "${_mlv##*[!A-Za-z0-9_]}" -a "${_mlv}" = "${_mlv#[0-9]}" ]; then
if [ -n "`eval 'echo ${'$_mlv'+x}'`" ]; then
_mlre="${_mlre:-}${_mlv}_modquar='`eval 'echo ${'$_mlv'}'`' ";
fi;
_mlrv="MODULES_RUNENV_${_mlv}";
_mlre="${_mlre:-}${_mlv}='`eval 'echo ${'$_mlrv':-}'`' ";
fi;
done;
if [ -n "${_mlre:-}" ]; then
eval `eval ${_mlre} /usr/bin/tclsh /usr/share/Modules/libexec/modulecmd.tcl bash '"$@"'`;
else
eval `/usr/bin/tclsh /usr/share/Modules/libexec/modulecmd.tcl bash "$@"`;
fi;
_mlstatus=$?;
if [ -n "${_mlIFS+x}" ]; then
IFS=$_mlIFS;
else
unset IFS;
fi;
unset _mlre _mlv _mlrv _mlIFS;
if [ -n "${_mlshdbg:-}" ]; then
set -$_mlshdbg;
fi;
unset _mlshdbg;
return $_mlstatus
}
BASH_FUNC_switchml%%=() { typeset swfound=1;
if [ "${MODULES_USE_COMPAT_VERSION:-0}" = '1' ]; then
typeset swname='main';
if [ -e /usr/share/Modules/libexec/modulecmd.tcl ]; then
typeset swfound=0;
unset MODULES_USE_COMPAT_VERSION;
fi;
else
typeset swname='compatibility';
if [ -e /usr/share/Modules/libexec/modulecmd-compat ]; then
typeset swfound=0;
MODULES_USE_COMPAT_VERSION=1;
export MODULES_USE_COMPAT_VERSION;
fi;
fi;
if [ $swfound -eq 0 ]; then
echo "Switching to Modules $swname version";
source /usr/share/Modules/init/bash;
else
echo "Cannot switch to Modules $swname version, command not found";
return 1;
fi
}
BASH_FUNC_scl%%=() { if [ "$1" = "load" -o "$1" = "unload" ]; then
eval "module $@";
else
/usr/bin/scl "$@";
fi
}
BASH_FUNC_ml%%=() { module ml "$@"
}
_=/usr/bin/env
quarkus-dev-services-ide (main) $ 
radcortez
commented
11 months ago Ok, let me have a look.
l0rd
commented
11 months ago Note that the build is happening in a container and kubedock is used under the hood for testcontainers to work. Although from the error logs the 2 things seams unrelated it came to my mind that this could be an important detail to share.
radcortez
commented
11 months ago The issue here is with an invalid env var QUARKUS_DATASOURCE_DEVSERVICES_VOLUMES_. Please, remove the env var or set it correctly.
I've fixed SmallRye Config in https://github.com/smallrye/smallrye-config/pull/1023 to handle these cases better, but we also need some code in Quarkus to report properties like these as unknown. Our current code is skipping these cases.
l0rd
commented
11 months ago @radcortez if I remove QUARKUS_DATASOURCE_DEVSERVICES_VOLUMES_ tests are failing:
java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
[error]: Build step io.quarkus.datasource.deployment.devservices.DevServicesDatasourceProcessor#launchDatabases threw an exception: java.lang.RuntimeException: org.testcontainers.containers.ContainerLaunchException: Container startup failed for image docker.io/postgres:14The reason is that /var/lib/postgresql/ is read-only for unprivileged users in image docker.io/postgres:14 and the container fails to start. When setting QUARKUS_DATASOURCE_DEVSERVICES_VOLUMES_=/var/lib/postgresql/ the folder is mounted as a volume and the container starts successfully. So I cannot remove the env variable. How can I set it correctly?
radcortez
commented
11 months ago We change the structure of the datasources configuration and it seems the old one was less restrictive. While it shouldn't be possible to set a property like QUARKUS_DATASOURCE_DEVSERVICES_VOLUMES_, it was being accepted as an empty key, which is not intended.
Please, refer to the following documentation to set the var correctly: https://quarkus.io/guides/databases-dev-services#mapping-volumes-into-dev-services-for-database
l0rd
commented
11 months ago @radcortez thank you, it worked adding the following 2 lines in application.properties:
quarkus.datasource.db-kind=postgresql
quarkus.datasource.devservices.volumes."/"=/var/lib/postgresql/Is it possible to use an environment variable as an alternative to the property file?
radcortez
commented
11 months ago In this case, no, because if the key path contains '/', these cannot be represented in an environment variable name.
@yrodiere what do you think?
yrodiere
commented
11 months ago @yrodiere what do you think?
I'm not sure we can do better than detect these cases and report them as error... Given the lossy nature of the representation of property keys as environment variable names, it's woefully unsuitable for Map configuration.
In fact, when using environment variables to set the values of a configuration Map, I'd say any attempt from the Java code to iterate through the keys/entries of that Map should result in an error, because we simply don't know what the keys are exactly. At best that Map can implement Map#get, but no more.
yrodiere
commented
11 months ago I suppose I should rephrase:
when using environment variables to set the values of a configuration
Mapand theMapkeys contain underscores
radcortez
commented
11 months ago Maybe we need to change the mapping to accept a List<String> instead?
yrodiere
commented
11 months ago Right, that would be the solution for this configuration property and this extension. Some key-value syntax, e.g. /=/var/lib/postgresql/, would work better. Can't use : as the separator, I'm afraid, because of the classpath: syntax :/
Doesn't change the problem for all other uses of Map in all other extensions though.
radcortez
commented
11 months ago Would it be possible like this:
yrodiere
commented
11 months ago Would it be possible like this:
* "foo:bar" (filesystem) * "classpath:foo:bar" (classpath)
Would be nice, but I raise to you:
classpath:foo:z (classpath directory on the FS, with z flag for cgroups to stop being insufferable)Not sure the z flag is currently supported, but that's definitely something we should plan for.
radcortez
commented
11 months ago Yes., I realize that classpath could also be a filesystem directory. We would have that limitation, where you can't have a filesystem directory named classpath :)
Anyway, I'll let you make a decision on what we should do here :)
l0rd
commented
11 months ago From my point of view the main problem here is that plain volumes are not supported, only bind mounts. See the difference between volumes and bind mounts in docker documentation. This is problematic because when running on Kubernetes I usually cannot bind mount a folder of the host machine in a Pod container. And the property is named quarkus.datasource.devservices.volumes when that's about bind mounts. Supporting plain volumes (without the path-from key) would also address problems of key/value encoding in the environment variable.
yrodiere
commented
11 months ago Supporting plain volumes (without the path-from key) would also address problems of key/value encoding in the environment variable.
That would be nice as well, but IMO orthogonal: the fact that one can't bind-mount through environment variables can be considered a bug, independently from the fact that plain volumes aren't supported yet.
Feel free to open a corresponding feature/enhancement request (or a PR!), unless there's already one somewhere. We'll need to discuss a few things, in particular we need to check whether TestContainers supports plain volumes.
l0rd
commented
11 months ago That would be nice as well, but IMO orthogonal: the fact that one can't bind can't bind-mount through environment variables can be considered a bug, independently from the fact that plain volumes aren't supported yet.
I agree. Different issues.
Feel free to open a corresponding feature/enhancement request (or a PR!), unless there's already one somewhere. We'll need to discuss a few things, in particular we need to check whether TestContainers supports plain volumes.
Yes, that might be a TestContainers issue.
Anyway I have a workaround now. I can keep this issue open until setting quarkus.datasource.devservices.volumes via an env variable is possible again (or at least the exception message is improved). Or close it if you won't fix it.
yrodiere
commented
10 months ago Would it be possible like this:
* "foo:bar" (filesystem) * "classpath:foo:bar" (classpath)Would be nice, but I raise to you:
* `classpath:foo:z` (`classpath` directory on the FS, with `z` flag for cgroups to stop being insufferable)Not sure the
zflag is currently supported, but that's definitely something we should plan for.
I wonder if a more structured syntax like docker-compose's syntax wouldn't be a better alternative... ?
E.g. something like the snippet below:
quarkus.datasource.devservices.volumes."foo".source=/path/from
quarkus.datasource.devservices.volumes."foo".target=/container/toOf course it looks verbose, but if you think about it, it makes sense to configure these separately: you could set the target in your src/main/resources/application.properties, but you'll very likely want to let each developer set source to something that makes sense on their particular machine.
Also, it would look less verbose in YAML...
quarkus:
datasource:
devservices:
volumes:
foo:
source: /path/from
target: /container/toOne benefit of this syntax is that it could very easily allow more complicated setups in the future:
quarkus.datasource.devservices.volumes."foo".type=bind
quarkus.datasource.devservices.volumes."foo".source=/path/from
quarkus.datasource.devservices.volumes."foo".target=/container/to
Describe the bug
NOTE: I am reproducing this error on OpenShift Developer Sandbox IDE (Dev Spaces). I cannot reproduce it on my laptop nor on GitHub Codespaces. I cannot tell if it's a Dev Spaces or Quarkus issue. All I know is that it works with Quarkus
3.2.6.Finaland fails with later versions.Trying to run
mvn testusing this sample application fails with the following stack trace if the Quarkus version is 3.3.0 or higher (it works if it's 3.2.6.Final or earlier):Expected behavior
mvn testworks successfully for latest Quarkus as it does for version3.2.6.Final.Actual behavior
mvn testfails with Quarkus versions 3.3.0 to 3.4.1 but works with earlier versions.How to Reproduce?
Open the workspace on Developer Sandbox, open a terminal and type
mvn test. See it fail. Now changequarkus.platform.versionproperty inpom.xmlto3.2.6.Finaland see it work.https://github.com/quarkusio/quarkus/assets/606959/7471e639-c6ba-4892-bfa8-d50a308b9e00
Output of
uname -aorverLinux workspacef70e503f05f442c4-6748f7c8f7-lwmr9 5.14.0-284.32.1.el9_2.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Sep 6 08:21:33 EDT 2023 x86_64 x86_64 x86_64 GNU/Linux
Output of
java -versionopenjdk version "17.0.3" 2022-04-19 OpenJDK Runtime Environment Temurin-17.0.3+7 (build 17.0.3+7) OpenJDK 64-Bit Server VM Temurin-17.0.3+7 (build 17.0.3+7, mixed mode, sharing)
GraalVM version (if different from Java)
No response
Quarkus version or git rev
3.2.6.Final
Build tool (ie. output of
mvnw --versionorgradlew --version)Apache Maven 3.9.4 (dfbb324ad4a7c8fb0bf182e6d91b0ae20e3d2dd9) Maven home: /home/user/.sdkman/candidates/maven/current Java version: 17.0.3, vendor: Eclipse Adoptium, runtime: /home/user/.sdkman/candidates/java/17.0.3-tem Default locale: en, platform encoding: UTF-8 OS name: "linux", version: "5.14.0-284.32.1.el9_2.x86_64", arch: "amd64", family: "unix"
Additional information
No response