cloudfoundry / java-buildpack

Cloud Foundry buildpack for running Java applications
Apache License 2.0
435 stars 2.58k forks source link

Update zing_jre.rb #1005

Closed ushaazul closed 1 year ago

ushaazul commented 1 year ago

Fix to address : ERR bash: -Djava.io.tmpdir=temp_dir: No such file or directory

pivotal-david-osullivan commented 1 year ago

It looks like this change would disable the logic here as well as disabling the flag that replaces the function of the JVMKill Agent which was not added to this JRE implementation.

The error in your description indicates an issue with the temp directory which, as per the link above, is set to the value of $TMPDIR. Can you share more details about the problem you are seeing?

ushaazul commented 1 year ago

Hi David, Adding some context here When we try to run with Zing with PCF, we get the following error ERR undefined method add_preformatted_options’ for #<JavaBuildpack::Component::droplet:0x0000557874531a28> The proposal from VMWare was to use @droplet.java_opts.add_preformatted_options ‘-XX:+ExitOnOutOfMemoryError’ instead of @droplet.add_preformatted_options ‘-XX:+ExitOnOutOfMemoryError’ but this leads to error ERR bash: -Djava.io.tmpdir=temp_dir: No such file or directory So the solution from Vmware was to comment out the following two lines of code

super

@droplet.add_preformatted_options '-XX:+ExitOnOutOfMemoryError'

pivotal-david-osullivan commented 1 year ago

The change to @droplet.java_opts.add_preformatted_options ‘-XX:+ExitOnOutOfMemoryError’ is correct and I think this will fix the core bug here.

I expect the error ERR bash: -Djava.io.tmpdir=temp_dir: No such file or directory is unrelated - as mentioned above the default for this, set here is $TMPDIR and on a cloudfoundry container this should be /home/vcap/tmp. Something is possibly overriding this environment variable to temp_dir as per the value in the error.

You could test this by setting JAVA_OPTS manually, e.g. cf set-env <app> JAVA_OPTS '-Djava.io.tmpdir=/tmp' (or /home/vcap/tmp), this should override the JVM flag to bypass any other values. Full logs for this error scenario might help if you cannot make progress.

ushaazul commented 1 year ago

Here is the error log 2023-03-24T10:59:07.44-0400 [CELL/0] OUT Starting health monitoring of container 2023-03-24T10:59:07.72-0400 [APP/PROC/WEB/0] ERR bash: -Djava.io.tmpdir=/home/vcap/tmp: No such file or directory I think that the error is thrown not because there is no /home/vcap/tmp, but because the buildpack interprets token “-Djava.io.tmpdir=/home/vcap/tmp” as a separate shell command. Pls correct me if I am wrong.

schelini commented 1 year ago

This is the error after changing to @droplet.java_opts.add_preformatted_options ‘-XX:+ExitOnOutOfMemoryError’

   2023-03-29T13:04:39.62-0400 [CELL/0] OUT Cell da211db8-8722-433e-abc2-c331b868c74d successfully created container for instance de21b026-899f-4908-5221-1bfd
   2023-03-29T13:04:40.24-0400 [CELL/0] OUT Downloading droplet...
   2023-03-29T13:04:41.80-0400 [CELL/0] OUT Downloaded droplet
   2023-03-29T13:04:41.80-0400 [CELL/0] OUT Starting health monitoring of container
   2023-03-29T13:04:42.17-0400 [APP/PROC/WEB/0] ERR bash: -Djava.io.tmpdir=/home/vcap/tmp: No such file or directory
   2023-03-29T13:04:42.18-0400 [APP/PROC/WEB/0] OUT Exit status 127

It seems to mess up the default start command and tries to run the java options as commands after setting the JAVA_OPTS variable (fails on the first one at && -Djava.io.tmpdir=$TMPDIR)

start command:   JAVA_OPTS="-Djava.io.tmpdir=$TMPDIR -XX:ActiveProcessorCount=$(nproc) -XX:+ExitOnOutOfMemoryError
                 -Djava.ext.dirs= -Djava.security.properties=$PWD/.java-buildpack/java_security/java.security
                 $JAVA_OPTS" && -Djava.io.tmpdir=$TMPDIR && -XX:ActiveProcessorCount=$(nproc) &&
                 -XX:+ExitOnOutOfMemoryError && -Djava.ext.dirs= &&
                 -Djava.security.properties=$PWD/.java-buildpack/java_security/java.security && $JAVA_OPTS &&
                 SERVER_PORT=$PORT eval exec $PWD/.java-buildpack/zing_jre/bin/java $JAVA_OPTS -cp
                 $PWD/.:$PWD/.java-buildpack/container_security_provider/container_security_provider-1.19.0_RELEASE.jar
                 org.springframework.boot.loader.JarLauncher

I think the expected command would look something like this

start command:   JAVA_OPTS="-Djava.io.tmpdir=$TMPDIR -XX:ActiveProcessorCount=$(nproc) -XX:+ExitOnOutOfMemoryError
                 -Djava.ext.dirs= -Djava.security.properties=$PWD/.java-buildpack/java_security/java.security
                 $JAVA_OPTS" &&
                 SERVER_PORT=$PORT eval exec $PWD/.java-buildpack/zing_jre/bin/java $JAVA_OPTS -cp
                 $PWD/.:$PWD/.java-buildpack/container_security_provider/container_security_provider-1.19.0_RELEASE.jar
                 org.springframework.boot.loader.JarLauncher

Commenting out @droplet.java_opts.add_preformatted_options ‘-XX:+ExitOnOutOfMemoryError’ was just a temp workaround

ushaazul commented 1 year ago

cftest@zen:~$ cf set-env zingapp JAVA_OPTS ‘-[Djava.io](http://djava.io/).tmpdir=/tmp’
Setting env variable JAVA_OPTS for app zingapp in org cloudfoundry / space development as admin...
OK
TIP: Use ‘cf restage zingapp’ to ensure your env variable changes take effect.
cftest@zen:~$ cf restage zingapp

Info from logs
`2023-03-27T19:12:03.22+0300 [CELL/0] OUT Downloaded droplet
   2023-03-27T19:12:04.80+0300 [APP/PROC/WEB/0] ERR bash: -Djava.io.tmpdir=/home/vcap/tmp: No such file or directory
   2023-03-27T19:12:04.99+0300 [APP/PROC/WEB/0] OUT Exit status 127
   2023-03-27T19:12:05.04+0300 [CELL/SSHD/0] OUT Exit status 0
   2023-03-27T19:12:12.17+0300 [CELL/0] OUT Cell 2d92cd2b-6170-4b9f-af3e-664be1a65770 stopping instance f75783e9-bcdf-4a18-5288-08d9
   2023-03-27T19:12:12.17+0300 [CELL/0] OUT Cell 2d92cd2b-6170-4b9f-af3e-664be1a65770 destroying container for instance f75783e9-bcdf-4a18-5288-08d9
   2023-03-27T19:12:12.35+0300 [API/0] OUT Process has crashed with type: "web"
   2023-03-27T19:12:12.60+0300 [CELL/0] OUT Cell 2d92cd2b-6170-4b9f-af3e-664be1a65770 creating container for instance f0df4621-95f4-43a8-577e-dc38
   2023-03-27T19:12:12.66+0300 [API/0] OUT App instance exited with guid 7be731f2-98be-4733-b9e2-568ee90c7aec payload: {"instance"=>"f75783e9-bcdf-4a18-5288-08d9", "index"=>0, "cell_id"=>"2d92cd2b-6170-4b9f-af3e-664be1a65770", "reason"=>"CRASHED", "exit_description"=>"APP/PROC/WEB: Exited with status 127", "crash_count"=>2, "crash_timestamp"=>1679933531939882206, "version"=>"af47956a-8cc4-4d09-8f77-56a081d33225"}
   2023-03-27T19:12:13.60+0300 [PROXY/0] OUT Exit status 137`
ushaazul commented 1 year ago

Thanks Samuel for making the change - https://github.com/cloudfoundry/java-buildpack/pull/1008