/cnb/process/healthcheck triggers start of JVM

[ofirm93]

I'm using health-checker buildpack in my springboot application. I'm using it in conjunction with the following other buildpacks:

            'paketo-buildpacks/java',
            'gcr.io/paketo-buildpacks/opentelemetry',

and building using the springboot gradle plugin.

In my docker-compose I defined the following:

    environment:
      THC_PATH: /actuator/health
    healthcheck:
      test: "/cnb/process/health-check"

Then I start the build container using docker-compose.

Expected Behavior

I run docker inspect on the container, and expect to see in the health block records of thc run which should not return any output to STDOUT and just return error code 0.

Current Behavior

When I run docker inspect under the Health section I see the following:

"Health" : {
      "Status" : "healthy",
      "FailingStreak" : 0,
      "Log" : [ {
        "Start" : "2023-10-12T14:53:18.597425386Z",
        "End" : "2023-10-12T14:53:18.884618483Z",
        "ExitCode" : 0,
        "Output" : "Setting Active Processor Count to 8\nWARNING: could not count classes from all agent jars (skipped 1), class count and metaspace may not be sized correctly\nCalculating JVM memory based on 5299680K available memory\nFor more information on this calculation, see https://paketo.io/docs/reference/java-reference/#memory-calculator\nCalculated JVM Memory Configuration: -XX:MaxDirectMemorySize=10M -Xmx4626963K -XX:MaxMetaspaceSize=160716K -XX:ReservedCodeCacheSize=240M -Xss1M (Total Memory: 5299680K, Thread Count: 250, Loaded Class Count: 25961, Headroom: 0%)\nEnabling Java Native Memory Tracking\nAdding 141 container CA certificates to JVM truststore\nSpring Cloud Bindings Enabled\n"
      }, {
        "Start" : "2023-10-12T14:53:23.911449096Z",
        "End" : "2023-10-12T14:53:24.114763946Z",
        "ExitCode" : 0,
        "Output" : "Setting Active Processor Count to 8\nWARNING: could not count classes from all agent jars (skipped 1), class count and metaspace may not be sized correctly\nCalculating JVM memory based on 5290768K available memory\nFor more information on this calculation, see https://paketo.io/docs/reference/java-reference/#memory-calculator\nCalculated JVM Memory Configuration: -XX:MaxDirectMemorySize=10M -Xmx4618051K -XX:MaxMetaspaceSize=160716K -XX:ReservedCodeCacheSize=240M -Xss1M (Total Memory: 5290768K, Thread Count: 250, Loaded Class Count: 25961, Headroom: 0%)\nEnabling Java Native Memory Tracking\nAdding 141 container CA certificates to JVM truststore\nSpring Cloud Bindings Enabled\n"
      }, {
        "Start" : "2023-10-12T14:53:29.131017693Z",
        "End" : "2023-10-12T14:53:29.337171625Z",
        "ExitCode" : 0,
        "Output" : "Setting Active Processor Count to 8\nWARNING: could not count classes from all agent jars (skipped 1), class count and metaspace may not be sized correctly\nCalculating JVM memory based on 5286584K available memory\nFor more information on this calculation, see https://paketo.io/docs/reference/java-reference/#memory-calculator\nCalculated JVM Memory Configuration: -XX:MaxDirectMemorySize=10M -Xmx4613867K -XX:MaxMetaspaceSize=160716K -XX:ReservedCodeCacheSize=240M -Xss1M (Total Memory: 5286584K, Thread Count: 250, Loaded Class Count: 25961, Headroom: 0%)\nEnabling Java Native Memory Tracking\nAdding 141 container CA certificates to JVM truststore\nSpring Cloud Bindings Enabled\n"
      }, {
        "Start" : "2023-10-12T14:53:34.351272356Z",
        "End" : "2023-10-12T14:53:34.597761185Z",
        "ExitCode" : 0,
        "Output" : "Setting Active Processor Count to 8\nWARNING: could not count classes from all agent jars (skipped 1), class count and metaspace may not be sized correctly\nCalculating JVM memory based on 5288452K available memory\nFor more information on this calculation, see https://paketo.io/docs/reference/java-reference/#memory-calculator\nCalculated JVM Memory Configuration: -XX:MaxDirectMemorySize=10M -Xmx4615735K -XX:MaxMetaspaceSize=160716K -XX:ReservedCodeCacheSize=240M -Xss1M (Total Memory: 5288452K, Thread Count: 250, Loaded Class Count: 25961, Headroom: 0%)\nEnabling Java Native Memory Tracking\nAdding 141 container CA certificates to JVM truststore\nSpring Cloud Bindings Enabled\n"
      }, {
        "Start" : "2023-10-12T14:53:39.615137884Z",
        "End" : "2023-10-12T14:53:39.876173642Z",
        "ExitCode" : 0,
        "Output" : "Setting Active Processor Count to 8\nWARNING: could not count classes from all agent jars (skipped 1), class count and metaspace may not be sized correctly\nCalculating JVM memory based on 5289164K available memory\nFor more information on this calculation, see https://paketo.io/docs/reference/java-reference/#memory-calculator\nCalculated JVM Memory Configuration: -XX:MaxDirectMemorySize=10M -Xmx4616447K -XX:MaxMetaspaceSize=160716K -XX:ReservedCodeCacheSize=240M -Xss1M (Total Memory: 5289164K, Thread Count: 250, Loaded Class Count: 25961, Headroom: 0%)\nEnabling Java Native Memory Tracking\nAdding 141 container CA certificates to JVM truststore\nSpring Cloud Bindings Enabled\n"
      } ]
    }

Possible Solution

Unsure why this happened. Maybe it the /cnb/process/healthcheck binary somehow triggers other CNB process, like /cnb/process/web which starts a JVM in my case.

Motivations

As this is a small project I just moved to the previous alternative I had curl as a healthcheck utility. I think this case wasn't tested until now.

[dmikusa]

Sorry for the delay. I don't know exactly what would cause this, but I agree. It looks like it is running your main app instead of the health check, which is weird and unexpected.

What would help to troubleshoot this are the build logs. If you could attach or link to a full build log for your app and the full command used to build, if using pack, or your buildpack config block if using Spring Boot build tools. That will give us a lot of info about how things were built so we can understand what happened, and info for reproducing.

Second, you could run pack inspect <img> on the image it produces. This will show what process types have been set and what they do. That will just provide some more context about the state of the image that got generated.

Thanks

[Dr4K4n]

Hi, we are having the same issue, building a Spring Boot app with maven and seeing the JVM output in the health checker status. Maybe this is also the cause for some "false positives" of the health check, as we sometimes experience a restart of the application when it's working perfectly fine. I suspect that there is not enough memory available to start "another" JVM and then the health check will fail (and restart the Spring Boot app).

I currently don't have the pack binary installed, so this is the debug info I can provide right now. Let me know if you need more, then I'll install pack and take a closer look.

pom.xml maven-log.txt docker inspect health.json.txt

[dmikusa]

Thanks for providing this info.

In the build log, I can see that it's running the health checker buildpack and I can see that it's setting up the process type to run the health checker.

[INFO]     [creator]     Paketo Buildpack for Health Checkers 1.15.0
[INFO]     [creator]       https://github.com/paketo-buildpacks/health-checker
[INFO]     [creator]       Build Configuration:
[INFO]     [creator]         $BP_HEALTH_CHECKER_DEPENDENCY  thc   which health checker to contribute
[INFO]     [creator]         $BP_HEALTH_CHECKER_ENABLED     true  contributes a health checker if enabled
[INFO]     [creator]       Tiny Health Checker 0.26.0: Reusing cached layer
[INFO]     [creator]       Process types:
[INFO]     [creator]         health-check: thc (direct)

The health-check process should run tiny health checker according to that info.

Just like what was reported before, we can clearly see in the docker inspect health output that it's not running tiny health checker, it's running the main Java app process which is why it's running out of memory and crashing.

I don't see anything else in the build log that stands out. It's running the normal buildpacks, output looks normal, I see it setting up the normal process types, nothing appears to be overwriting the health-check process type.

Two additional things that would help understanding what's happened here:

1. What is your docker run command or what are you putting in your Docker Compose file? It would help to know how the container is being invoked, in particular what you're configuring for the health check. This is not documented well, but there's an example with a Spring Boot app here -> https://github.com/paketo-buildpacks/health-checker/issues/30#issuecomment-1732612325.
2. The output of pack inspect <img> which will tell us what process types are set up on the contain. This will confirm that the health-check process type is set up correctly.

Thanks!

[Dr4K4n]

our `docker-compose.yml for deployment in a docker swarm looks like this:

services:
  XXXXXX:
    image: XXXXXX
    healthcheck:
      test: ["CMD", "/cnb/process/health-check"]
      start_period: 720s
      interval: 10s
      timeout: 30s
      retries: 5

I also tried the following ways to specify the test command, without an affect:

      test: ["CMD", "health-check"]
      test: health-check
      test: /cnb/process/health-check

running pack inspect gives the following:

➜  ~ pack inspect XXXXXX
Inspecting image: XXXXXX

REMOTE:

Stack: io.buildpacks.stacks.jammy

Base Image:
  Reference: 37b17ff4c5a9e151c7534d399d0e563f487e9fa7b1b713ccf966e90b8032ab66
  Top Layer: sha256:cf338ec78186de9f64329308386b83c408887d6109e93ebfd3fd78e669abe690

Run Images:
  index.docker.io/paketobuildpacks/run-jammy-base:latest

Rebasable: true

Buildpacks:
  ID                                         VERSION        HOMEPAGE
  paketo-buildpacks/ca-certificates          3.6.8          https://github.com/paketo-buildpacks/ca-certificates
  paketo-buildpacks/bellsoft-liberica        10.5.5         https://github.com/paketo-buildpacks/bellsoft-liberica
  paketo-buildpacks/syft                     1.45.0         https://github.com/paketo-buildpacks/syft
  paketo-buildpacks/executable-jar           6.8.5          https://github.com/paketo-buildpacks/executable-jar
  paketo-buildpacks/dist-zip                 5.6.10         https://github.com/paketo-buildpacks/dist-zip
  paketo-buildpacks/spring-boot              5.27.11        https://github.com/paketo-buildpacks/spring-boot
  paketo-buildpacks/health-checker           1.15.0         https://github.com/paketo-buildpacks/health-checker

Processes:
  TYPE                  SHELL        COMMAND        ARGS                                                      WORK DIR
  web (default)                      java           org.springframework.boot.loader.launch.JarLauncher        /workspace
  executable-jar                     java           org.springframework.boot.loader.launch.JarLauncher        /workspace
  health-check                       thc                                                                      /workspace
  task                               java           org.springframework.boot.loader.launch.JarLauncher        /workspace

LOCAL:

Stack: io.buildpacks.stacks.jammy

Base Image:
  Reference: 37b17ff4c5a9e151c7534d399d0e563f487e9fa7b1b713ccf966e90b8032ab66
  Top Layer: sha256:cf338ec78186de9f64329308386b83c408887d6109e93ebfd3fd78e669abe690

Run Images:
  index.docker.io/paketobuildpacks/run-jammy-base:latest

Rebasable: true

Buildpacks:
  ID                                         VERSION        HOMEPAGE
  paketo-buildpacks/ca-certificates          3.6.8          https://github.com/paketo-buildpacks/ca-certificates
  paketo-buildpacks/bellsoft-liberica        10.5.5         https://github.com/paketo-buildpacks/bellsoft-liberica
  paketo-buildpacks/syft                     1.45.0         https://github.com/paketo-buildpacks/syft
  paketo-buildpacks/executable-jar           6.8.5          https://github.com/paketo-buildpacks/executable-jar
  paketo-buildpacks/dist-zip                 5.6.10         https://github.com/paketo-buildpacks/dist-zip
  paketo-buildpacks/spring-boot              5.27.11        https://github.com/paketo-buildpacks/spring-boot
  paketo-buildpacks/health-checker           1.15.0         https://github.com/paketo-buildpacks/health-checker

Processes:
  TYPE                  SHELL        COMMAND        ARGS                                                      WORK DIR
  web (default)                      java           org.springframework.boot.loader.launch.JarLauncher        /workspace
  executable-jar                     java           org.springframework.boot.loader.launch.JarLauncher        /workspace
  health-check                       thc                                                                      /workspace
  task                               java           org.springframework.boot.loader.launch.JarLauncher        /workspace

Our current workaround is to use the paketobuildpacks/builder-jammy-full builder which includes curl and then changing the healthcheck to test: curl --fail http://localhost:8080 || exit 1. Sadly this also increases the size of our image from ~250 MB to ~450 MB. A "better" workaround might be to create our own builder based on the base builder (as documented here) and just add curl to it. Nonetheless this works for now and I hope we (mostly you, sorry :D) can fix the underlying issue. Thank you very much so far.

[dmikusa]

Well, the process type is there. You can see health-check. That means calling /cnb/process/health-check should result in running that command.

I had a suspicion that if the process type were missing, it could cause the issue you're seeing. Because everything under /cnb/process is just a symlink to the launcher. The launcher then looks at arg[0] (i.e. the command name) and uses that to determine how to start the process (at least last time I looked at the code). Thus if you try to run a process type that doesn't actually exist, it might run the default process which is your Java app.

That doesn't seem to be the case here though. The metadata shows the process type is set.

The only other thing you could do to check would be to docker exec into the container when it's running and look at /cnb/process and see what all is present. I can't imagine the metadata being out of sync with what's in the container, but I suppose it's worth verifying.

Another option. If you can reproduce this with a dummy app and you can push the container up to Docker, I'd be happy to take a look at it directly. Part of the trouble is that I'm not able to reproduce the error.

Many thanks for your feedback on this!

[dmikusa]

I was playing around with this more and I'm not sure there is actually a problem here. It is certainly a little confusing, but it's not launching the JVM as was originally thought.

Setting Active Processor Count to 16
Calculating JVM memory based on 48430256K available memory
For more information on this calculation, see https://paketo.io/docs/reference/java-reference/#memory-calculator
Calculated JVM Memory Configuration: -XX:MaxDirectMemorySize=10M -Xmx47701742K -XX:MaxMetaspaceSize=216513K -XX:ReservedCodeCacheSize=240M -Xss1M (Total Memory: 48430256K, Thread Count: 250, Loaded Class Count: 35812, Headroom: 0%)
Enabling Java Native Memory Tracking
Adding 137 container CA certificates to JVM truststore
Spring Cloud Bindings Enabled
Error: 
request error: http://localhost:8080/: Connection Failed: Connect error: Connection refused (os error 111)

If I look into the output from your docker inspect, we see the above info.

The first lines, Setting Active... through ...Bindings Enabled are not from the app, but are from helper binaries that get installed by the buildpacks. These make sure that the application environment is configured correctly and they are run by the buildpacks launcher prior to starting your actual application.

Because we are pointing to /cnb/process/* for the health check process, it is being executed through the launcher and you see those messages. The processes that run those are Go binaries and should be extremely small, lightweight, short-running and low-memory.

Then we see the line Error: ... which is from thc and is telling us that the health check failed to make a connection.

I don't see any output from the JVM though. If the JVM were starting, you'd see the line Picked up JAVA_TOOL_OPTIONS:... which is the first log line you'd see from the JVM.

I see the same thing in the original poster's logs and I see the same thing when I run docker inspect locally (I wasn't looking at this before cause my container was coming up healthy).

If you wanted to remove that extra logging and extra processes running, you can invoke thc directly. The reason we invoke it as a process type through launcher is just because it's easier. You don't need to know the process name or where it's installed (it won't be on $PATH unless you run it through the launcher, because the launcher is what adds it to the path).

If you want to invoke it directly, you can though. If you open your app container with dive and scroll down to the layer that's added by the health checker buildpack, you can see the path where it installs the binary. The path is pretty unlikely to change and is currently set to /layers/paketo-buildpacks_health-checker/thc/bin/thc.

So you could do something like:

services:
  maven:
    image: apps/maven:latest
    environment:
      THC_PATH: /actuator/health
      THC_PORT: 8080
    healthcheck:
      test: ["CMD", "/layers/paketo-buildpacks_health-checker/thc/bin/thc"]
      start_period: 10s
      interval: 1s
      timeout: 20s
      retries: 5

and it'll run without those other commands being invoked:

{
  "Status": "healthy",
  "FailingStreak": 0,
  "Log": [
    {
      "Start": "2024-04-30T13:30:45.277534213Z",
      "End": "2024-04-30T13:30:45.343303379Z",
      "ExitCode": 0,
      "Output": ""
    },
    {
      "Start": "2024-04-30T13:30:46.34835888Z",
      "End": "2024-04-30T13:30:46.41740938Z",
      "ExitCode": 0,
      "Output": ""
    },
    {
      "Start": "2024-04-30T13:30:47.421350714Z",
      "End": "2024-04-30T13:30:47.48149563Z",
      "ExitCode": 0,
      "Output": ""
    },
    {
      "Start": "2024-04-30T13:30:48.487616089Z",
      "End": "2024-04-30T13:30:48.543178089Z",
      "ExitCode": 0,
      "Output": ""
    },
    {
      "Start": "2024-04-30T13:30:49.550032715Z",
      "End": "2024-04-30T13:30:49.61748434Z",
      "ExitCode": 0,
      "Output": ""
    }
  ]
}

Hope that helps!

[azatoth]

We are having the same issue with a spring boot application and the health checker pack. Though we've concluded it's not health-checker per se that's the cause, but that the launcher is the culpit:

root@ip-10-0-0-85:/workspace# THC_PATH=/actuator/health health-check 
Setting Active Processor Count to 2
Calculated JVM Memory Configuration: -XX:MaxDirectMemorySize=10M -Xmx3505223K -XX:MaxMetaspaceSize=177080K -XX:ReservedCodeCacheSize=240M -Xss1M (Total Memory: 4G, Thread Count: 250, Loaded Class Count: 28850, Headroom: 0%)
Enabling Java Native Memory Tracking
Adding 137 container CA certificates to JVM truststore
Spring Cloud Bindings Enabled
root@ip-10-0-0-85:/workspace# echo $?
0
root@ip-10-0-0-85:/workspace# THC_PATH=/actuator/health /layers/paketo-buildpacks_health-checker/thc/bin/thc
root@ip-10-0-0-85:/workspace# echo $?
0
root@ip-10-0-0-85:/workspace# which health-check
/cnb/process/health-check
root@ip-10-0-0-85:/workspace# ls -l /cnb/process/health-check
lrwxrwxrwx 1 root root 23 Jan  1  1980 /cnb/process/health-check -> /cnb/lifecycle/launcher
root@ip-10-0-0-85:/workspace# /cnb/lifecycle/launcher --help
Setting Active Processor Count to 2
Calculated JVM Memory Configuration: -XX:MaxDirectMemorySize=10M -Xmx3505223K -XX:MaxMetaspaceSize=177080K -XX:ReservedCodeCacheSize=240M -Xss1M (Total Memory: 4G, Thread Count: 250, Loaded Class Count: 28850, Headroom: 0%)
Enabling Java Native Memory Tracking
Adding 137 container CA certificates to JVM truststore
Spring Cloud Bindings Enabled
bash: --: invalid option
Usage:  bash [GNU long option] [option] ...
    bash [GNU long option] [option] script-file ...
GNU long options:
    --debug
    --debugger
    --dump-po-strings
    --dump-strings
    --help
    --init-file
    --login
    --noediting
    --noprofile
    --norc
    --posix
    --rcfile
    --restricted
    --verbose
    --version
Shell options:
    -ilrsD or -c command or -O shopt_option     (invocation only)
    -abefhkmnptuvxBCHP or -o option
root@ip-10-0-0-85:/workspace# 

it's clear that the printed out text comes from spring-boot but I don't know why or how it's leaking. I suspect it's due to the executable-jar process.

Regardless the issue is probably external this package but I'm not sure to where it should be referred to.

[dmikusa]

Setting Active Processor Count to 2
Calculated JVM Memory Configuration: -XX:MaxDirectMemorySize=10M -Xmx3505223K -XX:MaxMetaspaceSize=177080K -XX:ReservedCodeCacheSize=240M -Xss1M (Total Memory: 4G, Thread Count: 250, Loaded Class Count: 28850, Headroom: 0%)
Enabling Java Native Memory Tracking
Adding 137 container CA certificates to JVM truststore
Spring Cloud Bindings Enabled

This all comes from Go binary helper files, not Spring Boot. These get executed when you run health-check via the /cnb/process/health-check because it points to the launcher and the laucher ensures that environment variables are set up correctly and that helper (exec.d in the spec) are executed. This is all as described in the spec and working correctly.

If you don't want that extra output you can run the tiny health check binary directly, like you're doing above. That bypasses the launcher. The only reason we set up the /cnb/process/health-check binary is that it makes it easier because you don't need the path to the tiny health check binary. It's not doing anything else, so calling the binary directly is just fine if you'd prefer to do that. The path to it could technically change, but it's very unlikely.

[azatoth]

I think I understand what you mean now; The launcher will recreate the same environment for all commands, and the base image/layer (libjvm) is injecting this expensive procedure into there.

[herder]

Are you running with the regular JVM or with Corretto? We're seeing the same behavior: when running directly against the thc binary we can call the health check indefinitely, but when running with the wrapper it slows down visibly after maybe 10-15 seconds. The openssl-certificate-loader in the Amazon Corretto pack seems to be a suspect: it pops up regularly with high CPU load, then vanishes a while until returning again.

[azatoth]

Are you running with the regular JVM or with Corretto? [...] The openssl-certificate-loader in the Amazon Corretto pack seems to be a suspect

We're using Amazon Corretto, and yes, we noticed openssl-certificate-loader hogging 2GB+ memory and up to 10s delay each time the health check was to be executed.

I guess it would be reasonable to open up a ticket against https://github.com/paketo-buildpacks/amazon-corretto

[dmikusa]

Hmm, this is good feedback. Thanks @herder and @azatoth. Those helper processes should be very fast, that was the initial though, but it appears something is happening with the openssl-certificate-loader where that is not the case.

At this point, I would definitely recommend running the health check thc binary directly and bypassing the loader. That is going to be the most optimal situation. As I mentioned before, we added the /cnb/process/health-check symlink for convenience but that may just be a bad idea and not worth it, since it's always going to do additional work when you run the launcher and your health check process is meant to run often (which creates a lot of waste).

[azatoth]

As I mentioned before, we added the /cnb/process/health-check symlink for convenience but that may just be a bad idea and not worth it

There's no possibility to add a symlink to the actual executable in $PATH?

[dmikusa]

There's no possibility to add a symlink to the actual executable in $PATH?

That's what the launcher does, so it is not without the launcher and its drawbacks. ~I will ask and see if there's a way to use the launcher and not run the exec.d processes, but I think~ it's required based on the cloud-native buildpack spec.

[azatoth]

There's no possibility to add a symlink to the actual executable in $PATH?

That's what the launcher does, so it is not without the launcher and its drawbacks. ~I will ask and see if there's a way to use the launcher and not run the exec.d processes, but I think~ it's required based on the cloud-native buildpack spec.

I must confess I'm a bit confused for the apparent convoluted setup with the launcher and how it executes executables depending on the name of the source symlink pointing to the launcher instead of just having the source symlink able to point directly to the target executable in the layer. I'm sure it's setup this way for a reason but it's beyond my knowledge.

[herder]

There's no possibility to add a symlink to the actual executable in $PATH?

That's what the launcher does, so it is not without the launcher and its drawbacks. ~I will ask and see if there's a way to use the launcher and not run the exec.d processes, but I think~ it's required based on the cloud-native buildpack spec.

That's interesting - I wonder if there's a requirement in the spec that exec.d processes have to start quickly, or if it's just not assumed that any processes will run regularly like the health check does?

I'm not sure if this is a bug or feature in the Corretto ssl helper that it starts like this - maybe that process should be run during build time instead of runtime?

Not sure if I'm misunderstanding something, so I'm hesitant to file a bug there yet.

[dmikusa]

That's interesting - I wonder if there's a requirement in the spec that exec.d processes have to start quickly, or if it's just not assumed that any processes will run regularly like the health check does?

There are no requirements to run under a certain time frame, but exec.d processes all run before your main application starts so there is an implication that they'll run fast enough to not block your app from starting in a reasonable amount of time. Reasonable is subjective, but some systems require apps to start within a certain amount of time, to pass health checks, etc... So exec.d processes need to complete and still give the app sufficient time to start.

Paketo writes their exec.d processes in Go and they are usually very light weight, just adjusting env variables so they're usually very fast to execute. That's partly why I initially thought it should be OK to just run them, even though they're running quite often with the health checks.

I'm not sure if this is a bug or feature in the Corretto ssl helper that it starts like this - maybe that process should be run during build time instead of runtime?

I think the exec.d helper is fine. What it's doing is still lightweight, but just has a little more overhead than setting some env variables. In this case, it's modifying files on disk, in particular the truststore for the JVM. The reason it does this at runtime is so that changes to trusted certificates can be applied dynamically and not require a container rebuild.

I think this is a case where the exec.d process was written assuming that it would only run once. The overhead is small, but it's modifying the truststore, which is something that doesn't make sense after the JVM has started. I'm going to open an issue and suggest that we could improve this particular exec.d process by making it only run once, before the JVM has started.

That will help here a little, but ultimately, I still think it's probably best to not invoke the launcher every time we want the health check to run. That happens a lot and there's always some overhead there.

[herder]

I think this is a case where the exec.d process was written assuming that it would only run once. The overhead is small, but it's modifying the truststore, which is something that doesn't make sense after the JVM has started. I'm going to open an issue and suggest that we could improve this particular exec.d process by making it only run once, before the JVM has started.

That will help here a little, but ultimately, I still think it's probably best to not invoke the launcher every time we want the health check to run. That happens a lot and there's always some overhead there.

Thanks, that makes sense. Though the overhead is not that small imho: it does make the check fail after a while. But maybe adding a check to the Corretto helper that makes it a noop if it has already run would be an improvement here. Though there may come other helpers that assume that they are only run at startup, so I wonder if the specification should at least clarify that this might not always be the case, like here?

[dmikusa]

Ok, I'm going to close this issue out. We've released health-checker 2.0.0, which removes the /cnb/process/health-check symlink. It adds a convenience symlink at /workspace/health-check or you can invoke thc with the full path to the binary. The output of the buildpack now prints that information as a reminder.

  Tiny Health Checker 0.27.0: Reusing cached layer
    The Tiny Health Checker binary is available at /layers/paketo-buildpacks_health-checker/thc/bin/thc
    A symlink is available at /workspace/health-check for your convenience

Please note that since we're not using the launcher to invoke the health check binary, we cannot set env variables on the image at build time. All environment variables to configure the health check binary must be set when you run the container.

The release notes are here: https://github.com/paketo-buildpacks/health-checker/releases/tag/v2.0.0