search

buildpacks / lifecycle

Reference implementation of the Cloud Native Buildpacks lifecycle
https://buildpacks.io
Apache License 2.0
186 stars 104 forks source link

$USER unset when using creator lifecycle #780

Open ajdergute opened 2 years ago

ajdergute commented 2 years ago

I'm building a Spring Boot application using Gradle. I tried using the builder:base image. It's failing with the following error:

Paketo Gradle Buildpack 6.0.1
  https://github.com/paketo-buildpacks/gradle
  Build Configuration:
    $BP_GRADLE_BUILD_ARGUMENTS  --no-daemon assemble  the arguments to pass to Gradle
    $BP_GRADLE_BUILT_ARTIFACT   build/libs/*.[jw]ar   the built application artifact explicitly.  Supersedes $BP_GRADLE_BUILT_MODULE
    $BP_GRADLE_BUILT_MODULE                           the module to find application artifact in
Paketo Gradle Buildpack 6.0.1
  unable to resolve home directory
  unable to determine user home directory
  user: Current requires cgo or $USER set in environment
ERROR: failed to build: exit status 1

I double checked my environment and I'm sure $HOME and $USER are set and exported. My Docker container is running with flag -umyuser,thus no entry in/etc/passwd` exists.

The Gradle Buildpack uses user.Current(), which seems to be fine to me.

But SetEnvironmentForCurrentUser only uses lookupId, which in my case doesn't resolve user name nor home directory. https://github.com/buildpacks/lifecycle/blob/42be2d161b283c8f831f78a695df4c7c35782709/priv/user_linux.go#L98-L108

I think this is known, cause user.Current() already has a comment in there: https://go.dev/src/os/user/lookup_stubs.go

IMHO SetEnvironmentForCurrentUser should not change values of those environment variables if they are already set.

ajdergute commented 2 years ago

Working example:

user@dude:/tmp/demo1000# ll -n
insgesamt 32
-rw-r--r-- 1 1000 1000  427  8. Dez 13:46 build.gradle
drwxr-xr-x 3 1000 1000 4096  8. Dez 13:46 gradle
-rwxr-xr-x 1 1000 1000 8070  8. Dez 13:46 gradlew
-rw-r--r-- 1 1000 1000 2763  8. Dez 13:46 gradlew.bat
-rw-r--r-- 1 1000 1000  574  8. Dez 13:46 HELP.md
-rw-r--r-- 1 1000 1000   26  8. Dez 13:46 settings.gradle
drwxr-xr-x 4 1000 1000 4096  8. Dez 13:46 src
user@dude:/tmp/demo1000# docker run --rm -it -v "$PWD":/usr/src/app -w /usr/src/app paketobuildpacks/builder:base bash
cnb@03ae8025332a:/usr/src/app$ /cnb/lifecycle/creator -app=. demo-app:latest
Warning: Not restoring or caching layer data, no cache flag specified.
===> DETECTING
8 of 19 buildpacks participating
paketo-buildpacks/ca-certificates   3.0.1
paketo-buildpacks/bellsoft-liberica 9.0.1
paketo-buildpacks/syft              1.0.1
paketo-buildpacks/gradle            6.0.1
paketo-buildpacks/executable-jar    6.0.1
paketo-buildpacks/apache-tomcat     7.0.1
paketo-buildpacks/dist-zip          5.0.1
paketo-buildpacks/spring-boot       5.1.0
===> ANALYZING
Previous image with name "demo-app:latest" not found
===> RESTORING
===> BUILDING

Paketo CA Certificates Buildpack 3.0.1
  https://github.com/paketo-buildpacks/ca-certificates
  Launch Helper: Contributing to layer
    Creating /layers/paketo-buildpacks_ca-certificates/helper/exec.d/ca-certificates-helper

Paketo BellSoft Liberica Buildpack 9.0.1
  https://github.com/paketo-buildpacks/bellsoft-liberica
  Build Configuration:
    $BP_JVM_TYPE                 JRE             the JVM type - JDK or JRE
    $BP_JVM_VERSION              11              the Java version
  Launch Configuration:
    $BPL_DEBUG_ENABLED           false           enables Java remote debugging support
    $BPL_DEBUG_PORT              8000            configure the remote debugging port
    $BPL_DEBUG_SUSPEND           false           configure whether to suspend execution until a debugger has attached
    $BPL_HEAP_DUMP_PATH                          write heap dumps on error to this path
    $BPL_JAVA_NMT_ENABLED        true            enables Java Native Memory Tracking (NMT)
    $BPL_JAVA_NMT_LEVEL          summary         configure level of NMT, summary or detail
    $BPL_JFR_ARGS                                configure custom Java Flight Recording (JFR) arguments
    $BPL_JFR_ENABLED             false           enables Java Flight Recording (JFR)
    $BPL_JMX_ENABLED             false           enables Java Management Extensions (JMX)
    $BPL_JMX_PORT                5000            configure the JMX port
    $BPL_JVM_HEAD_ROOM           0               the headroom in memory calculation
    $BPL_JVM_LOADED_CLASS_COUNT  35% of classes  the number of loaded classes in memory calculation
    $BPL_JVM_THREAD_COUNT        250             the number of threads in memory calculation
    $JAVA_TOOL_OPTIONS                           the JVM launch flags
  BellSoft Liberica JDK 11.0.13: Contributing to layer
    Downloading from https://github.com/bell-sw/Liberica/releases/download/11.0.13+8/bellsoft-jdk11.0.13+8-linux-amd64.tar.gz
    Verifying checksum
    Expanding to /layers/paketo-buildpacks_bellsoft-liberica/jdk
    Adding 128 container CA certificates to JVM truststore
    Writing env.build/JAVA_HOME.override
    Writing env.build/JDK_HOME.override
  BellSoft Liberica JRE 11.0.13: Contributing to layer
    Downloading from https://github.com/bell-sw/Liberica/releases/download/11.0.13+8/bellsoft-jre11.0.13+8-linux-amd64.tar.gz
    Verifying checksum
    Expanding to /layers/paketo-buildpacks_bellsoft-liberica/jre
    Adding 128 container CA certificates to JVM truststore
    Writing env.launch/BPI_APPLICATION_PATH.default
    Writing env.launch/BPI_JVM_CACERTS.default
    Writing env.launch/BPI_JVM_CLASS_COUNT.default
    Writing env.launch/BPI_JVM_SECURITY_PROVIDERS.default
    Writing env.launch/JAVA_HOME.default
    Writing env.launch/JAVA_TOOL_OPTIONS.append
    Writing env.launch/JAVA_TOOL_OPTIONS.delim
    Writing env.launch/MALLOC_ARENA_MAX.default
  Launch Helper: Contributing to layer
    Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/active-processor-count
    Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/java-opts
    Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/jvm-heap
    Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/link-local-dns
    Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/memory-calculator
    Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/openssl-certificate-loader
    Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/security-providers-configurer
    Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/jmx
    Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/jfr
    Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/nmt
    Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/security-providers-classpath-9
    Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/debug-9
  Java Security Properties: Contributing to layer
    Writing env.launch/JAVA_SECURITY_PROPERTIES.default
    Writing env.launch/JAVA_TOOL_OPTIONS.append
    Writing env.launch/JAVA_TOOL_OPTIONS.delim

Paketo Syft Buildpack 1.0.1
  https://github.com/paketo-buildpacks/syft
    Downloading from https://github.com/anchore/syft/releases/download/v0.30.1/syft_0.30.1_linux_amd64.tar.gz
    Verifying checksum

Paketo Gradle Buildpack 6.0.1
  https://github.com/paketo-buildpacks/gradle
  Build Configuration:
    $BP_GRADLE_BUILD_ARGUMENTS  --no-daemon assemble  the arguments to pass to Gradle
    $BP_GRADLE_BUILT_ARTIFACT   build/libs/*.[jw]ar   the built application artifact explicitly.  Supersedes $BP_GRADLE_BUILT_MODULE
    $BP_GRADLE_BUILT_MODULE                           the module to find application artifact in
    Creating cache directory /home/cnb/.gradle
  Compiled Application: Contributing to layer
    Executing gradlew --no-daemon assemble
      Downloading https://services.gradle.org/distributions/gradle-7.3.1-bin.zip
      ...........10%...........20%...........30%...........40%...........50%...........60%...........70%...........80%...........90%...........100%

      Welcome to Gradle 7.3.1!

      Here are the highlights of this release:
       - Easily declare new test suites in Java projects
       - Support for Java 17
       - Support for Scala 3

      For more details see https://docs.gradle.org/7.3.1/release-notes.html

      To honour the JVM settings for this build a single-use Daemon process will be forked. See https://docs.gradle.org/7.3.1/userguide/gradle_daemon.html#sec:disabling_the_daemon.
      Daemon will be stopped at the end of the build
      > Task :compileJava
      > Task :processResources
      > Task :classes
      > Task :bootJarMainClassName
      > Task :bootJar
      > Task :jar
      > Task :assemble

      BUILD SUCCESSFUL in 28s
      5 actionable tasks: 5 executed
  Removing source code

Paketo Executable JAR Buildpack 6.0.1
  https://github.com/paketo-buildpacks/executable-jar
  Class Path: Contributing to layer
    Writing env/CLASSPATH.delim
    Writing env/CLASSPATH.prepend
  Process types:
    executable-jar: java org.springframework.boot.loader.JarLauncher (direct)
    task:           java org.springframework.boot.loader.JarLauncher (direct)
    web:            java org.springframework.boot.loader.JarLauncher (direct)

Paketo Spring Boot Buildpack 5.1.0
  https://github.com/paketo-buildpacks/spring-boot
  Creating slices from layers index
    dependencies
    spring-boot-loader
    snapshot-dependencies
    application
  Launch Helper: Contributing to layer
    Creating /layers/paketo-buildpacks_spring-boot/helper/exec.d/spring-cloud-bindings
  Spring Cloud Bindings 1.8.0: Contributing to layer
    Downloading from https://repo.spring.io/release/org/springframework/cloud/spring-cloud-bindings/1.8.0/spring-cloud-bindings-1.8.0.jar
    Verifying checksum
    Copying to /layers/paketo-buildpacks_spring-boot/spring-cloud-bindings
  Web Application Type: Contributing to layer
    Non-web application detected
    Writing env.launch/BPL_JVM_THREAD_COUNT.default
  4 application slices
  Image labels:
    org.springframework.boot.version
===> EXPORTING
Adding layer 'paketo-buildpacks/ca-certificates:helper'
Adding layer 'paketo-buildpacks/bellsoft-liberica:helper'
Adding layer 'paketo-buildpacks/bellsoft-liberica:java-security-properties'
Adding layer 'paketo-buildpacks/bellsoft-liberica:jre'
Adding layer 'paketo-buildpacks/executable-jar:classpath'
Adding layer 'paketo-buildpacks/spring-boot:helper'
Adding layer 'paketo-buildpacks/spring-boot:spring-cloud-bindings'
Adding layer 'paketo-buildpacks/spring-boot:web-application-type'
Adding 5/5 app layer(s)
Adding layer 'launcher'
Adding layer 'config'
Adding label 'io.buildpacks.lifecycle.metadata'
Adding label 'io.buildpacks.build.metadata'
Adding label 'io.buildpacks.project.metadata'
Adding label 'org.springframework.boot.version'
Saving demo-app:latest...

NOT Working example:

user@dude:/tmp/demo1234# ll -n
insgesamt 32
-rw-r--r-- 1 1234 1234  427  8. Dez 14:55 build.gradle
drwxr-xr-x 3 1234 1234 4096  8. Dez 14:55 gradle
-rwxr-xr-x 1 1234 1234 8070  8. Dez 14:55 gradlew
-rw-r--r-- 1 1234 1234 2763  8. Dez 14:55 gradlew.bat
-rw-r--r-- 1 1234 1234  574  8. Dez 14:55 HELP.md
-rw-r--r-- 1 1234 1234   26  8. Dez 14:55 settings.gradle
drwxr-xr-x 4 1234 1234 4096  8. Dez 14:55 src
user@dude:/tmp/demo1234# docker run --rm -u 1234:1234 -it -v "$PWD":/usr/src/app -w /usr/src/app paketobuildpacks/builder:base bash
groups: cannot find name for group ID 1234
I have no name!@c829413821ac:/usr/src/app$ mkdir layers
I have no name!@c829413821ac:/usr/src/app$ env
HOSTNAME=c829413821ac
CNB_STACK_ID=io.buildpacks.stacks.bionic
PWD=/usr/src/app
HOME=/
CNB_USER_ID=1000
TERM=xterm
CNB_GROUP_ID=1000
SHLVL=1
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
_=/usr/bin/env
I have no name!@c829413821ac:/usr/src/app$ export HOME=/usr/src/app
I have no name!@c829413821ac:~$ export USER=test
I have no name!@c829413821ac:~$ /cnb/lifecycle/creator -app=. -uid 1234 -gid 1234 -layers ./layers/ demo-app:latest
Warning: Not restoring or caching layer data, no cache flag specified.
===> DETECTING
8 of 19 buildpacks participating
paketo-buildpacks/ca-certificates   3.0.1
paketo-buildpacks/bellsoft-liberica 9.0.1
paketo-buildpacks/syft              1.0.1
paketo-buildpacks/gradle            6.0.1
paketo-buildpacks/executable-jar    6.0.1
paketo-buildpacks/apache-tomcat     7.0.1
paketo-buildpacks/dist-zip          5.0.1
paketo-buildpacks/spring-boot       5.1.0
===> ANALYZING
Previous image with name "demo-app:latest" not found
===> RESTORING
===> BUILDING

Paketo CA Certificates Buildpack 3.0.1
  https://github.com/paketo-buildpacks/ca-certificates
  Launch Helper: Contributing to layer
    Creating /usr/src/app/layers/paketo-buildpacks_ca-certificates/helper/exec.d/ca-certificates-helper

Paketo BellSoft Liberica Buildpack 9.0.1
  https://github.com/paketo-buildpacks/bellsoft-liberica
  Build Configuration:
    $BP_JVM_TYPE                 JRE             the JVM type - JDK or JRE
    $BP_JVM_VERSION              11              the Java version
  Launch Configuration:
    $BPL_DEBUG_ENABLED           false           enables Java remote debugging support
    $BPL_DEBUG_PORT              8000            configure the remote debugging port
    $BPL_DEBUG_SUSPEND           false           configure whether to suspend execution until a debugger has attached
    $BPL_HEAP_DUMP_PATH                          write heap dumps on error to this path
    $BPL_JAVA_NMT_ENABLED        true            enables Java Native Memory Tracking (NMT)
    $BPL_JAVA_NMT_LEVEL          summary         configure level of NMT, summary or detail
    $BPL_JFR_ARGS                                configure custom Java Flight Recording (JFR) arguments
    $BPL_JFR_ENABLED             false           enables Java Flight Recording (JFR)
    $BPL_JMX_ENABLED             false           enables Java Management Extensions (JMX)
    $BPL_JMX_PORT                5000            configure the JMX port
    $BPL_JVM_HEAD_ROOM           0               the headroom in memory calculation
    $BPL_JVM_LOADED_CLASS_COUNT  35% of classes  the number of loaded classes in memory calculation
    $BPL_JVM_THREAD_COUNT        250             the number of threads in memory calculation
    $JAVA_TOOL_OPTIONS                           the JVM launch flags
  BellSoft Liberica JDK 11.0.13: Contributing to layer
    Downloading from https://github.com/bell-sw/Liberica/releases/download/11.0.13+8/bellsoft-jdk11.0.13+8-linux-amd64.tar.gz
    Verifying checksum
    Expanding to /usr/src/app/layers/paketo-buildpacks_bellsoft-liberica/jdk
    Adding 128 container CA certificates to JVM truststore
    Writing env.build/JAVA_HOME.override
    Writing env.build/JDK_HOME.override
  BellSoft Liberica JRE 11.0.13: Contributing to layer
    Downloading from https://github.com/bell-sw/Liberica/releases/download/11.0.13+8/bellsoft-jre11.0.13+8-linux-amd64.tar.gz
    Verifying checksum
    Expanding to /usr/src/app/layers/paketo-buildpacks_bellsoft-liberica/jre
    Adding 128 container CA certificates to JVM truststore
    Writing env.launch/BPI_APPLICATION_PATH.default
    Writing env.launch/BPI_JVM_CACERTS.default
    Writing env.launch/BPI_JVM_CLASS_COUNT.default
    Writing env.launch/BPI_JVM_SECURITY_PROVIDERS.default
    Writing env.launch/JAVA_HOME.default
    Writing env.launch/JAVA_TOOL_OPTIONS.append
    Writing env.launch/JAVA_TOOL_OPTIONS.delim
    Writing env.launch/MALLOC_ARENA_MAX.default
  Launch Helper: Contributing to layer
    Creating /usr/src/app/layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/active-processor-count
    Creating /usr/src/app/layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/java-opts
    Creating /usr/src/app/layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/jvm-heap
    Creating /usr/src/app/layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/link-local-dns
    Creating /usr/src/app/layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/memory-calculator
    Creating /usr/src/app/layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/openssl-certificate-loader
    Creating /usr/src/app/layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/security-providers-configurer
    Creating /usr/src/app/layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/jmx
    Creating /usr/src/app/layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/jfr
    Creating /usr/src/app/layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/nmt
    Creating /usr/src/app/layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/security-providers-classpath-9
    Creating /usr/src/app/layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/debug-9
  Java Security Properties: Contributing to layer
    Writing env.launch/JAVA_SECURITY_PROPERTIES.default
    Writing env.launch/JAVA_TOOL_OPTIONS.append
    Writing env.launch/JAVA_TOOL_OPTIONS.delim

Paketo Syft Buildpack 1.0.1 
  https://github.com/paketo-buildpacks/syft
    Downloading from https://github.com/anchore/syft/releases/download/v0.30.1/syft_0.30.1_linux_amd64.tar.gz
    Verifying checksum

Paketo Gradle Buildpack 6.0.1
  https://github.com/paketo-buildpacks/gradle
  Build Configuration:
    $BP_GRADLE_BUILD_ARGUMENTS  --no-daemon assemble  the arguments to pass to Gradle
    $BP_GRADLE_BUILT_ARTIFACT   build/libs/*.[jw]ar   the built application artifact explicitly.  Supersedes $BP_GRADLE_BUILT_MODULE
    $BP_GRADLE_BUILT_MODULE                           the module to find application artifact in

Paketo Gradle Buildpack 6.0.1
  unable to resolve home directory
  unable to determine user home directory
  user: Current requires cgo or $USER set in environment
ERROR: failed to build: exit status 1
ajdergute commented 2 years ago

I the mean time I believe the error is not caused by the priv package. It could be the case that the new build time env introduced in #700 is the root cause.

See: https://github.com/buildpacks/lifecycle/blob/main/env/build.go

Only a few env variables are populated to the new env. As a user I would expect the original environment is not modified or at least the environment is reset for each buildpack to the original one. Otherwise it would be impossible to pass custom variables to own buildpacks, right?

Does this make sense? Am I on the right path? In open for any comment or thought.

natalieparellano commented 2 years ago

Hi @ajdergute - thanks for reporting this. In looking into this, I do think it is a bug with the priv package. We're using user.Name to set $USER when we might need to use user.Username - see below test program and output.

When the lifecycle exec's the buildpack bin/build executable, $USER will be unset in the environment. I think that is what is causing the error you are seeing.

Test program:

import (
    "fmt"
    "os"
    "os/user"
)

func main() {
    _, err := user.Current()
    if err != nil {
        panic(fmt.Sprintf("getting current user: %s", err.Error()))
    }

    // show env
    fmt.Println("$HOME before:", os.Getenv("HOME"))
    fmt.Println("$USER before:", os.Getenv("USER"))

    // find user
    foundUser, err := user.LookupId(os.Args[1])
    if err != nil {
        panic(fmt.Sprintf("getting user: %s", err.Error()))
    }
    fmt.Printf("found user  : %+v\n", foundUser)

    // show values
    fmt.Println("$HOME after :", foundUser.HomeDir)
    fmt.Println("$USER after :", foundUser.Name)
    fmt.Println("Username:   :", foundUser.Username)
}

Run the following:

goroutine 1 [running]: main.main() /tmp/test/test.go:12 +0x5ac

I have no name!@1450da0229bc:/usr/src/app$ export USER=test I have no name!@1450da0229bc:/usr/src/app$ ./test 1234 $HOME before: / $USER before: test found user : &{Uid:1234 Gid:1234 Username:test Name: HomeDir:/} $HOME after : / <- $HOME is respected $USER after : <- $USER is unset Username: : test

ajdergute commented 2 years ago

Hi @natalieparellano, thanks for looking into this. Good catch. That seems very reasonable for me.

If there's something to test on my side I could do.

natalieparellano commented 2 years ago

Hi @ajdergute in looking into this a bit further, I think passing $USER through to the buildpack might require further discussion and/or a spec change. See here the environment variables that are specifically allowed by the lifecycle (which doesn't include $USER). We might indeed still want to do this, but we should bring it up with the community so that it doesn't produce behavior that is unexpected. I'll bring this up at the next office hours (details if interested, you are very welcome to join!).

In the meantime, you could pass $USER as a platform-provided variable (which would circumvent the lifecycle's include list) by writing the desired value to a file at /platform/env/USER in the build container.

ajdergute commented 2 years ago

Hi @natalieparellano, thank you very much for this update. I will try the platform variable approach.

Thanks also for your invitation.

ajdergute commented 2 years ago

The platform variable approach works like a charm. I was able to push my image to my local registry.

natalieparellano commented 2 years ago

@ekcasey to inquire if the buildpack really needs to have $USER

@jabrown85 to check if this behavior occurs when running builder (vs creator)

jabrown85 commented 2 years ago

I tested this just now using a sample builder with USER set. Just to see if buildpacks have access.

Neither trusted or untrusted flows allow access to USER, which is expected given this and the spec here.

The creator code called out does look to be using the incorrect value, but even if it weren't there wouldn't be access to USER from any buildpack in either trust or untrusted execution.

natalieparellano commented 2 years ago

Labeling this as "requires RFC" because we'd at a minimum need a spec change to pass the $USER env var through to buildpacks.