Workbench session init go cp

skyeturriff commented 1 week ago

Addresses https://github.com/rstudio/rstudio-pro/issues/7006

This pull request introduces changes to the workbench-session-init image, including the addition of a Go-based entrypoint, updates to the Dockerfile, and new tests for the Go code. The main changes are grouped into Dockerfile updates, Go code additions, and test additions.

Dockerfile Updates:

Updated the Dockerfile.ubuntu2204 to include a multi-stage build process, adding Go installation and setting up the Go workspace. The final image now includes the compiled Go binary and session components. [1] [2]
The first stage, labeled as builder, sets up the environment to compile the Go application and download necessary components.
The second stage, labeled as build, starts with another Ubuntu 22.04 base image. I was originally using the base scratch image here, but this wouldn't work with the goss tests and there is not much difference in final image size. This stage just copies the compiled Go binary and session components from the builder stage.

Go Code Additions:

Added a new main.go file that contains the logic for copying session components based on the session type, validating the target directory, and ensuring the copy operation preserves permissions, times, and owner.
Introduced a go.mod file to manage Go module dependencies for the project.
It uses the otiai10/copy package to handle the copying process, preserving important file attributes and optimizing performance with parallel execution. (I believe hosted used this at one point, or still does).

Test Additions:

Added main_test.go with tests for the getFilesToCopy function and the copyFiles function, ensuring correct behaviour for various session types and verifying file copying operations.

Removal of Old Shell Script:

Removed the old run.sh shell script that previously handled the copying of session components.

Test Configuration Update:

Updated the goss.yaml test configuration to reflect the new entrypoint binary instead of the old shell script.

Testing

Verified that the Docker image builds successfully and includes the compiled Go binary and session components.
Tested the Go program to ensure it correctly copies files and directories while preserving permissions and ownership.
Confirmed that the entrypoint script executes the Go program and logs relevant information for debugging.

just preview-bake workbench-session-init-daily
mkdir init
docker run -v $(pwd)/init:/mnt/init rstudio/workbench-session-init-preview:workbench-session-init-go-cp-jammy-daily

skyeturriff commented 1 week ago

The Release - Build, Test, and Push / Workbench Session Init (pull_request) check is failing as the Get Version step is returning the version as 2024.12.0+439.pro1, which doesn't exist:

Fetch Workbench session init container version

Run WORKBENCH_SESSION_INIT_VERSION=$(just -f ci.Justfile get-version workbench --type=daily --local)
  WORKBENCH_SESSION_INIT_VERSION=$(just -f ci.Justfile get-version workbench --type=daily --local)
  echo "WORKBENCH_SESSION_INIT_VERSION=$WORKBENCH_SESSION_INIT_VERSION" >> $GITHUB_OUTPUT
  shell: /usr/bin/bash -e {0}
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.12.7/x64
    PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.12.7/x64/lib/pkgconfig
    Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.7/x64
    Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.7/x64
    Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.7/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.12.7/x64/lib
./tools/get-version.py workbench --type=daily --local
Swapping product 'workbench' for 'workbench'
Providing version for product: 'workbench' and version type: 'daily'
Got daily version: '2024.12.0+439.pro1'

It should be 2024.12.0+439.pro2. And so the constructed URL is invalid:

Workbench Session Init

#9 [builder 3/8] RUN mkdir -p /pwb-staging &&     RSW_VERSION_URL=$(echo -n "2024.12.0+439.pro1" | sed 's/+/-/g') &&     curl -fsSL -o /pwb-staging/rsp-session-multi-linux.tar.gz "https://s3.amazonaws.com/rstudio-ide-build/session/multi/x86_64/rsp-session-multi-linux-${RSW_VERSION_URL}-x86_64.tar.gz" &&     mkdir -p /opt/session-components &&     tar -C /opt/session-components -xpf /pwb-staging/rsp-session-multi-linux.tar.gz &&     chmod 755 /opt/session-components &&     curl -fsSL -o /pwb-staging/go.tar.gz "https://go.dev/dl/go1.22.2.linux-amd64.tar.gz" &&     tar -C /usr/local -xf /pwb-staging/go.tar.gz &&     rm -rf /pwb-staging
#9 0.150 curl: (22) The requested URL returned error: 404

When I run Get Version step locally, I get the correct version:

❯ just -f ci.Justfile get-version workbench --type=daily --local
./tools/get-version.py workbench --type=daily --local
Swapping product 'workbench' for 'workbench'
Providing version for product: 'workbench' and version type: 'daily'
Got daily version: '2024.12.0+439.pro2'
2024.12.0+439.pro2

ianpittwood commented 1 week ago

The Release - Build, Test, and Push / Workbench Session Init (pull_request) check is failing as the Get Version step is returning the version as 2024.12.0+439.pro1, which doesn't exist:

Fetch Workbench session init container version

Run WORKBENCH_SESSION_INIT_VERSION=$(just -f ci.Justfile get-version workbench --type=daily --local)
  WORKBENCH_SESSION_INIT_VERSION=$(just -f ci.Justfile get-version workbench --type=daily --local)
  echo "WORKBENCH_SESSION_INIT_VERSION=$WORKBENCH_SESSION_INIT_VERSION" >> $GITHUB_OUTPUT
  shell: /usr/bin/bash -e {0}
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.12.7/x64
    PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.12.7/x64/lib/pkgconfig
    Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.7/x64
    Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.7/x64
    Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.7/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.12.7/x64/lib
./tools/get-version.py workbench --type=daily --local
Swapping product 'workbench' for 'workbench'
Providing version for product: 'workbench' and version type: 'daily'
Got daily version: '2024.12.0+439.pro1'

It should be 2024.12.0+439.pro2. And so the constructed URL is invalid:

Workbench Session Init

#9 [builder 3/8] RUN mkdir -p /pwb-staging &&     RSW_VERSION_URL=$(echo -n "2024.12.0+439.pro1" | sed 's/+/-/g') &&     curl -fsSL -o /pwb-staging/rsp-session-multi-linux.tar.gz "https://s3.amazonaws.com/rstudio-ide-build/session/multi/x86_64/rsp-session-multi-linux-${RSW_VERSION_URL}-x86_64.tar.gz" &&     mkdir -p /opt/session-components &&     tar -C /opt/session-components -xpf /pwb-staging/rsp-session-multi-linux.tar.gz &&     chmod 755 /opt/session-components &&     curl -fsSL -o /pwb-staging/go.tar.gz "https://go.dev/dl/go1.22.2.linux-amd64.tar.gz" &&     tar -C /usr/local -xf /pwb-staging/go.tar.gz &&     rm -rf /pwb-staging
#9 0.150 curl: (22) The requested URL returned error: 404

When I run Get Version step locally, I get the correct version:

❯ just -f ci.Justfile get-version workbench --type=daily --local
./tools/get-version.py workbench --type=daily --local
Swapping product 'workbench' for 'workbench'
Providing version for product: 'workbench' and version type: 'daily'
Got daily version: '2024.12.0+439.pro2'
2024.12.0+439.pro2

@skyeturriff I suggest we merge versions into workbench-session-init to help make reruns easier in the future: https://github.com/rstudio/rstudio-docker-products/pull/875

rstudio / rstudio-docker-products