microsoft / ripgrep-prebuilt

Builds ripgrep on Azure Pipelines for multiple platforms and makes the binaries available as Github releases
MIT License
46 stars 15 forks source link

try to move linux arm gnueahibf onto container #29

Open andreamah opened 1 year ago

andreamah commented 1 year ago

Fixes #24

lex-ibm commented 1 year ago

Looks like the container is running rootless, so we can't install sudo or any of the other packages. From my understanding, we could use the image mentioned in the issue (without having to install any specific packages). But this needs the VSCodeHub registry set up (I'm external, so I don't know if this repo has the registry setup). If this repo has access to VSCodeHub to pull images from there, then we can just have:

container: VSCodeHub/vscode-linux-build-agent:centos7-devtoolset8-x64

No need to change x64 or install anything else.

lex-ibm commented 1 year ago

I'm guessing that's the correct path from https://github.com/microsoft/vscode-linux-build-agent/blob/main/build.yml

andreamah commented 1 year ago

Looks like the container is running rootless, so we can't install sudo or any of the other packages. From my understanding, we could use the image mentioned in the issue (without having to install any specific packages). But this needs the VSCodeHub registry set up (I'm external, so I don't know if this repo has the registry setup). If this repo has access to VSCodeHub to pull images from there, then we can just have:

container: VSCodeHub/vscode-linux-build-agent:centos7-devtoolset8-x64

No need to change x64 or install anything else.

I asked about using that container, and the team preferred that I don't rely on it since it might be changing often. Instead, they proposed that I extract what I need from it and run my own container. That's what I'm trying to do here (with my limited container experience)...

lex-ibm commented 1 year ago

Hmmmm well... you can install nodejs from a binary release, that should not require superuser, but git will.

From what I've read from other issues, it doesn't look like we can install packages on the running container without some more complicated steps. I wrote a PoC using the steps mentioned in the issue above but I don't really feel like that's going a be an accepted solution (too fragile and might change in the future).

I would reconsider using vscode-linux-build-agent:centos7-devtoolset8-x64, or maybe creating another image. Also, as a sidenote, this issue is present in the binary for PowerPC64LE, so it might be in other architectures as well.

andreamah commented 1 year ago

Yeah, I would imagine that this is an issue for any non-musl linux releases. I think that I could maybe install packages like they do in zeromq-prebuilt https://github.com/microsoft/zeromq-prebuilt/blob/55d19504bf58b0ff04c7e17f015ad5b6a826bbf3/build/main.yml#L178-L193, but I'm unsure how to cleanly run everything in that container in this case. Would I need to run everything from a script rather than yml file steps?

lex-ibm commented 1 year ago

Hmmm do you think this will work then?

- job: linux_arm
  pool:
    vmImage: 'ubuntu-latest'
  container:
      image: mcr.microsoft.com/mirror/docker/library/ubuntu:18.04
      options: '--name build-container'
  steps:
    - script: |
        set -ex
        docker exec -t -u root build-container mv /etc/sudoers /etc/sudoers.bak
        docker exec -t -e DEBIAN_FRONTEND=noninteractive -u root build-container apt-get -qq update
        docker exec -t -e DEBIAN_FRONTEND=noninteractive  -u root build-container apt-get -qq install sudo curl git nodejs
        docker exec -t -u root build-container mv /etc/sudoers.bak /etc/sudoers
      displayName: Container Pre-reqs
      target: host
    - template: linux.yml
      parameters:
        target: arm-unknown-linux-gnueabihf

We create the container with the ubuntu:18.04 image and name it build-container, and then on "Container Pre-reqs" we apt update and apt install without changing the /etc/sudoers (this in theory runs docker on the host with target: host). If all succeeds, everything after this runs in the modified container with the packages sudo, git, nodejs, and curl installed.