search

iterative / cml

♾️ CML - Continuous Machine Learning | CI/CD for ML
http://cml.dev
Apache License 2.0
4k stars 338 forks source link

Unable to create a report in a gitlab runner with a custom python docker image #1392

Closed NicoLivesey closed 1 year ago

NicoLivesey commented 1 year ago

Hi I get an error when I try to create a report from a gitlab runner on a custom Docker image.

TO REPRODUCE (in a gitlab ci yaml):

image: python:3.9.14-slim-bullseye
script:
  - curl -sL https://deb.nodesource.com/setup_16.x | bash
  - apt-get update && apt-get install -y nodejs
  - npm i -g @dvcorg/cml
  - export REPO_TOKEN=...
  - pip install --no-cache-dir poetry==1.5.0
  - poetry install
  - DOING SOME THINGS
  - cml comment create report.md

When the CI triggers I get this error {"level":"error","message":"Cannot read properties of null (reading 'split')","stack":"TypeError: Cannot read properties of null (reading 'split')\n at addSafeDirectory (/usr/lib/node_modules/@dvcorg/cml/src/cml.js:102:7)\n at fixGitSafeDirectory (/usr/lib/node_modules/@dvcorg/cml/src/cml.js:110:3)\n at new CML (/usr/lib/node_modules/@dvcorg/cml/src/cml.js:125:5)\n at setupOpts (/usr/lib/node_modules/@dvcorg/cml/bin/cml.js:72:14)\n at /usr/lib/node_modules/@dvcorg/cml/node_modules/yargs/build/index.cjs:1:4778\n at Array.reduce (<anonymous>)\n at C (/usr/lib/node_modules/@dvcorg/cml/node_modules/yargs/build/index.cjs:1:4576)\n at _.handleValidationAndGetResult (/usr/lib/node_modules/@dvcorg/cml/node_modules/yargs/build/index.cjs:1:8966)\n at _.applyMiddlewareAndGetResult (/usr/lib/node_modules/@dvcorg/cml/node_modules/yargs/build/index.cjs:1:9604)\n at _.runCommand (/usr/lib/node_modules/@dvcorg/cml/node_modules/yargs/build/index.cjs:1:7231)"}

Already open question: https://stackoverflow.com/questions/72745070/cml-send-report-is-giving-cannot-read-properties-of-null-reading-split

dacbd commented 1 year ago

@NicoLivesey can you try the prebuilt binary for cml in your container image?

In your docker image / dockerfile I would add something to the effect of:

mkdir -p /opt/cml/
curl --location --url https://github.com/iterative/cml/releases/download/v0.19.1/cml-linux --output /opt/cml/cml-linux
chmod +x /opt/cml/cml-linux
ln -s /opt/cml/cml-linux /usr/bin/cml
NicoLivesey commented 1 year ago

Hey thanks a lot for your response. I get the exact same error with the prebuilt binary ! 

{"level":"error","message":"Cannot read properties of null (reading 'split')","stack":"TypeError: Cannot read properties of null (reading 'split')
    at addSafeDirectory (/snapshot/cml/src/cml.js:102:7)
    at fixGitSafeDirectory (/snapshot/cml/src/cml.js:110:3)
    at new CML (/snapshot/cml/src/cml.js:125:5)
    at setupOpts (/snapshot/cml/bin/cml.js:72:14)
    at /snapshot/cml/node_modules/yargs/build/index.cjs:1:4778
    at Array.reduce (<anonymous>)
    at C (/snapshot/cml/node_modules/yargs/build/index.cjs:1:4576)
    at M.handleValidationAndGetResult (/snapshot/cml/node_modules/yargs/build/index.cjs:1:8864)
    at M.applyMiddlewareAndGetResult (/snapshot/cml/node_modules/yargs/build/index.cjs:1:9502)
    at M.runCommand (/snapshot/cml/node_modules/yargs/build/index.cjs:1:7231)"}
NicoLivesey commented 1 year ago

The gitlab file looks like this now

image: python:3.9.14-slim-bullseye
script:
  - export REPO_TOKEN=...
  - mkdir -p /opt/cml/
  - curl --location --url https://github.com/iterative/cml/releases/download/v0.19.1/cml-linux --output /opt/cml/cml-linux
  - chmod +x /opt/cml/cml-linux
  - ln -s /opt/cml/cml-linux /usr/bin/cml
  - pip install --no-cache-dir poetry==1.5.0
  - poetry install
  - DOING SOME THINGS
  - cml comment create report.md
dacbd commented 1 year ago

Thanks, I'll do some investigating.

On Mon, Jun 26, 2023, 14:48 Nicolas Henry @.***> wrote:

The gitlab file looks like this now

image: python:3.9.14-slim-bullseye script:

— Reply to this email directly, view it on GitHub https://github.com/iterative/cml/issues/1392#issuecomment-1608333908, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIN7M73YNYEDPXG6KZP3ILXNH7RNANCNFSM6AAAAAAZRYZMUQ . You are receiving this because you commented.Message ID: @.***>

dacbd commented 1 year ago

In your - DOING SOME THINGS do you navigate out of the git repository?

NicoLivesey commented 1 year ago

Hmm no but in - DOING SOME THINGS DVC is initialized with no-scm, maybe this is linked ?

dacbd commented 1 year ago

I don't think so, there error if coming from some code that is trying to set the git safe directory.

I just checked the container, it does not have git installed which I think is the source of this issue. Can you try adding it? add-apt-repository ppa:git-core/ppa --yes && apt-get update && apt-get install --yes git

NicoLivesey commented 1 year ago

Ok this is it ! Thanks a lot !