iterative / example-repos-dev

Source code and generator scripts for example DVC projects
https://dvc.org/doc
21 stars 13 forks source link

experiments: add GHA workflow with CML and Studio Live metrics #168

Closed alex000kim closed 1 year ago

alex000kim commented 1 year ago

run-studio-experiments.yaml:

name: CML
on: 
  push:
    paths:
      - ".github/workflows/run-experiment.yaml"
      - "data/**"
      - "src/**"
      - "params.yaml"
      - "dvc.*"
jobs:
  deploy-runner:
    if: github.actor == 'iterative-studio[bot]'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: iterative/setup-cml@v1
      - uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-region: us-east-1
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
      - name: Create Runner
        env:
          REPO_TOKEN: ${{ secrets.PERSONAL_ACCESS_TOKEN }}
        run: |
          cml runner launch --single \
            --labels=cml \
            --cloud=aws \
            --cloud-region=us-east \
            --cloud-hdd-size=40 \
            --cloud-type=g5.2xlarge
  runner-job:
    needs: deploy-runner
    runs-on: [ self-hosted, cml ]
    container:
      image: iterativeai/cml:0-dvc2-base1-gpu
      options: --gpus all --ipc host
    steps:
      - uses: actions/checkout@v3
      - uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-region: us-east-1
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
      - name: training
        env:
          REPO_TOKEN: ${{ secrets.PERSONAL_ACCESS_TOKEN }}
          STUDIO_TOKEN: ${{ secrets.STUDIO_TOKEN }}
          DVCLIVE_LOGLEVEL: DEBUG
        run: |
          pip install -r requirements.txt
          dvc pull
          cml comment create --publish --watch results/train/report.md & dvc repro
          dvc push
          cml pr --squash --skip-ci .
          cml comment create --pr --publish results/evaluate/report.md
daavoo commented 1 year ago

@alex000kim This is currently added in the P.R. but needs to tweak the credentials and stuff in the rendered repo.

I tried to use the sandbox role but didn't debug the failure:

https://github.com/iterative/example-get-started-experiments/actions/runs/4057404190/jobs/6983119575

alex000kim commented 1 year ago

Oh, right, I forgot about that. Closing the issue, then.

alex000kim commented 1 year ago

I think this PR should resolve this: https://github.com/iterative/itops/pull/905