freelawproject / courtlistener

A fully-searchable and accessible archive of court data including growing repositories of opinions, oral arguments, judges, judicial financial records, and federal filings.
https://www.courtlistener.com
Other
552 stars 151 forks source link

Auto Update Juriscraper etc. #3609

Closed flooie closed 2 months ago

flooie commented 10 months ago

@mlissner

I looked into automating the update for juriscraper etc using dependabot but I think it's either difficult or unwieldy if you want to do it on our own schedule.

I think the simplest and most elegant method would be to implement something like this.

name: PR update Free Law Project dependencies

on:
  workflow_dispatch:

jobs:
  update-juriscraper:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v2

      - name: Set up Python
        uses: actions/setup-python@v2
        with:
          python-version: '3.x'

      - name: Install Poetry
        run: |
          curl -sSL https://install.python-poetry.org | python3 -

      - name: Update juriscraper
        run: |
          poetry update juriscraper reporters-db courts-db eyecite

      - name: Create Pull Request
        uses: peter-evans/create-pull-request@v3
        with:
          token: ${{ secrets.TOKEN }}
          commit-message: Update freelawproject dependencies
          title: Update freelawproject dependencies
          body: This PR updates the juriscraper package.
          branch: update-freelawproject-dependencies

this one would be triggered manually by selecting the action and running it - but we could easily set up a cross repository trigger to run this code after one of the pypi.yml actions was run.

mlissner commented 10 months ago

Per call today, for future readers, we discussed three options of increasing complexity and decreasing human input:

  1. Just have the action in Github and let people run it manually by clicking a button. (We chose this, for now.)

  2. Make it automatic on a cron job in Github Actions.

  3. Make it get a signal from eyecite, juriscraper, etc, so that it runs only when needed.

We also discussed automatic merges of the PRs into CL. We could do it for patch releases, but that seems complicated, so we'll just have to have a human involved. The Auto-merge feature makes this pretty painless.