openfoodfacts / openfoodfacts-server

Open Food Facts database, API server and web interface - 🐪🦋 Perl, CSS and JS coders welcome 😊 For helping in Python, see Robotoff or taxonomy-editor
http://openfoodfacts.github.io/openfoodfacts-server/
GNU Affero General Public License v3.0
660 stars 389 forks source link

Automatic product refresh after taxonomy deployment #11011

Open teolemon opened 1 week ago

teolemon commented 1 week ago

Problem

Pseudo code

name: Gather Changed Entries and Run Script

on:
  push:
    tags:
      - 'v*.*.*'

jobs:
  gather-changes:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout repository
      uses: actions/checkout@v3

    - name: Get the latest two tags
      id: get-tags
      run: |
        latest_tag=$(git describe --tags `git rev-list --tags --max-count=1`)
        previous_tag=$(git describe --tags `git rev-list --tags --skip=1 --max-count=1`)
        echo "::set-output name=latest_tag::$latest_tag"
        echo "::set-output name=previous_tag::$previous_tag"

    - name: Gather changes
      id: gather-changes
      run: |
        git diff ${{ steps.get-tags.outputs.previous_tag }} ${{ steps.get-tags.outputs.latest_tag }} -- taxonomies/ > changes.diff
        grep -E '^\+\s*.*en:' changes.diff > filtered_changes.txt
        tags=$(grep -o 'en:[^,]*' filtered_changes.txt | sort | uniq)
        echo "::set-output name=tags::$tags"

    - name: Run update_all_products.pl for each tag
      run: |
        for tag in ${{ steps.gather-changes.outputs.tags }}
        do
          ./scripts/update_all_products.pl --query categories_tags=en:some-old-entry-that-is-now-a-synonym!
        done