Update crawler workflow to override Gemfile and Gemfile.lock

[CAMOBAP]

After meeting with IETF Team we agreed that:

• https://github.com/relaton/support/blob/master/.github/workflows/crawler.yml workflow will/can be reused by ietf-tools organization
• The workflow should provide an input to accept external Gemfile/Gemfile.lock
• We need to find some way to avoid crawler.rb duplication

Also things that don't block ietf-tools team directly are:

• Update all relaton-data-* repos on relaton side

@strogonoff feel free to update/comment if I forgot something

[CAMOBAP]

A few PR still left for review but generally crawler workflow is done

• https://github.com/relaton/relaton-data-iec/pull/4
• https://github.com/relaton/relaton-data-iana/pull/3
• https://github.com/relaton/relaton-data-rfcs/pull/2

Now workflow accept override-files input (added in https://github.com/relaton/support/commit/776c9df4ff58391df6794a0bf857512b021d3bf8)

So you can include it as:

name: Crawler

on:
  push:
    branches: [ master, main ]
    tags: [ v* ]
  pull_request:
  schedule:
  - cron: '0 14 * * *'
  workflow_dispatch:

jobs:
  crawl:
    uses: relaton/support/.github/workflows/crawler.yml@master
    with:
      override-files: |
        https://raw.githubusercontent.com/ietf-tools/bibxml-service/main/Gemfile=Gemfile
        https://raw.githubusercontent.com/ietf-tools/bibxml-service/main/Gemfile.lock=Gemfile.lock

@strogonoff feel free to reopen if you have any comment suggestions

[strogonoff]

@stefanomunarini could you take a look at this too?

[stefanomunarini]

• relaton/support#7 move to external workflow relaton-data-iec#4
• relaton/support#7 move to external workflow relaton-data-iana#3
• relaton/support#7 move to external workflow relaton-data-rfcs#2

What about the rest of the relaton-data-* repos? @CAMOBAP

[CAMOBAP]

• relaton/support#7 move to external workflow relaton-data-iec#4
• relaton/support#7 move to external workflow relaton-data-iana#3
• relaton/support#7 move to external workflow relaton-data-rfcs#2

What about the rest of the relaton-data-* repos? @CAMOBAP

Those PRs are merged now, in there is no new data repos created, everything should be up-to-date

P.S. soon I will merge workflow that will monitor newly created repos and notify me to update CI configurations

[stefanomunarini]

Thanks @CAMOBAP .

Does this workflow need to be implemented in relaton-data-ieee (https://github.com/relaton/relaton-data-ieee and https://github.com/ietf-tools/relaton-data-ieee) , relaton-data-nist (https://github.com/relaton/relaton-data-nist), relaton-data-w3c , and all the other data repositories?

And.. when are these workflow ready to be merged into ietf-tools relaton-data-* repos?

Thanks

[CAMOBAP]

@stefanomunarini Sorry for ignore

Does this workflow need to be implemented in relaton-data-ieee (https://github.com/relaton/relaton-data-ieee and https://github.com/ietf-tools/relaton-data-ieee) , relaton-data-nist (https://github.com/relaton/relaton-data-nist), relaton-data-w3c , and all the other data repositories?

AFAIK it's already done for all data repos

And.. when are these workflow ready to be merged into ietf-tools relaton-data-* repos?

Because in ietf-tools you have a requirement to use specific Gemfile you probably should not merge from relaton but use the example which I provided in https://github.com/relaton/support/issues/4#issuecomment-1497095045