github_importer.rb could probably benefit from using GraphQL

[bfabio]

github_importer.rb right now uses the GitHub REST APIs and takes about 2 minutes to complete.

We can probably reduce that time by using the GitHub GraphQL APIs instead, especially in the fetch_teams and fetch_issues methods.

Context. What's github_importer.rb?

github_importer.rb is a script that gets executed periodically and it downloads data from GitHub resulting in the following files getting committed to this repo:

• github_issues.json, loaded by Javascript at https://developers.italia.it/en/to-do (and https://developers.italia.it/it/cosa-fare)

• github_teams.yml, github_members.yml, github_tech_list.yml, used by Jekyll in italia/developers.italia.it to build:

  • The team members in each Platform page (eg. https://developers.italia.it/en/anpr/)
  • The contributors at https://developers.italia.it/en/contacts
  • The list of technologies at https://developers.italia.it/en/to-do

[maurovitale]

Hi @bfabio, I want to try to solve this issue but I need the GITHUB_ACCESS_TOKEN to test the ruby script. Any way to obtain it?

[bfabio]

@maurovitale We need to invite you to the organization, so you can create your own token. cc @sebbalex

[sebbalex]

Hi @maurovitale I sent you an invitation for the org /italia please accept it. Tnx

[maurovitale]

Hi @sebbalex, thx for the invitation I accepted and I'll try to solve this issue. I update you soon!

[maurovitale]

Hi @sebbalex @bfabio , some updates about this problem. I'm using the github explorer to interrogate the graphql github endpoint and it's quite easy to obtain the data. I created a ruby test script on the project (that I will integrate to github_importer.rb, once is finished) to test the requests; to do so, I'm using the 'gqli' lib.

[bfabio]

github_importer.rb is now in this repo (italia/developers.italia.it#1045)