Refresh indexes with latest projects list

mozilla / participation-metrics-org

Participation metrics planning repository

4 stars 4 forks source link

Refresh indexes with latest projects list #196

Closed canasdiaz closed 5 years ago

canasdiaz commented 5 years ago

The aim of this task is to update the data set with the list of projects provided by the Mozilla folks. The changes affected Github and Bugzilla data sources.

The list of projects is available at https://docs.google.com/spreadsheets/d/1xFrt9JZkZ6m8peasYaRfPO5kC1JDo-vDJJuYfMwcGqM/edit

canasdiaz commented 5 years ago

I'm reviewing the data analyzed for Github (including Git) and Bugzilla in order to make sure we have all the data included in the spreadsheet. A brief update will be provided during Tue morning (CET).

HerminaC commented 5 years ago

Thanks a lot for the update, Luis!

On Mon, Mar 18, 2019 at 5:13 PM Luis Cañas-Díaz notifications@github.com wrote:

I'm reviewing the data analyzed for Github (including Git) and Bugzilla in order to make sure we have all the data included in the spreadsheet. A brief update will be provided during Tue morning (CET).

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/mozilla/participation-metrics-org/issues/196#issuecomment-473982826, or mute the thread https://github.com/notifications/unsubscribe-auth/AHEDbyPHF7hIFLgFuGPKfI6X_EwJ6NWHks5vX7sPgaJpZM4b6Md2 .

-- Staff Program Manager

canasdiaz commented 5 years ago

This is the status for the data sources changed:

Git index:

I've found data is not ready for the repository https://github.com/mozilla/gecko-projects.git. Data is collected from upstream but it is not ready in the enriched index. This repo contains 700K commits. It can be ready by tomorrow morning. The worst case scenario is Thursday morning.
A&T reported they miss two fields in the enriched index. This is going to be investigated. No ETA yet.

Github Index:

We've seen not all the data (issues and PRs) for the project https://github.com/webcompat/web-bugs is ready (we miss 1K), so we will need to re-launch the process for this repository. It can be ready by tomorrow morning.

Bugzilla:

Data is ready now.

This morning I've been talking to A&T and in order to avoid blocking them data will be updated from 6PM to 6AM. The ETA I included above took into account this limitation.

canasdiaz commented 5 years ago

Find below the status.

Git:

Git index contains all the commits expected. This is done.
A&T reported they miss two fields in the index. Those fields are created by a second enrichment which extends the Git index. This is being investigated.

Github:

Github index contains all the issues & PRs expected. This is done.

The updates were executed in the time frame agreed with the A&T team (from 6PM to 6AM CET)

hmitsch commented 5 years ago

Done according to @sanacl. Closing issue.