ossf / Project-Security-Metrics

Collect, curate, and communicate relevant security metrics for open source projects.
https://openssf.org
Apache License 2.0
63 stars 24 forks source link

Convert Scorecard loading from "bq" to the BigQuery API #54

Open scovetta opened 3 years ago

scovetta commented 3 years ago

Currently, we're running a series of BigQuery commands (bq query, bq extract, and gsutil cp) to obtain the scorecard data dump in order to reload the database.

This requires an account to be logged in locally (using gcloud auth login). The account itself was created explicitly for this purpose, but it still isn't ideal. I also don't know how long until the authorization expires and the job would break.

I think by using the API, we can use a token of some sort. I'm not super familiar with the capabilities here. We should have someone look into this and convert the scorecard import job accordingly.