Open atvaccaro opened 1 year ago
I'm not sure whether it is new since the Cal-ITP GTFS Hosting doc was written, but GCP has a set of steps for this at https://cloud.google.com/storage/docs/hosting-static-website#command-line_1. The general steps above still apply, but that documentation is useful for the specific gcloud
commands.
[ ] Create a new bucket and configure it as a GCP-hosted static site behind a load balancer; see the GCS docs
Looking through the existing buckets (for naming patterns), there are three that are public:
calitp-map-tiles
which (I hope) contains map tilescalitp-publish-data-analysis
which contains downloadable data assets that are useful for analysiscalitp-metabase-data-public
which contains GeoJSON boundary files for metabase to use in aggregationsI'll stick to the calitp-
prefixing and call this one calitp-data-analyses-portfolio
. Note that a pattern that some use is to use the domain name as the bucket name (or as a subset of the bucket name), and this does not follow that pattern.
Command:
gcloud storage buckets create calitp-data-analyses-portfolio \
--project=cal-itp-data-infra \
--location=us-west2
gcloud storage buckets add-iam-policy-binding gs://calitp-data-analyses-portfolio \
--member=allUsers \
--role=roles/storage.objectViewer
[ ] Create a GCP role and a service account that can be used to deploy from GH Actions
This role should be set up similar to the GTFS flex hosting IAM configuration. Ideally these would be configured as IAC with terraform in the data-infra repo.
ID projects/cal-itp-data-infra/roles/CustomGCSPublisher
8 assigned permissions
resourcemanager.projects.get
storage.buckets.get
storage.buckets.list
storage.objects.create
storage.objects.delete
storage.objects.get
storage.objects.list
storage.objects.update
Command:
gcloud iam roles create CustomGCSPublisher \
--project=cal-itp-data-infra \
--title="Custom GCS Publisher" \
--description="Custom role for publishing to GCS" \
--permissions=resourcemanager.projects.get,storage.buckets.get,storage.buckets.list,storage.objects.create,storage.objects.delete,storage.objects.get,storage.objects.list,storage.objects.update
Email gh-actions-publisher@cal-itp-data-infra.iam.gserviceaccount.com
Assign Role projects/cal-itp-data-infra/roles/CustomGCSPublisher
Command:
gcloud iam service-accounts create gh-actions-publisher \
--project=cal-itp-data-infra \
--description="Service account for GH Actions to publish to GCS" \
--display-name="GH Actions Publisher"
gcloud iam service-accounts keys create ~/key.json \
--iam-account gh-actions-publisher@cal-itp-data-infra.iam.gserviceaccount.com
[ ] Add the key to the GH repo secrets
GCS_BUCKET
: calitp-data-analyses-portfolio
GCS_CREDENTIALS
: cat ~/key.json | base64
[ ] Change portfolio.py
to write to the proper subpaths in the bucket for individual sites; this would involve replacing netlify deploy
with gcloud storage cp
[ ] (Potential) May have to configure JupyterBook to render links etc. under a subpath
[ ] Remove _redirects
and modify index.html
if needed to function properly as the bucket home page
[ ] (Optional) Configure CI to allow deploys with a button (i.e. workflow dispatches) for at least the index
@mjumbewu once you have GCS credentials set up, let's make sure there's a runbook about GitHub GCS credential rotation as part of acceptance. Maybe this is something to pair on?
We briefly discussed future plans for the portfolio site awhile back (e.g. https://github.com/cal-itp/data-analyses/issues/715) and we decided it's time to finally start exploring this migration. We've struggled with netlify recently (specifically deploying individual sites under redirects) and we don't have paid support, so it's probably worthwhile to just migrate to a static site served from a GCS bucket.
The general steps are:
portfolio.py
to write to the proper subpaths in the bucket for individual sites; this would involve replacingnetlify deploy
withgsutil
orgcsfs
_redirects
and modifyindex.html
if needed to function properly as the bucket home page