mwvgroup / pittgoogle-user-demos

GNU General Public License v3.0
2 stars 0 forks source link

Containerize and deploy to Cloud Run #6

Closed troyraen closed 2 years ago

troyraen commented 2 years ago

This PR contains the necessary files to deploy the module to Cloud Run and attach a subscription.

Other changes that are included: bug fix for the broker ingest timestamp; switch to loading the schema map from a local file; and add/start a script that performs some one-time project setup tasks.

Example Usage:

  1. Containerize and deploy:
SURVEY="elasticc"
TESTID="-testid"          # note the dash in front
moduledir="classifier"  # assumes you're in the repo's root dir
config="${moduledir}/cloudbuild.yaml"

gcloud builds submit --config="${config}" \
  --substitutions=_SURVEY="${SURVEY}",_TESTID="${TESTID}" \
  "${moduledir}"

Once the service deploys, a service URL will be printed to stdout in your terminal. You will need it below.

  1. Create and attach a push subscription to trigger the module
topic="elasticc-loop"
topic_project="avid-heading-329016"
subscrip="elasticc-loop${TESTID}"
url="<enter the url from the previous step>"
route="/"
runinvoker_svcact="cloud-run-invoker@${GOOGLE_CLOUD_PROJECT}.iam.gserviceaccount.com"

gcloud pubsub subscriptions create "${subscrip}" \
    --topic "${topic}" \
    --topic-project "${topic_project}" \
    --ack-deadline=600 \
    --push-endpoint="${url}${route}" \
    --push-auth-service-account="${runinvoker_svcact}"
troyraen commented 2 years ago

@hernandezc1 I've requested your review on this PR. Look over my changes and let me know if you have any questions/concerns. Once you are satisfied, approve the PR.

Note that your branch (currently main) has some issues in classifier/main.py and classifier/requirements.txt that will prevent this from deploying and running properly. They are things that we have discussed before, so I'll let you find them. But ask me if you get stuck.