ahmetb / cloud-run-faq

Unofficial FAQ and everything you've been wondering about Google Cloud Run.
https://cloud.run
Creative Commons Attribution 4.0 International
2.31k stars 124 forks source link

Running Workloads without WEB on CloudRun #166

Open LMSSonos opened 2 years ago

LMSSonos commented 2 years ago

What is the recommendation to run workloads (containers) which execute a command line tool to do some work (like sending emails, cleaning up data, making calculations, etc) in CloudRun.

ImAnAutie commented 2 years ago

Kinda depends how often you need to run it, is it on a set schedule?

LMSSonos commented 2 years ago

It is set on schedule. We have like a current project where all is in cloudrun but run the jobs (about 20 different jobs) that are scheduled from every 1 minute, to every 1h or once a day. Often it's the same container/Tool but different cli arguments.

sohail288 commented 2 years ago

Seems like this will be a reality pretty soon -- there may be support for "server-less" jobs on cloud run. It's in alpha so they may scrap the functionality. Seems promising though. The only documentation I've found is https://cloud.google.com/sdk/gcloud/reference/alpha/run/jobs and some demos at https://github.com/GoogleCloudPlatform/jobs-demos

LMSSonos commented 2 years ago

@sohail288 - awesome, that was exactly what i was looking for

I just wanted to test it and run into following issue

I tried to run my own container

gcloud alpha run jobs create mytest --image=eu.gcr.io/my-project-name/mycontainer:latest --platform managed --parallelism=1 --region=europe-westX --service-account=mytest-serviceaccount@my-project-name.iam.gserviceaccount.com

Then I get the following exception. I tried to follow https://github.com/GoogleCloudPlatform/jobs-demos/tree/main/screenshot to get it up and running, but failing with the Namespaces instance issue and can't figure out which part I'm missing.

job to Cloud Run service [mytest] in project [my-project-name] region [europe-westX]
X Creating and starting job...
  . Provisioning resources...
Job failed
ERROR: (gcloud.alpha.run.jobs.create) Namespaces instance [my-project-name] not found: Requested entity was not found.
sohail288 commented 2 years ago

@LMSSonos not sure about the namespace error. I was able to run a simple python based docker image using the alpha API without any issue (besides the documentation not matching up with the actual exposed gcloud commands). FYI I was running it in us-central1.

LMSSonos commented 2 years ago

@sohail288 thanks, works now well for me. I received an update for gcloud and with following version it works now as expected.

alpha 2022.04.26
bq 2.0.74
core 2022.04.26
gsutil 5.9

Will start to move some real workload to it ;-)

phoward38 commented 2 years ago

@LMSSonos Another option is using Cloud Scheduler to trigger an endpoint in your Cloud Run service. There's brief documentation about it here.