search

DataBiosphere / dsub

Open-source command-line tool to run batch computing tasks and workflows on backend services such as Google Cloud.
Apache License 2.0
265 stars 44 forks source link

RefreshError #194

Closed metanav closed 4 years ago

metanav commented 4 years ago

The same command line worked for months but today I am getting following error:

dsub --provider google-v2 --name test-project --project test0001 --regions asia-northeast1 --machine-type n1-standard-32 --disk-size 2000 --logging gs://test/logs --image some/docker:latest --script script.sh --tasks tasks.tsv --ssh

Job: pacbio-vir--mirai--200501-170028-09 2020-05-01 17:00:28.305799: Caught exception RefreshError 2020-05-01 17:00:28.305799: This request is being retried

metanav commented 4 years ago

After printing the exception in the library code it seems token issue: ('invalid_grant: Invalid JWT: Token must be a short-lived token (60 minutes) and in a reasonable timeframe. Check your iat and exp values in the JWT claim.', '{"error":"invalid_grant","error_description":"Invalid JWT: Token must be a short-lived token (60 minutes) and in a reasonable timeframe. Check your iat and exp values in the JWT claim."}')

mbookman commented 4 years ago

Hi @metanav!

Does the retry succeed or does this end up failing? The bit you pasted indicates that it caught the exception and is retrying.

What is the context in which this is being run? From a laptop/workstation, Cloud Shell, Compute Engine?

metanav commented 4 years ago

It was clock issue. The machine system date/time was off by 5 minutes.