search

wandb / server

W&B Server is the self hosted version of Weights & Biases
MIT License
263 stars 21 forks source link

system-admin -- Application Error Unable to reach the backend api. If this perists, check the system logs. #46

Open DDDlk opened 3 years ago

DDDlk commented 3 years ago

when i enter the Frontend Host image so i can not enter the license.

vanpelt commented 3 years ago

@DDDlk are you able to select "Debug Bundle" from the menu in the upper right corner of the /system-admin page and share it here or directly to vanpelt@wandb.com?

PeterJaq commented 3 years ago

/system-admin

1637736246(1)

I also meet the same problem, when I apply for the licence and click the update setting button in /system-admin page. This application error appeared.

tresiwald commented 3 years ago

Same problem here. Please find the debug bundle: debug.zip

GuodongQi commented 3 years ago

same here

MPGek commented 3 years ago

Same for me, on 0.9.45 and on 0.9.46.

vanpelt commented 2 years ago

@MPGek @GuodongQi can you share the debug bundles as described in my comment. @tresiwald unfortunately that debug file is corrupted and I can't read it. Feel free to send them directly to vanpelt@wandb.com

GuodongQi commented 2 years ago

@MPGek @GuodongQi can you share the debug bundles as described in my comment. @tresiwald unfortunately that debug file is corrupted and I can't read it. Feel free to send them directly to vanpelt@wandb.com

I have tried to generated it many times, and the debug file is still corrupted too. debug.zip

vanpelt commented 2 years ago

@GuodongQi can you exec into the container and share the following logs?

docker exec -it wandb-local bash
cat /var/logs/local.log
cat /var/logs/gorilla.log
littlerookie commented 2 years ago

@vanpelt I meet the same problem, /var/log/local.log

127.0.0.1 - - [09/Dec/2021:10:05:24 +0000] "GET / HTTP/1.1" 200 2283 127.0.0.1 - - [09/Dec/2021:10:05:24 +0000] "GET /static/css/2.19edfb45.chunk.css HTTP/1.1" 200 593 127.0.0.1 - - [09/Dec/2021:10:05:24 +0000] "GET /static/css/main.fc582b62.chunk.css HTTP/1.1" 200 592606 127.0.0.1 - - [09/Dec/2021:10:05:25 +0000] "GET /static/js/2.6b703198.chunk.js HTTP/1.1" 200 634272 127.0.0.1 - - [09/Dec/2021:10:05:25 +0000] "GET /static/js/main.39a4926a.chunk.js HTTP/1.1" 200 22575 127.0.0.1 - - [09/Dec/2021:10:05:26 +0000] "GET /static/css/2.19edfb45.chunk.css.map HTTP/1.1" 200 1806 127.0.0.1 - - [09/Dec/2021:10:05:26 +0000] "GET /static/css/main.fc582b62.chunk.css.map HTTP/1.1" 200 823163 127.0.0.1 - - [09/Dec/2021:10:05:26 +0000] "GET /static/js/2.6b703198.chunk.js.map HTTP/1.1" 200 2743241 127.0.0.1 - - [09/Dec/2021:10:05:26 +0000] "GET /static/js/main.39a4926a.chunk.js.map HTTP/1.1" 200 57565 127.0.0.1 - - [09/Dec/2021:10:05:26 +0000] "GET /api/env HTTP/1.1" 200 2125

/var/log/gorilla.log

`{"level":"INFO","time":"2021-12-09T10:17:20.206222467Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":75},"data":{"requestID":"ebbe6c70-dc08-4625-a44c-bcfc6e56426c","path":"/oidc/auth"},"message":"Starting request ebbe6c70-dc08-4625-a44c-bcfc6e56426c on /oidc/auth","logging.googleapis.com/trace":""}

{"level":"INFO","time":"2021-12-09T10:17:20.206818669Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/session.go:134","pid":75},"data":{"request":"/oidc/auth","userAgent":"Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36","domain":"my_ip","method":"GET","origin":"http://my_ip:8080"},"message":"Request origin not allowed: http://my_ip:8080","logging.googleapis.com/trace":""}

{"level":"INFO","time":"2021-12-09T10:17:20.20865296Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":75},"data":{"requestID":"ebbe6c70-dc08-4625-a44c-bcfc6e56426c","duration":"2.478743ms","path":"/oidc/auth"},"message":"Finished request ebbe6c70-dc08-4625-a44c-bcfc6e56426c in %!f(time.Duration=2478743) seconds on /oidc/auth","logging.googleapis.com/trace":""}`

screenshot: image

GuodongQi commented 2 years ago

Same as @littlerookie, except that the port is changed to 8083. image

vanpelt commented 2 years ago

Ahh, I see what's happening. When you're accessing the instance not from 127.0.0.1:8080 you'll need to tell the instance how you'll be accessing it. You can do this by specifying the HOST environment variable on boot, i.e.

docker stop wandb-local
wandb local -e HOST=http://10.13.71.162:8083

Or by setting the Host value on the settings page from 127.0.0.1 and saving the settings.

GuodongQi commented 2 years ago

Solved! Thanks!

littlerookie commented 2 years ago

Ahh, I see what's happening. When you're accessing the instance not from 127.0.0.1:8080 you'll need to tell the instance how you'll be accessing it. You can do this by specifying the HOST environment variable on boot, i.e.

docker stop wandb-local
wandb local -e HOST=http://10.13.71.162:8083

Or by setting the Host value on the settings page from 127.0.0.1 and saving the settings.

@vanpelt I use docker command with default port 8080 directly deploy the wandb-local container in my server docker run --rm -d -v /wandb/data:/vol -p 8080:8080 --name wandb-local wandb/local and use my computer to access the server_ip:8080, so what should I set?

vanpelt commented 2 years ago

@littlerookie just add -e HOST=http://server_ip:8080 to your docker run command.

littlerookie commented 2 years ago

It works for me. Great Thanks.

Lionelsy commented 2 years ago

Excuse me. I just met the same problem as above when I wanted to set up the license. (run the docker with wandb local --port 8960)

Screen Shot 2022-03-15 at 11 34 43 PM

I just followed the operations you mentioned, but it seemed that it didn't work :(

docker stop wandb-local
wandb local -e HOST=http://10.20.84.191:8960

What's more, the running port is 8080 instead of 8960. The debug.zip was corrupted though I tried to download it many times.

Screen Shot 2022-03-16 at 12 03 32 AM

Looking forward to your kindly reply.

/var/log/local.log :

(base) shuyu@scai-g3:~$ docker exec -it wandb-local bash
wandb@38646080d6c7:~$ cat /var/log/local.log
2022/03/15 16:03:23 Server Listening on Port 8083
127.0.0.1 - - [15/Mar/2022:16:11:20 +0000] "GET / HTTP/1.1" 200 2283
127.0.0.1 - - [15/Mar/2022:16:11:20 +0000] "GET /static/css/2.19edfb45.chunk.css HTTP/1.1" 200 593
127.0.0.1 - - [15/Mar/2022:16:11:20 +0000] "GET /static/css/main.fc582b62.chunk.css HTTP/1.1" 200 592606
127.0.0.1 - - [15/Mar/2022:16:11:20 +0000] "GET /static/js/main.b8480399.chunk.js HTTP/1.1" 200 24349
127.0.0.1 - - [15/Mar/2022:16:11:20 +0000] "GET /static/js/2.39401b79.chunk.js HTTP/1.1" 200 634280
127.0.0.1 - - [15/Mar/2022:16:11:22 +0000] "GET /api/env HTTP/1.1" 200 2237
wandb@38646080d6c7:~$ cat /var/log/gorilla.log
{"level":"INFO","time":"2022-03-15T16:03:36.771768597Z","info":{"program":"gorilla","source":"gorilla/main.go:379","pid":62},"data":{"config":"{Port:8081,MetadataStore:[REDACTED],MetadataStoreReplica:[REDACTED],MetadataStoreReplicaRate:0,FastSummaryMetricsRate:0,HistoryStore:[REDACTED],FileStore:s3://tazQbvV0trE5LuOW4TMxhg==:rgBwKG+xd5QHWovuh4EocgEnunotTfOBg6sWPMaDz70=@127.0.0.1:9000/local-files,FileStoreIsProxied:true,FileStreamStore:{Address:mysql://wandb_local:wandb_local@127.0.0.1:3306/wandb_local MySQL:{DialTimeout:0s ReadTimeout:0s WriteTimeout:0s MaxIdleConns:0 MaxOpenConns:0 ConnMaxLifetime:0s} BucketURL:},EmailSink:sendgrid://null,EmailFromAddress:noreply@wandb.com,SweepProvider:http://127.0.0.1:8082,SweepProviderShadow:,SettingsCache:redis://127.0.0.1:6379,MetadataCache:noop://,Cache:noop://,Limiter:redis://127.0.0.1:6379,Locker:redis://127.0.0.1:6379,Tracer:noop://,TaskQueue:noop://,NotificationsQueue:,BenchmarkSubmissionNotifier:noop://,DataFrameStore:noop://,RepoInsightsPlotStore:noop://,TrendingRepoStore:noop://,OIDCClientID:,OIDCIssuer:,OIDCAuthMethod:,OIDCAuthorizePath:,AuthCookie:wandb,FrontendHost:http://10.20.84.191:8960,FileHost:http://10.20.84.191:8960,StaticHost:http://10.20.84.191:8960,CORSOrigins:,Dev:false,InsideDocker:false,Secret:[REDACTED],SessionKey:[REDACTED],SessionLength:720h0m0s,Statsd:{Host:127.0.0.1 Port:0},GCPProject:dev~wandb-local,GoogleApplicationCredentials:,StorageBucket:s3://local-files,ArtifactBucketRegions:,DefaultRegion:minio-local,SentryDSN:,SentryEnvironment:onprem-local,SendgridAPIKey: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,ServicesHost:,ServicesSecret:[REDACTED],SlackClientID:,SlackSecret:[REDACTED],StripeKey:[REDACTED],StripeSigningSecret:[REDACTED],DiscourseSSOSecret:[REDACTED],GitHubOAuthClientID:,GitHubOAuthSecret:[REDACTED],GitHubAppInstallationID:0,GitHubAppIntegrationID:0,AnalyticsSink:mysql://wandb_local:wandb_local@127.0.0.1:3306/wandb_local,AuthJWKURL:http://127.0.0.1:8083/api/jwks.json,Auth0Domain:,Auth0ClientID:,Auth0MgmtClientID:,Auth0MgmtSecret:[REDACTED],AuthAdvisorySecret:[REDACTED],GoogleIapAudience:,TrustedProxyEmailHeader:,InsecureAllowImpersonation:false,InsecureAllowAPIKeyAdminAccess:false,DisableSSOProvisioning:false,License:,LicenseKeyPath:,LicenseCertPath:/var/app/jwks.json,AzureAccountKey:,Onprem:true,OnpremAPIKeyPrefix:local,MySQL:{DialTimeout:0s ReadTimeout:0s WriteTimeout:0s MaxIdleConns:0 MaxOpenConns:0 ConnMaxLifetime:0s},DefaultRateLimits:{FileStream:0 GraphQL:0},PaidRateLimits:{FileStream:0 GraphQL:0},PProfAddr:,MinCliVersion:0.8.20,MaxCliVersion:0.12.10,FileMetadataSource:redis://127.0.0.1:6379/filemetadata,FileMetadataSourceIsInternal:false,UseRunQueueForSweeps:0,DisableTelemetry:false,HeapProfileInterval:5m0s,HeapProfileThreshold:4294967296,PrintVersionAndExit:false,SlatePath:/usr/local/bin/slate-linux,SecretsStoreSource:noop://}"},"message":"Running with config {Port:8081,MetadataStore:[REDACTED],MetadataStoreReplica:[REDACTED],MetadataStoreReplicaRate:0,FastSummaryMetricsRate:0,HistoryStore:[REDACTED],FileStore:s3://tazQbvV0trE5LuOW4TMxhg==:rgBwKG+xd5QHWovuh4EocgEnunotTfOBg6sWPMaDz70=@127.0.0.1:9000/local-files,FileStoreIsProxied:true,FileStreamStore:{Address:mysql://wandb_local:wandb_local@127.0.0.1:3306/wandb_local MySQL:{DialTimeout:0s ReadTimeout:0s WriteTimeout:0s MaxIdleConns:0 MaxOpenConns:0 ConnMaxLifetime:0s} BucketURL:},EmailSink:sendgrid://null,EmailFromAddress:noreply@wandb.com,SweepProvider:http://127.0.0.1:8082,SweepProviderShadow:,SettingsCache:redis://127.0.0.1:6379,MetadataCache:noop://,Cache:noop://,Limiter:redis://127.0.0.1:6379,Locker:redis://127.0.0.1:6379,Tracer:noop://,TaskQueue:noop://,NotificationsQueue:,BenchmarkSubmissionNotifier:noop://,DataFrameStore:noop://,RepoInsightsPlotStore:noop://,TrendingRepoStore:noop://,OIDCClientID:,OIDCIssuer:,OIDCAuthMethod:,OIDCAuthorizePath:,AuthCookie:wandb,FrontendHost:http://10.20.84.191:8960,FileHost:http://10.20.84.191:8960,StaticHost:http://10.20.84.191:8960,CORSOrigins:,Dev:false,InsideDocker:false,Secret:[REDACTED],SessionKey:[REDACTED],SessionLength:720h0m0s,Statsd:{Host:127.0.0.1 Port:0},GCPProject:dev~wandb-local,GoogleApplicationCredentials:,StorageBucket:s3://local-files,ArtifactBucketRegions:,DefaultRegion:minio-local,SentryDSN:,SentryEnvironment:onprem-local,SendgridAPIKey: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,ServicesHost:,ServicesSecret:[REDACTED],SlackClientID:,SlackSecret:[REDACTED],StripeKey:[REDACTED],StripeSigningSecret:[REDACTED],DiscourseSSOSecret:[REDACTED],GitHubOAuthClientID:,GitHubOAuthSecret:[REDACTED],GitHubAppInstallationID:0,GitHubAppIntegrationID:0,AnalyticsSink:mysql://wandb_local:wandb_local@127.0.0.1:3306/wandb_local,AuthJWKURL:http://127.0.0.1:8083/api/jwks.json,Auth0Domain:,Auth0ClientID:,Auth0MgmtClientID:,Auth0MgmtSecret:[REDACTED],AuthAdvisorySecret:[REDACTED],GoogleIapAudience:,TrustedProxyEmailHeader:,InsecureAllowImpersonation:false,InsecureAllowAPIKeyAdminAccess:false,DisableSSOProvisioning:false,License:,LicenseKeyPath:,LicenseCertPath:/var/app/jwks.json,AzureAccountKey:,Onprem:true,OnpremAPIKeyPrefix:local,MySQL:{DialTimeout:0s ReadTimeout:0s WriteTimeout:0s MaxIdleConns:0 MaxOpenConns:0 ConnMaxLifetime:0s},DefaultRateLimits:{FileStream:0 GraphQL:0},PaidRateLimits:{FileStream:0 GraphQL:0},PProfAddr:,MinCliVersion:0.8.20,MaxCliVersion:0.12.10,FileMetadataSource:redis://127.0.0.1:6379/filemetadata,FileMetadataSourceIsInternal:false,UseRunQueueForSweeps:0,DisableTelemetry:false,HeapProfileInterval:5m0s,HeapProfileThreshold:4294967296,PrintVersionAndExit:false,SlatePath:/usr/local/bin/slate-linux,SecretsStoreSource:noop://}","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:03:36.772088218Z","info":{"program":"gorilla","source":"gorilla/main.go:384","pid":62},"data":{},"message":"created tracer","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:03:36.772128312Z","info":{"program":"gorilla","source":"gorilla/main.go:398","pid":62},"data":{},"message":"initialized sentry","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:03:36.772331021Z","info":{"program":"gorilla","source":"gorilla/main.go:410","pid":62},"data":{},"message":"read schema","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:03:36.821111638Z","info":{"program":"gorilla","source":"gorilla/main.go:460","pid":62},"data":{},"message":"created metadata store","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:03:36.821193656Z","info":{"program":"gorilla","source":"gorilla/main.go:461","pid":62},"data":{},"message":"created audit store","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:03:36.821229118Z","info":{"program":"gorilla","source":"gorilla/main.go:468","pid":62},"data":{},"message":"created dataframe store","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:03:36.821252278Z","info":{"program":"gorilla","source":"gorilla/main.go:478","pid":62},"data":{},"message":"created repo insights plot store","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:03:36.821394139Z","info":{"program":"gorilla","source":"gorilla/main.go:510","pid":62},"data":{},"message":"created file store","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:03:36.821504778Z","info":{"program":"gorilla","source":"gorilla/main.go:514","pid":62},"data":{},"message":"created artifacts bucket","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:03:36.824944262Z","info":{"program":"gorilla","source":"gorilla/main.go:528","pid":62},"data":{},"message":"created history store","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:03:36.825030101Z","info":{"program":"gorilla","source":"gorilla/main.go:537","pid":62},"data":{},"message":"created task executor","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:03:36.827940133Z","info":{"program":"gorilla","source":"gorilla/main.go:549","pid":62},"data":{},"message":"created file stream store","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:03:36.828022288Z","info":{"program":"gorilla","source":"gorilla/main.go:554","pid":62},"data":{},"message":"created email sink","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:03:36.831422582Z","info":{"program":"gorilla","source":"gorilla/main.go:559","pid":62},"data":{},"message":"created analytics sink","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:03:36.831515906Z","info":{"program":"gorilla","source":"gorilla/main.go:583","pid":62},"data":{},"message":"created notifier","logging.googleapis.com/trace":""}
sweep provider is: http://127.0.0.1:8082
http://127.0.0.1:8082
{"level":"INFO","time":"2022-03-15T16:03:36.831593236Z","info":{"program":"gorilla","source":"gorilla/main.go:611","pid":62},"data":{},"message":"created sweep provider","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:03:36.832059692Z","info":{"program":"gorilla","source":"gorilla/main.go:616","pid":62},"data":{},"message":"created locker","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:03:36.832133693Z","info":{"program":"gorilla","source":"gorilla/main.go:635","pid":62},"data":{},"message":"created auth store","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:03:36.832471031Z","info":{"program":"gorilla","source":"gorilla/main.go:640","pid":62},"data":{},"message":"sweep run queues enabled: %!t(float64=0)","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:03:36.833033706Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/redis/cache.go:116","pid":62},"data":{},"message":"Error retrieving value for gorilla.replicaReadRate, using default: defaultValue=0, error=key does not exist","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:03:36.833188066Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/redis/cache.go:116","pid":62},"data":{},"message":"Error retrieving value for gorilla.fastSummaryMetricsRate, using default: defaultValue=0, error=key does not exist","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:03:36.88059627Z","info":{"program":"gorilla","source":"gorilla/main.go:724","pid":62},"data":{},"message":"parsed schema","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:03:36.882786589Z","info":{"program":"gorilla","source":"gorilla/main.go:881","pid":62},"data":{},"message":"created session store","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:03:36.882828301Z","info":{"program":"gorilla","source":"gorilla/main.go:891","pid":62},"data":{},"message":"created secret store","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:03:36.882866187Z","info":{"program":"gorilla","source":"gorilla/main.go:964","pid":62},"data":{"port":"8081"},"message":"Listening on 8081","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:04:00.797922247Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"50442aa2-0117-4fa1-8c80-f6b6c30682dc","path":"/admin/build_artifact_deadlists"},"message":"Starting request 50442aa2-0117-4fa1-8c80-f6b6c30682dc on /admin/build_artifact_deadlists","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:04:00.798225006Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"3edc2591-7553-4082-98df-b14827f7295a","path":"/admin/garbage_collect_artifacts"},"message":"Starting request 3edc2591-7553-4082-98df-b14827f7295a on /admin/garbage_collect_artifacts","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:04:00.798242586Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"3696bc13-7be1-4cae-bd45-a0690954069a","path":"/admin/clean_run_storage_stats"},"message":"Starting request 3696bc13-7be1-4cae-bd45-a0690954069a on /admin/clean_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:04:00.7987494Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"6ac261dd-41dc-4fc3-a207-6f13bc7b84de","path":"/admin/backfill_run_compute_hours"},"message":"Starting request 6ac261dd-41dc-4fc3-a207-6f13bc7b84de on /admin/backfill_run_compute_hours","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:04:00.798733053Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"e0c01477-0091-41f7-bb66-71c27a35bed1","path":"/admin/recompute_artifact_storage_bytes"},"message":"Starting request e0c01477-0091-41f7-bb66-71c27a35bed1 on /admin/recompute_artifact_storage_bytes","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:04:00.798893088Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"8d4f9ba6-71b6-48f8-8cd1-e16b964b8a11","path":"/admin/early_stop"},"message":"Starting request 8d4f9ba6-71b6-48f8-8cd1-e16b964b8a11 on /admin/early_stop","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:04:00.799038593Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"07eb6c0f-8d71-4894-80d9-383982d2bc48","path":"/admin/clean_artifact_collection_storage_stats"},"message":"Starting request 07eb6c0f-8d71-4894-80d9-383982d2bc48 on /admin/clean_artifact_collection_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:04:00.799664709Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"c339032a-7f6a-435d-98ae-e76011fb9a45","path":"/admin/delete_projects"},"message":"Starting request c339032a-7f6a-435d-98ae-e76011fb9a45 on /admin/delete_projects","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:04:00.799699675Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"5f39166d-c15d-4b5f-b7df-b2e7742fb074","path":"/admin/delete_artifact_collections"},"message":"Starting request 5f39166d-c15d-4b5f-b7df-b2e7742fb074 on /admin/delete_artifact_collections","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:04:00.79973225Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"cf9cc59e-38db-4bc9-b339-256762e2c32a","path":"/admin/backfill_run_storage_stats"},"message":"Starting request cf9cc59e-38db-4bc9-b339-256762e2c32a on /admin/backfill_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:04:00.799776239Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"50442aa2-0117-4fa1-8c80-f6b6c30682dc","duration":"1.94175ms","path":"/admin/build_artifact_deadlists"},"message":"Finished request 50442aa2-0117-4fa1-8c80-f6b6c30682dc in %!f(time.Duration=1941750) seconds on /admin/build_artifact_deadlists","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:04:00.800156959Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"e0c01477-0091-41f7-bb66-71c27a35bed1","duration":"1.464985ms","path":"/admin/recompute_artifact_storage_bytes"},"message":"Finished request e0c01477-0091-41f7-bb66-71c27a35bed1 in %!f(time.Duration=1464985) seconds on /admin/recompute_artifact_storage_bytes","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:04:00.800265202Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"8d4f9ba6-71b6-48f8-8cd1-e16b964b8a11","duration":"1.439033ms","path":"/admin/early_stop"},"message":"Finished request 8d4f9ba6-71b6-48f8-8cd1-e16b964b8a11 in %!f(time.Duration=1439033) seconds on /admin/early_stop","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:04:00.800472447Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"07eb6c0f-8d71-4894-80d9-383982d2bc48","duration":"1.42536ms","path":"/admin/clean_artifact_collection_storage_stats"},"message":"Finished request 07eb6c0f-8d71-4894-80d9-383982d2bc48 in %!f(time.Duration=1425360) seconds on /admin/clean_artifact_collection_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:04:00.800853795Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"5f39166d-c15d-4b5f-b7df-b2e7742fb074","duration":"1.169397ms","path":"/admin/delete_artifact_collections"},"message":"Finished request 5f39166d-c15d-4b5f-b7df-b2e7742fb074 in %!f(time.Duration=1169397) seconds on /admin/delete_artifact_collections","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:04:00.80136019Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"3edc2591-7553-4082-98df-b14827f7295a","duration":"3.176363ms","path":"/admin/garbage_collect_artifacts"},"message":"Finished request 3edc2591-7553-4082-98df-b14827f7295a in %!f(time.Duration=3176363) seconds on /admin/garbage_collect_artifacts","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:04:00.801855261Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"c339032a-7f6a-435d-98ae-e76011fb9a45","duration":"2.142295ms","path":"/admin/delete_projects"},"message":"Finished request c339032a-7f6a-435d-98ae-e76011fb9a45 in %!f(time.Duration=2142295) seconds on /admin/delete_projects","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:04:00.804167744Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"cf9cc59e-38db-4bc9-b339-256762e2c32a","duration":"4.477851ms","path":"/admin/backfill_run_storage_stats"},"message":"Finished request cf9cc59e-38db-4bc9-b339-256762e2c32a in %!f(time.Duration=4477851) seconds on /admin/backfill_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:04:00.932069167Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"3696bc13-7be1-4cae-bd45-a0690954069a","duration":"133.872718ms","path":"/admin/clean_run_storage_stats"},"message":"Finished request 3696bc13-7be1-4cae-bd45-a0690954069a in %!f(time.Duration=133872718) seconds on /admin/clean_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:04:01.00844316Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"6ac261dd-41dc-4fc3-a207-6f13bc7b84de","duration":"209.685891ms","path":"/admin/backfill_run_compute_hours"},"message":"Finished request 6ac261dd-41dc-4fc3-a207-6f13bc7b84de in %!f(time.Duration=209685891) seconds on /admin/backfill_run_compute_hours","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:05:00.810031601Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"1475f45e-ba28-4dad-9529-353bde954c26","path":"/admin/build_artifact_deadlists"},"message":"Starting request 1475f45e-ba28-4dad-9529-353bde954c26 on /admin/build_artifact_deadlists","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:05:00.810096278Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"8bd3b39b-d9cf-4d69-851f-87545fc3fa33","path":"/admin/delete_projects"},"message":"Starting request 8bd3b39b-d9cf-4d69-851f-87545fc3fa33 on /admin/delete_projects","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:05:00.810036053Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"cddf4ec1-9fa6-404b-9639-f774991c2b42","path":"/admin/clean_run_storage_stats"},"message":"Starting request cddf4ec1-9fa6-404b-9639-f774991c2b42 on /admin/clean_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:05:00.810031848Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"477d2f60-3c54-492a-a472-1c3f5cf1a03d","path":"/admin/crashed_launch_agents"},"message":"Starting request 477d2f60-3c54-492a-a472-1c3f5cf1a03d on /admin/crashed_launch_agents","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:05:00.810910184Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"032cc99b-fc1b-4e16-8ec2-6dcf3dcb472e","path":"/admin/stale_runs"},"message":"Starting request 032cc99b-fc1b-4e16-8ec2-6dcf3dcb472e on /admin/stale_runs","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:05:00.810930044Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"01881c83-02e9-4ac7-95d1-551425b382de","path":"/admin/delete_artifact_collections"},"message":"Starting request 01881c83-02e9-4ac7-95d1-551425b382de on /admin/delete_artifact_collections","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:05:00.811109085Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"477d2f60-3c54-492a-a472-1c3f5cf1a03d","duration":"1.101643ms","path":"/admin/crashed_launch_agents"},"message":"Finished request 477d2f60-3c54-492a-a472-1c3f5cf1a03d in %!f(time.Duration=1101643) seconds on /admin/crashed_launch_agents","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:05:00.811851258Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"01881c83-02e9-4ac7-95d1-551425b382de","duration":"947.294µs","path":"/admin/delete_artifact_collections"},"message":"Finished request 01881c83-02e9-4ac7-95d1-551425b382de in %!f(time.Duration=947294) seconds on /admin/delete_artifact_collections","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:05:00.811999031Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"8bd3b39b-d9cf-4d69-851f-87545fc3fa33","duration":"1.927646ms","path":"/admin/delete_projects"},"message":"Finished request 8bd3b39b-d9cf-4d69-851f-87545fc3fa33 in %!f(time.Duration=1927646) seconds on /admin/delete_projects","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:05:00.812174601Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"1475f45e-ba28-4dad-9529-353bde954c26","duration":"2.187373ms","path":"/admin/build_artifact_deadlists"},"message":"Finished request 1475f45e-ba28-4dad-9529-353bde954c26 in %!f(time.Duration=2187373) seconds on /admin/build_artifact_deadlists","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:05:00.812781921Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"4f2da160-0bcc-43b8-91f9-5af091b110d8","path":"/admin/backfill_run_storage_stats"},"message":"Starting request 4f2da160-0bcc-43b8-91f9-5af091b110d8 on /admin/backfill_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:05:00.812812592Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"e1fbdeba-d7cb-49ae-ac34-6ab67e952bd7","path":"/admin/stale_launch_agents"},"message":"Starting request e1fbdeba-d7cb-49ae-ac34-6ab67e952bd7 on /admin/stale_launch_agents","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:05:00.813404757Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"628c46e5-4d8e-4277-9b75-c7878cfc6f32","path":"/admin/clean_artifact_collection_storage_stats"},"message":"Starting request 628c46e5-4d8e-4277-9b75-c7878cfc6f32 on /admin/clean_artifact_collection_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:05:00.813591379Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"c90a30c9-9d47-4c5d-aca1-1e569e7b93d2","path":"/admin/early_stop"},"message":"Starting request c90a30c9-9d47-4c5d-aca1-1e569e7b93d2 on /admin/early_stop","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:05:00.814656546Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"b4daeb9b-1040-4089-9a9b-32c64baaa023","path":"/admin/backfill_run_compute_hours"},"message":"Starting request b4daeb9b-1040-4089-9a9b-32c64baaa023 on /admin/backfill_run_compute_hours","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:05:00.815220249Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"628c46e5-4d8e-4277-9b75-c7878cfc6f32","duration":"1.809786ms","path":"/admin/clean_artifact_collection_storage_stats"},"message":"Finished request 628c46e5-4d8e-4277-9b75-c7878cfc6f32 in %!f(time.Duration=1809786) seconds on /admin/clean_artifact_collection_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:05:00.815397133Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"e1fbdeba-d7cb-49ae-ac34-6ab67e952bd7","duration":"2.503725ms","path":"/admin/stale_launch_agents"},"message":"Finished request e1fbdeba-d7cb-49ae-ac34-6ab67e952bd7 in %!f(time.Duration=2503725) seconds on /admin/stale_launch_agents","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:05:00.815497492Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"467e207f-4bfc-4e95-b840-98007bbc3414","path":"/admin/recompute_artifact_storage_bytes"},"message":"Starting request 467e207f-4bfc-4e95-b840-98007bbc3414 on /admin/recompute_artifact_storage_bytes","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:05:00.815772692Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"c90a30c9-9d47-4c5d-aca1-1e569e7b93d2","duration":"2.183549ms","path":"/admin/early_stop"},"message":"Finished request c90a30c9-9d47-4c5d-aca1-1e569e7b93d2 in %!f(time.Duration=2183549) seconds on /admin/early_stop","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:05:00.817259131Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"b0a3c6dc-b0db-4458-95d4-2556b81b8ee1","path":"/admin/garbage_collect_artifacts"},"message":"Starting request b0a3c6dc-b0db-4458-95d4-2556b81b8ee1 on /admin/garbage_collect_artifacts","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:05:00.818027345Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"467e207f-4bfc-4e95-b840-98007bbc3414","duration":"2.574337ms","path":"/admin/recompute_artifact_storage_bytes"},"message":"Finished request 467e207f-4bfc-4e95-b840-98007bbc3414 in %!f(time.Duration=2574337) seconds on /admin/recompute_artifact_storage_bytes","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:05:00.818340085Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"4f2da160-0bcc-43b8-91f9-5af091b110d8","duration":"5.568278ms","path":"/admin/backfill_run_storage_stats"},"message":"Finished request 4f2da160-0bcc-43b8-91f9-5af091b110d8 in %!f(time.Duration=5568278) seconds on /admin/backfill_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:05:00.818875109Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"b0a3c6dc-b0db-4458-95d4-2556b81b8ee1","duration":"1.672425ms","path":"/admin/garbage_collect_artifacts"},"message":"Finished request b0a3c6dc-b0db-4458-95d4-2556b81b8ee1 in %!f(time.Duration=1672425) seconds on /admin/garbage_collect_artifacts","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:05:00.831106418Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"032cc99b-fc1b-4e16-8ec2-6dcf3dcb472e","duration":"20.212381ms","path":"/admin/stale_runs"},"message":"Finished request 032cc99b-fc1b-4e16-8ec2-6dcf3dcb472e in %!f(time.Duration=20212381) seconds on /admin/stale_runs","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:05:00.935522244Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"cddf4ec1-9fa6-404b-9639-f774991c2b42","duration":"125.493074ms","path":"/admin/clean_run_storage_stats"},"message":"Finished request cddf4ec1-9fa6-404b-9639-f774991c2b42 in %!f(time.Duration=125493074) seconds on /admin/clean_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:05:00.963877972Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"b4daeb9b-1040-4089-9a9b-32c64baaa023","duration":"149.789663ms","path":"/admin/backfill_run_compute_hours"},"message":"Finished request b4daeb9b-1040-4089-9a9b-32c64baaa023 in %!f(time.Duration=149789663) seconds on /admin/backfill_run_compute_hours","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:06:00.867134264Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"0cb97594-8cab-4b5e-bd53-92406aa59713","path":"/admin/build_artifact_deadlists"},"message":"Starting request 0cb97594-8cab-4b5e-bd53-92406aa59713 on /admin/build_artifact_deadlists","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:06:00.868547378Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"b5dfd166-295b-4efc-a7d3-4e8fb91af8bf","path":"/admin/clean_run_storage_stats"},"message":"Starting request b5dfd166-295b-4efc-a7d3-4e8fb91af8bf on /admin/clean_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:06:00.868653078Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"851a0850-594d-45c0-91b6-43a4b001a713","path":"/admin/garbage_collect_artifacts"},"message":"Starting request 851a0850-594d-45c0-91b6-43a4b001a713 on /admin/garbage_collect_artifacts","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:06:00.868989162Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"0cb97594-8cab-4b5e-bd53-92406aa59713","duration":"1.936567ms","path":"/admin/build_artifact_deadlists"},"message":"Finished request 0cb97594-8cab-4b5e-bd53-92406aa59713 in %!f(time.Duration=1936567) seconds on /admin/build_artifact_deadlists","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:06:00.869318011Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"639e4586-cef2-45c2-b7de-94d4a7302d67","path":"/admin/clean_artifact_collection_storage_stats"},"message":"Starting request 639e4586-cef2-45c2-b7de-94d4a7302d67 on /admin/clean_artifact_collection_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:06:00.869738838Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"81b31868-a87d-4b36-9291-e943364c87c8","path":"/admin/recompute_artifact_storage_bytes"},"message":"Starting request 81b31868-a87d-4b36-9291-e943364c87c8 on /admin/recompute_artifact_storage_bytes","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:06:00.869819698Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"851a0850-594d-45c0-91b6-43a4b001a713","duration":"1.197612ms","path":"/admin/garbage_collect_artifacts"},"message":"Finished request 851a0850-594d-45c0-91b6-43a4b001a713 in %!f(time.Duration=1197612) seconds on /admin/garbage_collect_artifacts","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:06:00.870325974Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"639e4586-cef2-45c2-b7de-94d4a7302d67","duration":"1.036197ms","path":"/admin/clean_artifact_collection_storage_stats"},"message":"Finished request 639e4586-cef2-45c2-b7de-94d4a7302d67 in %!f(time.Duration=1036197) seconds on /admin/clean_artifact_collection_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:06:00.870605229Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"f1cb2854-b52d-433d-ac5a-ad52b1a069b6","path":"/admin/delete_projects"},"message":"Starting request f1cb2854-b52d-433d-ac5a-ad52b1a069b6 on /admin/delete_projects","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:06:00.870714006Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"a2e5613b-0127-4969-bc2c-dd6dcabd8f3b","path":"/admin/delete_artifact_collections"},"message":"Starting request a2e5613b-0127-4969-bc2c-dd6dcabd8f3b on /admin/delete_artifact_collections","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:06:00.870862132Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"81b31868-a87d-4b36-9291-e943364c87c8","duration":"1.14351ms","path":"/admin/recompute_artifact_storage_bytes"},"message":"Finished request 81b31868-a87d-4b36-9291-e943364c87c8 in %!f(time.Duration=1143510) seconds on /admin/recompute_artifact_storage_bytes","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:06:00.871486942Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"f1cb2854-b52d-433d-ac5a-ad52b1a069b6","duration":"885.375µs","path":"/admin/delete_projects"},"message":"Finished request f1cb2854-b52d-433d-ac5a-ad52b1a069b6 in %!f(time.Duration=885375) seconds on /admin/delete_projects","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:06:00.871647476Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"a2e5613b-0127-4969-bc2c-dd6dcabd8f3b","duration":"968.081µs","path":"/admin/delete_artifact_collections"},"message":"Finished request a2e5613b-0127-4969-bc2c-dd6dcabd8f3b in %!f(time.Duration=968081) seconds on /admin/delete_artifact_collections","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:06:00.871730097Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"7f009b25-e6d2-484c-ac71-8654b9ee1dc6","path":"/admin/backfill_run_storage_stats"},"message":"Starting request 7f009b25-e6d2-484c-ac71-8654b9ee1dc6 on /admin/backfill_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:06:00.871873094Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"95ea7b7d-4d82-4b49-9811-7bd5596d431c","path":"/admin/backfill_run_compute_hours"},"message":"Starting request 95ea7b7d-4d82-4b49-9811-7bd5596d431c on /admin/backfill_run_compute_hours","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:06:00.872004163Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"0cf6f795-3973-4d86-96ca-358c4ec1acb0","path":"/admin/early_stop"},"message":"Starting request 0cf6f795-3973-4d86-96ca-358c4ec1acb0 on /admin/early_stop","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:06:00.872703752Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"0cf6f795-3973-4d86-96ca-358c4ec1acb0","duration":"708.257µs","path":"/admin/early_stop"},"message":"Finished request 0cf6f795-3973-4d86-96ca-358c4ec1acb0 in %!f(time.Duration=708257) seconds on /admin/early_stop","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:06:00.874334352Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"7f009b25-e6d2-484c-ac71-8654b9ee1dc6","duration":"2.631492ms","path":"/admin/backfill_run_storage_stats"},"message":"Finished request 7f009b25-e6d2-484c-ac71-8654b9ee1dc6 in %!f(time.Duration=2631492) seconds on /admin/backfill_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:06:00.987708348Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"b5dfd166-295b-4efc-a7d3-4e8fb91af8bf","duration":"119.207548ms","path":"/admin/clean_run_storage_stats"},"message":"Finished request b5dfd166-295b-4efc-a7d3-4e8fb91af8bf in %!f(time.Duration=119207548) seconds on /admin/clean_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:06:01.018229584Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"95ea7b7d-4d82-4b49-9811-7bd5596d431c","duration":"146.365571ms","path":"/admin/backfill_run_compute_hours"},"message":"Finished request 95ea7b7d-4d82-4b49-9811-7bd5596d431c in %!f(time.Duration=146365571) seconds on /admin/backfill_run_compute_hours","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:00.867591566Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"b6336aba-4a58-4833-876e-e773b515595b","path":"/admin/garbage_collect_artifacts"},"message":"Starting request b6336aba-4a58-4833-876e-e773b515595b on /admin/garbage_collect_artifacts","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:00.868324503Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"6a682462-95c4-4af9-be25-300fbd0ddea4","path":"/admin/recompute_artifact_storage_bytes"},"message":"Starting request 6a682462-95c4-4af9-be25-300fbd0ddea4 on /admin/recompute_artifact_storage_bytes","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:00.86943645Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"9ff4284f-b2fd-444c-8597-d00de8e76384","path":"/admin/build_artifact_deadlists"},"message":"Starting request 9ff4284f-b2fd-444c-8597-d00de8e76384 on /admin/build_artifact_deadlists","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:00.869532569Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"b6336aba-4a58-4833-876e-e773b515595b","duration":"2.012524ms","path":"/admin/garbage_collect_artifacts"},"message":"Finished request b6336aba-4a58-4833-876e-e773b515595b in %!f(time.Duration=2012524) seconds on /admin/garbage_collect_artifacts","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:00.869618695Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"6a682462-95c4-4af9-be25-300fbd0ddea4","duration":"1.301401ms","path":"/admin/recompute_artifact_storage_bytes"},"message":"Finished request 6a682462-95c4-4af9-be25-300fbd0ddea4 in %!f(time.Duration=1301401) seconds on /admin/recompute_artifact_storage_bytes","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:00.869907332Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"a61b4e25-717e-4ce6-96c5-5b7d3a53be2f","path":"/admin/backfill_run_compute_hours"},"message":"Starting request a61b4e25-717e-4ce6-96c5-5b7d3a53be2f on /admin/backfill_run_compute_hours","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:00.870606459Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"7d3b2cbb-3d98-45eb-b2fc-cd198ff03bb9","path":"/admin/backfill_run_storage_stats"},"message":"Starting request 7d3b2cbb-3d98-45eb-b2fc-cd198ff03bb9 on /admin/backfill_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:00.87070253Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"9be6663e-d60a-4e36-b882-e21e30f6c42f","path":"/admin/clean_artifact_collection_storage_stats"},"message":"Starting request 9be6663e-d60a-4e36-b882-e21e30f6c42f on /admin/clean_artifact_collection_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:00.871028491Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"9ff4284f-b2fd-444c-8597-d00de8e76384","duration":"1.654403ms","path":"/admin/build_artifact_deadlists"},"message":"Finished request 9ff4284f-b2fd-444c-8597-d00de8e76384 in %!f(time.Duration=1654403) seconds on /admin/build_artifact_deadlists","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:00.871097266Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"c68d53dc-4cd0-46f0-8f54-3d77d0619c03","path":"/admin/delete_projects"},"message":"Starting request c68d53dc-4cd0-46f0-8f54-3d77d0619c03 on /admin/delete_projects","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:00.871673393Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"a017914c-7bdb-4e6c-9f3b-c70245349ea6","path":"/admin/clean_run_storage_stats"},"message":"Starting request a017914c-7bdb-4e6c-9f3b-c70245349ea6 on /admin/clean_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:00.87177397Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"a078abdf-59ad-47f8-8f28-d959a6b95f7d","path":"/admin/delete_artifact_collections"},"message":"Starting request a078abdf-59ad-47f8-8f28-d959a6b95f7d on /admin/delete_artifact_collections","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:00.871992682Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"106ec688-95ef-435c-966d-2773777e5c36","path":"/admin/early_stop"},"message":"Starting request 106ec688-95ef-435c-966d-2773777e5c36 on /admin/early_stop","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:00.872045118Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"9be6663e-d60a-4e36-b882-e21e30f6c42f","duration":"1.361788ms","path":"/admin/clean_artifact_collection_storage_stats"},"message":"Finished request 9be6663e-d60a-4e36-b882-e21e30f6c42f in %!f(time.Duration=1361788) seconds on /admin/clean_artifact_collection_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:00.872623152Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"a078abdf-59ad-47f8-8f28-d959a6b95f7d","duration":"851.224µs","path":"/admin/delete_artifact_collections"},"message":"Finished request a078abdf-59ad-47f8-8f28-d959a6b95f7d in %!f(time.Duration=851224) seconds on /admin/delete_artifact_collections","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:00.872671449Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"c68d53dc-4cd0-46f0-8f54-3d77d0619c03","duration":"1.577836ms","path":"/admin/delete_projects"},"message":"Finished request c68d53dc-4cd0-46f0-8f54-3d77d0619c03 in %!f(time.Duration=1577836) seconds on /admin/delete_projects","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:00.872684825Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"106ec688-95ef-435c-966d-2773777e5c36","duration":"740.631µs","path":"/admin/early_stop"},"message":"Finished request 106ec688-95ef-435c-966d-2773777e5c36 in %!f(time.Duration=740631) seconds on /admin/early_stop","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:00.874646017Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"7d3b2cbb-3d98-45eb-b2fc-cd198ff03bb9","duration":"4.066293ms","path":"/admin/backfill_run_storage_stats"},"message":"Finished request 7d3b2cbb-3d98-45eb-b2fc-cd198ff03bb9 in %!f(time.Duration=4066293) seconds on /admin/backfill_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:00.985850893Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"a017914c-7bdb-4e6c-9f3b-c70245349ea6","duration":"114.208289ms","path":"/admin/clean_run_storage_stats"},"message":"Finished request a017914c-7bdb-4e6c-9f3b-c70245349ea6 in %!f(time.Duration=114208289) seconds on /admin/clean_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:01.009332992Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"a61b4e25-717e-4ce6-96c5-5b7d3a53be2f","duration":"139.446734ms","path":"/admin/backfill_run_compute_hours"},"message":"Finished request a61b4e25-717e-4ce6-96c5-5b7d3a53be2f in %!f(time.Duration=139446734) seconds on /admin/backfill_run_compute_hours","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:30.949393201Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"19cd7883-63f5-4f59-91a3-dad81290edb7","path":"/graphql"},"message":"Starting request 19cd7883-63f5-4f59-91a3-dad81290edb7 on /graphql","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:30.951169444Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"4164f715-2475-450c-9998-7bd2907094d4","path":"/graphql"},"message":"Starting request 4164f715-2475-450c-9998-7bd2907094d4 on /graphql","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:30.954170854Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/handler/relay.go:131","pid":62},"data":{"operationName":"Gallery","authUser":"shuyu","variables":{}},"message":"Graphql operation Gallery for user shuyu with variables map[]","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:30.955187245Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/handler/relay.go:131","pid":62},"data":{"operationName":"Viewer","authUser":"shuyu","variables":{"includeStorage":false}},"message":"Graphql operation Viewer for user shuyu with variables map[includeStorage:false]","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:30.95550211Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/handler/relay.go:188","pid":62},"data":{"latencyNs":1329306,"statusCode":200,"operationName":"Gallery","authUser":"shuyu","latencyStr":"1.329306ms"},"message":"Graphql operation Gallery for user shuyu finished in 1.329306ms","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:30.956015777Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"19cd7883-63f5-4f59-91a3-dad81290edb7","duration":"6.680786ms","path":"/graphql"},"message":"Finished request 19cd7883-63f5-4f59-91a3-dad81290edb7 in %!f(time.Duration=6680786) seconds on /graphql","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:30.960897859Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/handler/relay.go:188","pid":62},"data":{"latencyNs":5719924,"statusCode":200,"operationName":"Viewer","authUser":"shuyu","latencyStr":"5.719924ms"},"message":"Graphql operation Viewer for user shuyu finished in 5.719924ms","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:30.961481721Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"4164f715-2475-450c-9998-7bd2907094d4","duration":"10.29965ms","path":"/graphql"},"message":"Finished request 4164f715-2475-450c-9998-7bd2907094d4 in %!f(time.Duration=10299650) seconds on /graphql","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.344932159Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"c2bc688b-2a82-417f-ac0c-e77feb2ad6ab","path":"/graphql"},"message":"Starting request c2bc688b-2a82-417f-ac0c-e77feb2ad6ab on /graphql","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.34533063Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"3e4620f5-d43a-4417-a77b-2319357e2a72","path":"/graphql"},"message":"Starting request 3e4620f5-d43a-4417-a77b-2319357e2a72 on /graphql","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.34650079Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"7eaf4b86-51f5-4023-8052-e8970edf9aa5","path":"/graphql"},"message":"Starting request 7eaf4b86-51f5-4023-8052-e8970edf9aa5 on /graphql","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.347181818Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"9cb68992-09c9-4037-9f95-beb2b3812f1a","path":"/graphql"},"message":"Starting request 9cb68992-09c9-4037-9f95-beb2b3812f1a on /graphql","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.347215988Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"03a1b63f-c593-4743-b0e0-c2c37ce2a3e4","path":"/graphql"},"message":"Starting request 03a1b63f-c593-4743-b0e0-c2c37ce2a3e4 on /graphql","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.349247782Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/handler/relay.go:131","pid":62},"data":{"operationName":"SearchNavQuery","authUser":"shuyu","variables":{"sweepName":"","entityProjectsLimit":10000,"entityName":"","projectName":"","runName":""}},"message":"Graphql operation SearchNavQuery for user shuyu with variables map[entityName: entityProjectsLimit:10000 projectName: runName: sweepName:]","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.3496709Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/handler/relay.go:131","pid":62},"data":{"operationName":"ServerInfo","authUser":"shuyu","variables":{}},"message":"Graphql operation ServerInfo for user shuyu with variables map[]","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.35010315Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/handler/relay.go:188","pid":62},"data":{"statusCode":200,"latencyNs":432214,"operationName":"ServerInfo","authUser":"shuyu","latencyStr":"432.214µs"},"message":"Graphql operation ServerInfo for user shuyu finished in 432.214µs","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.350175591Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/handler/relay.go:131","pid":62},"data":{"operationName":"UserContext","authUser":"shuyu","variables":{}},"message":"Graphql operation UserContext for user shuyu with variables map[]","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.350409412Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"c2bc688b-2a82-417f-ac0c-e77feb2ad6ab","duration":"5.532307ms","path":"/graphql"},"message":"Finished request c2bc688b-2a82-417f-ac0c-e77feb2ad6ab in %!f(time.Duration=5532307) seconds on /graphql","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.350867636Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/handler/relay.go:131","pid":62},"data":{"operationName":"EntityStorage","authUser":"shuyu","variables":{"entityName":"shuyu","enableReferenceTracking":false}},"message":"Graphql operation EntityStorage for user shuyu with variables map[enableReferenceTracking:false entityName:shuyu]","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.351026967Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/handler/relay.go:131","pid":62},"data":{"operationName":"MessageOfTheDay","authUser":"shuyu","variables":{}},"message":"Graphql operation MessageOfTheDay for user shuyu with variables map[]","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.351676087Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/handler/relay.go:188","pid":62},"data":{"latencyNs":639618,"statusCode":200,"operationName":"MessageOfTheDay","authUser":"shuyu","latencyStr":"639.618µs"},"message":"Graphql operation MessageOfTheDay for user shuyu finished in 639.618µs","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.352008276Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"03a1b63f-c593-4743-b0e0-c2c37ce2a3e4","duration":"4.820049ms","path":"/graphql"},"message":"Finished request 03a1b63f-c593-4743-b0e0-c2c37ce2a3e4 in %!f(time.Duration=4820049) seconds on /graphql","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.352416442Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/handler/relay.go:188","pid":62},"data":{"latencyNs":2194107,"statusCode":200,"operationName":"UserContext","authUser":"shuyu","latencyStr":"2.194107ms"},"message":"Graphql operation UserContext for user shuyu finished in 2.194107ms","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.352973886Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"7eaf4b86-51f5-4023-8052-e8970edf9aa5","duration":"6.492274ms","path":"/graphql"},"message":"Finished request 7eaf4b86-51f5-4023-8052-e8970edf9aa5 in %!f(time.Duration=6492274) seconds on /graphql","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.355067703Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/handler/relay.go:188","pid":62},"data":{"latencyNs":4232527,"statusCode":200,"operationName":"EntityStorage","authUser":"shuyu","latencyStr":"4.232527ms"},"message":"Graphql operation EntityStorage for user shuyu finished in 4.232527ms","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.355385879Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"9cb68992-09c9-4037-9f95-beb2b3812f1a","duration":"8.265455ms","path":"/graphql"},"message":"Finished request 9cb68992-09c9-4037-9f95-beb2b3812f1a in %!f(time.Duration=8265455) seconds on /graphql","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.359794142Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/handler/relay.go:188","pid":62},"data":{"latencyNs":10536886,"statusCode":200,"operationName":"SearchNavQuery","authUser":"shuyu","latencyStr":"10.536886ms"},"message":"Graphql operation SearchNavQuery for user shuyu finished in 10.536886ms","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.360014593Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"3e4620f5-d43a-4417-a77b-2319357e2a72","duration":"14.735791ms","path":"/graphql"},"message":"Finished request 3e4620f5-d43a-4417-a77b-2319357e2a72 in %!f(time.Duration=14735791) seconds on /graphql","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.439781608Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"74c7a0c6-1714-4036-8a9a-495d7c60a7d9","path":"/graphql"},"message":"Starting request 74c7a0c6-1714-4036-8a9a-495d7c60a7d9 on /graphql","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.443218081Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/handler/relay.go:131","pid":62},"data":{"operationName":"HasLocalLicense","authUser":"shuyu","variables":{}},"message":"Graphql operation HasLocalLicense for user shuyu with variables map[]","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.44348435Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/handler/relay.go:188","pid":62},"data":{"latencyNs":262932,"statusCode":200,"operationName":"HasLocalLicense","authUser":"shuyu","latencyStr":"262.932µs"},"message":"Graphql operation HasLocalLicense for user shuyu finished in 262.932µs","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.443571154Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"74c7a0c6-1714-4036-8a9a-495d7c60a7d9","duration":"3.824903ms","path":"/graphql"},"message":"Finished request 74c7a0c6-1714-4036-8a9a-495d7c60a7d9 in %!f(time.Duration=3824903) seconds on /graphql","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.447941823Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"ba18fabc-e6b5-4999-9a5c-61cdc2201a1e","path":"/graphql"},"message":"Starting request ba18fabc-e6b5-4999-9a5c-61cdc2201a1e on /graphql","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.448655219Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"ff42e368-31f1-46a7-83c8-c7a4f2d0b5d1","path":"/graphql"},"message":"Starting request ff42e368-31f1-46a7-83c8-c7a4f2d0b5d1 on /graphql","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.451263901Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/handler/relay.go:131","pid":62},"data":{"operationName":"LicenseExpirationUI","authUser":"shuyu","variables":{}},"message":"Graphql operation LicenseExpirationUI for user shuyu with variables map[]","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.45165692Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/handler/relay.go:188","pid":62},"data":{"latencyNs":391138,"statusCode":200,"operationName":"LicenseExpirationUI","authUser":"shuyu","latencyStr":"391.138µs"},"message":"Graphql operation LicenseExpirationUI for user shuyu finished in 391.138µs","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.451748319Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"ba18fabc-e6b5-4999-9a5c-61cdc2201a1e","duration":"3.836099ms","path":"/graphql"},"message":"Finished request ba18fabc-e6b5-4999-9a5c-61cdc2201a1e in %!f(time.Duration=3836099) seconds on /graphql","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.452207194Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/handler/relay.go:131","pid":62},"data":{"operationName":"latestLocalVersionInfo","authUser":"shuyu","variables":{}},"message":"Graphql operation latestLocalVersionInfo for user shuyu with variables map[]","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.452702271Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/handler/relay.go:188","pid":62},"data":{"latencyNs":489105,"statusCode":200,"operationName":"latestLocalVersionInfo","authUser":"shuyu","latencyStr":"489.105µs"},"message":"Graphql operation latestLocalVersionInfo for user shuyu finished in 489.105µs","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:31.453000892Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"ff42e368-31f1-46a7-83c8-c7a4f2d0b5d1","duration":"4.371165ms","path":"/graphql"},"message":"Finished request ff42e368-31f1-46a7-83c8-c7a4f2d0b5d1 in %!f(time.Duration=4371165) seconds on /graphql","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:33.734086455Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"d9e75073-b63d-413d-a4c3-0d360b575f98","path":"/graphql"},"message":"Starting request d9e75073-b63d-413d-a4c3-0d360b575f98 on /graphql","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:33.735880833Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"b4df4b32-b65f-4291-af59-cb7b8f33cf31","path":"/graphql"},"message":"Starting request b4df4b32-b65f-4291-af59-cb7b8f33cf31 on /graphql","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:33.738246862Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/handler/relay.go:131","pid":62},"data":{"operationName":"ViewerRuns","authUser":"shuyu","variables":{}},"message":"Graphql operation ViewerRuns for user shuyu with variables map[]","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:33.739830058Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/handler/relay.go:131","pid":62},"data":{"operationName":"HomePageUserDetails","authUser":"shuyu","variables":{}},"message":"Graphql operation HomePageUserDetails for user shuyu with variables map[]","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:33.745653803Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/handler/relay.go:188","pid":62},"data":{"latencyNs":5656662,"statusCode":200,"operationName":"HomePageUserDetails","authUser":"shuyu","latencyStr":"5.656662ms"},"message":"Graphql operation HomePageUserDetails for user shuyu finished in 5.656662ms","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:33.746181968Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"b4df4b32-b65f-4291-af59-cb7b8f33cf31","duration":"10.260269ms","path":"/graphql"},"message":"Finished request b4df4b32-b65f-4291-af59-cb7b8f33cf31 in %!f(time.Duration=10260269) seconds on /graphql","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:33.762967029Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/handler/relay.go:188","pid":62},"data":{"latencyNs":24706191,"statusCode":200,"operationName":"ViewerRuns","authUser":"shuyu","latencyStr":"24.706191ms"},"message":"Graphql operation ViewerRuns for user shuyu finished in 24.706191ms","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:07:33.763912884Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"d9e75073-b63d-413d-a4c3-0d360b575f98","duration":"29.86963ms","path":"/graphql"},"message":"Finished request d9e75073-b63d-413d-a4c3-0d360b575f98 in %!f(time.Duration=29869630) seconds on /graphql","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:08:00.924387041Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"e274a67f-e27b-4121-8333-7d04ac2b801f","path":"/admin/clean_run_storage_stats"},"message":"Starting request e274a67f-e27b-4121-8333-7d04ac2b801f on /admin/clean_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:08:00.925798283Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"01d8fb77-e1f7-45ee-a22a-d440ac3c7a90","path":"/admin/build_artifact_deadlists"},"message":"Starting request 01d8fb77-e1f7-45ee-a22a-d440ac3c7a90 on /admin/build_artifact_deadlists","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:08:00.92657184Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"d4a56598-9261-4539-9396-b77e706a1b8f","path":"/admin/backfill_run_storage_stats"},"message":"Starting request d4a56598-9261-4539-9396-b77e706a1b8f on /admin/backfill_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:08:00.927185669Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"01d8fb77-e1f7-45ee-a22a-d440ac3c7a90","duration":"1.417639ms","path":"/admin/build_artifact_deadlists"},"message":"Finished request 01d8fb77-e1f7-45ee-a22a-d440ac3c7a90 in %!f(time.Duration=1417639) seconds on /admin/build_artifact_deadlists","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:08:00.927572841Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"13dbc61d-a102-4612-bc7f-a606a065d528","path":"/admin/garbage_collect_artifacts"},"message":"Starting request 13dbc61d-a102-4612-bc7f-a606a065d528 on /admin/garbage_collect_artifacts","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:08:00.928191734Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"e0454d7f-e066-4004-9308-db04a92131db","path":"/admin/delete_artifact_collections"},"message":"Starting request e0454d7f-e066-4004-9308-db04a92131db on /admin/delete_artifact_collections","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:08:00.928234322Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"e29d6b8a-be79-45ac-8e0a-6ed099673239","path":"/admin/delete_projects"},"message":"Starting request e29d6b8a-be79-45ac-8e0a-6ed099673239 on /admin/delete_projects","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:08:00.928480475Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"f614d959-96b2-4e36-a6e8-6dfc318bd511","path":"/admin/recompute_artifact_storage_bytes"},"message":"Starting request f614d959-96b2-4e36-a6e8-6dfc318bd511 on /admin/recompute_artifact_storage_bytes","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:08:00.928503711Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"46003911-a205-4027-9ce6-c53f23012f16","path":"/admin/backfill_run_compute_hours"},"message":"Starting request 46003911-a205-4027-9ce6-c53f23012f16 on /admin/backfill_run_compute_hours","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:08:00.928587302Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"13dbc61d-a102-4612-bc7f-a606a065d528","duration":"1.053381ms","path":"/admin/garbage_collect_artifacts"},"message":"Finished request 13dbc61d-a102-4612-bc7f-a606a065d528 in %!f(time.Duration=1053381) seconds on /admin/garbage_collect_artifacts","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:08:00.928782744Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"8a998c9a-4247-46b0-8108-f76ae8fddb15","path":"/admin/clean_artifact_collection_storage_stats"},"message":"Starting request 8a998c9a-4247-46b0-8108-f76ae8fddb15 on /admin/clean_artifact_collection_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:08:00.928812554Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"c4a936fb-aaa1-4a67-901b-82aeabbcf5d2","path":"/admin/early_stop"},"message":"Starting request c4a936fb-aaa1-4a67-901b-82aeabbcf5d2 on /admin/early_stop","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:08:00.929319137Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"e29d6b8a-be79-45ac-8e0a-6ed099673239","duration":"1.115206ms","path":"/admin/delete_projects"},"message":"Finished request e29d6b8a-be79-45ac-8e0a-6ed099673239 in %!f(time.Duration=1115206) seconds on /admin/delete_projects","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:08:00.929512054Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"e0454d7f-e066-4004-9308-db04a92131db","duration":"1.366302ms","path":"/admin/delete_artifact_collections"},"message":"Finished request e0454d7f-e066-4004-9308-db04a92131db in %!f(time.Duration=1366302) seconds on /admin/delete_artifact_collections","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:08:00.929544419Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"c4a936fb-aaa1-4a67-901b-82aeabbcf5d2","duration":"767.405µs","path":"/admin/early_stop"},"message":"Finished request c4a936fb-aaa1-4a67-901b-82aeabbcf5d2 in %!f(time.Duration=767405) seconds on /admin/early_stop","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:08:00.929621714Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"f614d959-96b2-4e36-a6e8-6dfc318bd511","duration":"1.182594ms","path":"/admin/recompute_artifact_storage_bytes"},"message":"Finished request f614d959-96b2-4e36-a6e8-6dfc318bd511 in %!f(time.Duration=1182594) seconds on /admin/recompute_artifact_storage_bytes","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:08:00.929876864Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"d4a56598-9261-4539-9396-b77e706a1b8f","duration":"3.309425ms","path":"/admin/backfill_run_storage_stats"},"message":"Finished request d4a56598-9261-4539-9396-b77e706a1b8f in %!f(time.Duration=3309425) seconds on /admin/backfill_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:08:00.929975905Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"8a998c9a-4247-46b0-8108-f76ae8fddb15","duration":"1.195132ms","path":"/admin/clean_artifact_collection_storage_stats"},"message":"Finished request 8a998c9a-4247-46b0-8108-f76ae8fddb15 in %!f(time.Duration=1195132) seconds on /admin/clean_artifact_collection_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:08:01.029375102Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"e274a67f-e27b-4121-8333-7d04ac2b801f","duration":"105.035456ms","path":"/admin/clean_run_storage_stats"},"message":"Finished request e274a67f-e27b-4121-8333-7d04ac2b801f in %!f(time.Duration=105035456) seconds on /admin/clean_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:08:01.057957476Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"46003911-a205-4027-9ce6-c53f23012f16","duration":"129.497527ms","path":"/admin/backfill_run_compute_hours"},"message":"Finished request 46003911-a205-4027-9ce6-c53f23012f16 in %!f(time.Duration=129497527) seconds on /admin/backfill_run_compute_hours","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:09:00.98497757Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"99b3082b-c1f5-4c58-9723-498c74ee32bb","path":"/admin/build_artifact_deadlists"},"message":"Starting request 99b3082b-c1f5-4c58-9723-498c74ee32bb on /admin/build_artifact_deadlists","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:09:00.985130836Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"535b8da4-a3ab-4e55-91f0-9ad2ea76843b","path":"/admin/clean_run_storage_stats"},"message":"Starting request 535b8da4-a3ab-4e55-91f0-9ad2ea76843b on /admin/clean_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:09:00.985270418Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"9a440be8-8dc0-4b8a-9554-5be87928883f","path":"/admin/delete_artifact_collections"},"message":"Starting request 9a440be8-8dc0-4b8a-9554-5be87928883f on /admin/delete_artifact_collections","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:09:00.98595619Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"19e1153a-8a58-4d75-ae1d-60b3878ce445","path":"/admin/early_stop"},"message":"Starting request 19e1153a-8a58-4d75-ae1d-60b3878ce445 on /admin/early_stop","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:09:00.986495336Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"2a926dc1-6033-4398-9cee-fd70b43e8c62","path":"/admin/backfill_run_compute_hours"},"message":"Starting request 2a926dc1-6033-4398-9cee-fd70b43e8c62 on /admin/backfill_run_compute_hours","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:09:00.98671534Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"99b3082b-c1f5-4c58-9723-498c74ee32bb","duration":"1.788044ms","path":"/admin/build_artifact_deadlists"},"message":"Finished request 99b3082b-c1f5-4c58-9723-498c74ee32bb in %!f(time.Duration=1788044) seconds on /admin/build_artifact_deadlists","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:09:00.98674359Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"9a440be8-8dc0-4b8a-9554-5be87928883f","duration":"1.508325ms","path":"/admin/delete_artifact_collections"},"message":"Finished request 9a440be8-8dc0-4b8a-9554-5be87928883f in %!f(time.Duration=1508325) seconds on /admin/delete_artifact_collections","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:09:00.986827039Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"45efbdec-675f-41f1-9930-d4c59fc0cfd1","path":"/admin/garbage_collect_artifacts"},"message":"Starting request 45efbdec-675f-41f1-9930-d4c59fc0cfd1 on /admin/garbage_collect_artifacts","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:09:00.986858979Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"19e1153a-8a58-4d75-ae1d-60b3878ce445","duration":"929.102µs","path":"/admin/early_stop"},"message":"Finished request 19e1153a-8a58-4d75-ae1d-60b3878ce445 in %!f(time.Duration=929102) seconds on /admin/early_stop","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:09:00.988150527Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"028932bd-ad24-41e9-99c4-0fc67cd2b280","path":"/admin/delete_projects"},"message":"Starting request 028932bd-ad24-41e9-99c4-0fc67cd2b280 on /admin/delete_projects","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:09:00.988213209Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"45efbdec-675f-41f1-9930-d4c59fc0cfd1","duration":"1.425243ms","path":"/admin/garbage_collect_artifacts"},"message":"Finished request 45efbdec-675f-41f1-9930-d4c59fc0cfd1 in %!f(time.Duration=1425243) seconds on /admin/garbage_collect_artifacts","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:09:00.988212895Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"20401500-4125-42f7-b3f7-a9eecc7e3f2f","path":"/admin/recompute_artifact_storage_bytes"},"message":"Starting request 20401500-4125-42f7-b3f7-a9eecc7e3f2f on /admin/recompute_artifact_storage_bytes","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:09:00.989271674Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"20401500-4125-42f7-b3f7-a9eecc7e3f2f","duration":"1.088532ms","path":"/admin/recompute_artifact_storage_bytes"},"message":"Finished request 20401500-4125-42f7-b3f7-a9eecc7e3f2f in %!f(time.Duration=1088532) seconds on /admin/recompute_artifact_storage_bytes","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:09:00.989329645Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"2a9f8cbc-c658-4514-a6b6-85a36b35118a","path":"/admin/clean_artifact_collection_storage_stats"},"message":"Starting request 2a9f8cbc-c658-4514-a6b6-85a36b35118a on /admin/clean_artifact_collection_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:09:00.989392782Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"072851af-30b1-4978-961b-e8ae7237f1f5","path":"/admin/backfill_run_storage_stats"},"message":"Starting request 072851af-30b1-4978-961b-e8ae7237f1f5 on /admin/backfill_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:09:00.989704805Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"028932bd-ad24-41e9-99c4-0fc67cd2b280","duration":"1.554045ms","path":"/admin/delete_projects"},"message":"Finished request 028932bd-ad24-41e9-99c4-0fc67cd2b280 in %!f(time.Duration=1554045) seconds on /admin/delete_projects","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:09:00.990129823Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"2a9f8cbc-c658-4514-a6b6-85a36b35118a","duration":"840.715µs","path":"/admin/clean_artifact_collection_storage_stats"},"message":"Finished request 2a9f8cbc-c658-4514-a6b6-85a36b35118a in %!f(time.Duration=840715) seconds on /admin/clean_artifact_collection_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:09:00.991393349Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"072851af-30b1-4978-961b-e8ae7237f1f5","duration":"2.002241ms","path":"/admin/backfill_run_storage_stats"},"message":"Finished request 072851af-30b1-4978-961b-e8ae7237f1f5 in %!f(time.Duration=2002241) seconds on /admin/backfill_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:09:01.107488034Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"535b8da4-a3ab-4e55-91f0-9ad2ea76843b","duration":"122.399168ms","path":"/admin/clean_run_storage_stats"},"message":"Finished request 535b8da4-a3ab-4e55-91f0-9ad2ea76843b in %!f(time.Duration=122399168) seconds on /admin/clean_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:09:01.133614678Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"2a926dc1-6033-4398-9cee-fd70b43e8c62","duration":"147.118939ms","path":"/admin/backfill_run_compute_hours"},"message":"Finished request 2a926dc1-6033-4398-9cee-fd70b43e8c62 in %!f(time.Duration=147118939) seconds on /admin/backfill_run_compute_hours","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:10:00.814674546Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"a972eaf0-6905-48b2-b765-fe7bce4c51c6","path":"/admin/prune_sessions"},"message":"Starting request a972eaf0-6905-48b2-b765-fe7bce4c51c6 on /admin/prune_sessions","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:10:00.815275794Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"925c513d-f682-4758-a9b0-c6a4aeaaa2d7","path":"/admin/stale_runs"},"message":"Starting request 925c513d-f682-4758-a9b0-c6a4aeaaa2d7 on /admin/stale_runs","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:10:00.816868269Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"a972eaf0-6905-48b2-b765-fe7bce4c51c6","duration":"2.223442ms","path":"/admin/prune_sessions"},"message":"Finished request a972eaf0-6905-48b2-b765-fe7bce4c51c6 in %!f(time.Duration=2223442) seconds on /admin/prune_sessions","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:10:00.816956625Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"cfb2bd4a-f7ee-4db2-88a8-fcb164f0c67d","path":"/admin/crashed_launch_agents"},"message":"Starting request cfb2bd4a-f7ee-4db2-88a8-fcb164f0c67d on /admin/crashed_launch_agents","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:10:00.81702049Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"b83ab567-cb78-44f0-8463-ec7612f2ede4","path":"/admin/stale_launch_agents"},"message":"Starting request b83ab567-cb78-44f0-8463-ec7612f2ede4 on /admin/stale_launch_agents","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:10:00.81782886Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"cfb2bd4a-f7ee-4db2-88a8-fcb164f0c67d","duration":"897.68µs","path":"/admin/crashed_launch_agents"},"message":"Finished request cfb2bd4a-f7ee-4db2-88a8-fcb164f0c67d in %!f(time.Duration=897680) seconds on /admin/crashed_launch_agents","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:10:00.819339886Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"b83ab567-cb78-44f0-8463-ec7612f2ede4","duration":"2.325934ms","path":"/admin/stale_launch_agents"},"message":"Finished request b83ab567-cb78-44f0-8463-ec7612f2ede4 in %!f(time.Duration=2325934) seconds on /admin/stale_launch_agents","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:10:00.831464978Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"925c513d-f682-4758-a9b0-c6a4aeaaa2d7","duration":"16.205432ms","path":"/admin/stale_runs"},"message":"Finished request 925c513d-f682-4758-a9b0-c6a4aeaaa2d7 in %!f(time.Duration=16205432) seconds on /admin/stale_runs","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:10:00.983571798Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"bf3c5d56-d3c7-4161-8bb1-699b3d42f68a","path":"/admin/recompute_artifact_storage_bytes"},"message":"Starting request bf3c5d56-d3c7-4161-8bb1-699b3d42f68a on /admin/recompute_artifact_storage_bytes","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:10:00.983593684Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"f81fe077-e380-4342-b5fb-4b96009354ec","path":"/admin/clean_run_storage_stats"},"message":"Starting request f81fe077-e380-4342-b5fb-4b96009354ec on /admin/clean_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:10:00.984080519Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"6a7b65e6-d243-477c-a898-a79342d5b8ed","path":"/admin/early_stop"},"message":"Starting request 6a7b65e6-d243-477c-a898-a79342d5b8ed on /admin/early_stop","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:10:00.984483028Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"bf3c5d56-d3c7-4161-8bb1-699b3d42f68a","duration":"937.463µs","path":"/admin/recompute_artifact_storage_bytes"},"message":"Finished request bf3c5d56-d3c7-4161-8bb1-699b3d42f68a in %!f(time.Duration=937463) seconds on /admin/recompute_artifact_storage_bytes","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:10:00.984957623Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"6a7b65e6-d243-477c-a898-a79342d5b8ed","duration":"889.641µs","path":"/admin/early_stop"},"message":"Finished request 6a7b65e6-d243-477c-a898-a79342d5b8ed in %!f(time.Duration=889641) seconds on /admin/early_stop","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:10:00.986023173Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"2d9e1bb8-d1ca-467d-b8d3-90cba3a5e7ae","path":"/admin/delete_artifact_collections"},"message":"Starting request 2d9e1bb8-d1ca-467d-b8d3-90cba3a5e7ae on /admin/delete_artifact_collections","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:10:00.98629156Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"a4175076-eb8e-4e35-91e9-1cc100f31ad2","path":"/admin/delete_projects"},"message":"Starting request a4175076-eb8e-4e35-91e9-1cc100f31ad2 on /admin/delete_projects","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:10:00.986295347Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"f3893fcb-aa0b-46a7-9e24-41ba2f4df29c","path":"/admin/garbage_collect_artifacts"},"message":"Starting request f3893fcb-aa0b-46a7-9e24-41ba2f4df29c on /admin/garbage_collect_artifacts","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:10:00.986552764Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"aeb63eae-9fe0-4195-babf-cfdcc57b823f","path":"/admin/build_artifact_deadlists"},"message":"Starting request aeb63eae-9fe0-4195-babf-cfdcc57b823f on /admin/build_artifact_deadlists","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:10:00.986594313Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"5ed5c4da-5e32-4ebd-8396-380f78a35169","path":"/admin/clean_artifact_collection_storage_stats"},"message":"Starting request 5ed5c4da-5e32-4ebd-8396-380f78a35169 on /admin/clean_artifact_collection_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:10:00.986953222Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"2d9e1bb8-d1ca-467d-b8d3-90cba3a5e7ae","duration":"943.654µs","path":"/admin/delete_artifact_collections"},"message":"Finished request 2d9e1bb8-d1ca-467d-b8d3-90cba3a5e7ae in %!f(time.Duration=943654) seconds on /admin/delete_artifact_collections","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:10:00.98708056Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"f3893fcb-aa0b-46a7-9e24-41ba2f4df29c","duration":"799.271µs","path":"/admin/garbage_collect_artifacts"},"message":"Finished request f3893fcb-aa0b-46a7-9e24-41ba2f4df29c in %!f(time.Duration=799271) seconds on /admin/garbage_collect_artifacts","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:10:00.987244354Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"5ed5c4da-5e32-4ebd-8396-380f78a35169","duration":"657.976µs","path":"/admin/clean_artifact_collection_storage_stats"},"message":"Finished request 5ed5c4da-5e32-4ebd-8396-380f78a35169 in %!f(time.Duration=657976) seconds on /admin/clean_artifact_collection_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:10:00.987559687Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"07e6aec8-2911-43b9-8df0-b02bb07e6484","path":"/admin/backfill_run_storage_stats"},"message":"Starting request 07e6aec8-2911-43b9-8df0-b02bb07e6484 on /admin/backfill_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:10:00.987587294Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"dd21ddf1-431b-430b-ac72-3d7f6aaf2435","path":"/admin/backfill_run_compute_hours"},"message":"Starting request dd21ddf1-431b-430b-ac72-3d7f6aaf2435 on /admin/backfill_run_compute_hours","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:10:00.987629475Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"aeb63eae-9fe0-4195-babf-cfdcc57b823f","duration":"1.094592ms","path":"/admin/build_artifact_deadlists"},"message":"Finished request aeb63eae-9fe0-4195-babf-cfdcc57b823f in %!f(time.Duration=1094592) seconds on /admin/build_artifact_deadlists","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:10:00.987775125Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"a4175076-eb8e-4e35-91e9-1cc100f31ad2","duration":"1.500613ms","path":"/admin/delete_projects"},"message":"Finished request a4175076-eb8e-4e35-91e9-1cc100f31ad2 in %!f(time.Duration=1500613) seconds on /admin/delete_projects","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:10:00.990279241Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"07e6aec8-2911-43b9-8df0-b02bb07e6484","duration":"2.744794ms","path":"/admin/backfill_run_storage_stats"},"message":"Finished request 07e6aec8-2911-43b9-8df0-b02bb07e6484 in %!f(time.Duration=2744794) seconds on /admin/backfill_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:10:01.101445886Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"f81fe077-e380-4342-b5fb-4b96009354ec","duration":"117.878959ms","path":"/admin/clean_run_storage_stats"},"message":"Finished request f81fe077-e380-4342-b5fb-4b96009354ec in %!f(time.Duration=117878959) seconds on /admin/clean_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:10:01.13280064Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"dd21ddf1-431b-430b-ac72-3d7f6aaf2435","duration":"145.265991ms","path":"/admin/backfill_run_compute_hours"},"message":"Finished request dd21ddf1-431b-430b-ac72-3d7f6aaf2435 in %!f(time.Duration=145265991) seconds on /admin/backfill_run_compute_hours","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:00.985552306Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"3fa2363b-f3d6-4e76-9ca0-a10652b2934d","path":"/admin/early_stop"},"message":"Starting request 3fa2363b-f3d6-4e76-9ca0-a10652b2934d on /admin/early_stop","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:00.986285229Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"820452ca-989b-44a1-b898-d03b6efc56cd","path":"/admin/clean_run_storage_stats"},"message":"Starting request 820452ca-989b-44a1-b898-d03b6efc56cd on /admin/clean_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:00.986648758Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"3fa2363b-f3d6-4e76-9ca0-a10652b2934d","duration":"1.139691ms","path":"/admin/early_stop"},"message":"Finished request 3fa2363b-f3d6-4e76-9ca0-a10652b2934d in %!f(time.Duration=1139691) seconds on /admin/early_stop","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:00.987532949Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"ac9d0c08-dd92-47e5-81be-db06101e0c83","path":"/admin/build_artifact_deadlists"},"message":"Starting request ac9d0c08-dd92-47e5-81be-db06101e0c83 on /admin/build_artifact_deadlists","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:00.988291602Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"be34eb85-b02c-4b52-b533-8dff534565e3","path":"/admin/delete_artifact_collections"},"message":"Starting request be34eb85-b02c-4b52-b533-8dff534565e3 on /admin/delete_artifact_collections","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:00.988883054Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"ac9d0c08-dd92-47e5-81be-db06101e0c83","duration":"1.382529ms","path":"/admin/build_artifact_deadlists"},"message":"Finished request ac9d0c08-dd92-47e5-81be-db06101e0c83 in %!f(time.Duration=1382529) seconds on /admin/build_artifact_deadlists","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:00.989462224Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"be34eb85-b02c-4b52-b533-8dff534565e3","duration":"1.193472ms","path":"/admin/delete_artifact_collections"},"message":"Finished request be34eb85-b02c-4b52-b533-8dff534565e3 in %!f(time.Duration=1193472) seconds on /admin/delete_artifact_collections","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:00.990528644Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"10855c04-7caf-4e15-adc4-e2b1beb9b018","path":"/admin/delete_projects"},"message":"Starting request 10855c04-7caf-4e15-adc4-e2b1beb9b018 on /admin/delete_projects","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:00.99065519Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"d450b92e-bfb6-492a-a343-40ce8a27cc9c","path":"/admin/backfill_run_storage_stats"},"message":"Starting request d450b92e-bfb6-492a-a343-40ce8a27cc9c on /admin/backfill_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:00.992094645Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"10855c04-7caf-4e15-adc4-e2b1beb9b018","duration":"1.590354ms","path":"/admin/delete_projects"},"message":"Finished request 10855c04-7caf-4e15-adc4-e2b1beb9b018 in %!f(time.Duration=1590354) seconds on /admin/delete_projects","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:00.993176446Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"5f8ec1b9-55d0-4f31-8945-8939afe5a2a0","path":"/admin/backfill_run_compute_hours"},"message":"Starting request 5f8ec1b9-55d0-4f31-8945-8939afe5a2a0 on /admin/backfill_run_compute_hours","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:00.99336741Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"191983c1-a4a6-48dd-8491-00730402ca84","path":"/admin/garbage_collect_artifacts"},"message":"Starting request 191983c1-a4a6-48dd-8491-00730402ca84 on /admin/garbage_collect_artifacts","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:00.993390014Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"eb53bd7c-f34c-4aea-818d-ae4ec1f93626","path":"/admin/recompute_artifact_storage_bytes"},"message":"Starting request eb53bd7c-f34c-4aea-818d-ae4ec1f93626 on /admin/recompute_artifact_storage_bytes","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:00.993477705Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"2c58b7ca-5c2a-4fea-bc55-c7fcba4ced43","path":"/admin/clean_artifact_collection_storage_stats"},"message":"Starting request 2c58b7ca-5c2a-4fea-bc55-c7fcba4ced43 on /admin/clean_artifact_collection_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:00.994128641Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"eb53bd7c-f34c-4aea-818d-ae4ec1f93626","duration":"773.942µs","path":"/admin/recompute_artifact_storage_bytes"},"message":"Finished request eb53bd7c-f34c-4aea-818d-ae4ec1f93626 in %!f(time.Duration=773942) seconds on /admin/recompute_artifact_storage_bytes","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:00.994271002Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"2c58b7ca-5c2a-4fea-bc55-c7fcba4ced43","duration":"797.636µs","path":"/admin/clean_artifact_collection_storage_stats"},"message":"Finished request 2c58b7ca-5c2a-4fea-bc55-c7fcba4ced43 in %!f(time.Duration=797636) seconds on /admin/clean_artifact_collection_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:00.994275766Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"d450b92e-bfb6-492a-a343-40ce8a27cc9c","duration":"3.662588ms","path":"/admin/backfill_run_storage_stats"},"message":"Finished request d450b92e-bfb6-492a-a343-40ce8a27cc9c in %!f(time.Duration=3662588) seconds on /admin/backfill_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:00.99449096Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"191983c1-a4a6-48dd-8491-00730402ca84","duration":"1.142404ms","path":"/admin/garbage_collect_artifacts"},"message":"Finished request 191983c1-a4a6-48dd-8491-00730402ca84 in %!f(time.Duration=1142404) seconds on /admin/garbage_collect_artifacts","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:01.107410932Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"820452ca-989b-44a1-b898-d03b6efc56cd","duration":"121.146505ms","path":"/admin/clean_run_storage_stats"},"message":"Finished request 820452ca-989b-44a1-b898-d03b6efc56cd in %!f(time.Duration=121146505) seconds on /admin/clean_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:01.144685967Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"5f8ec1b9-55d0-4f31-8945-8939afe5a2a0","duration":"151.500505ms","path":"/admin/backfill_run_compute_hours"},"message":"Finished request 5f8ec1b9-55d0-4f31-8945-8939afe5a2a0 in %!f(time.Duration=151500505) seconds on /admin/backfill_run_compute_hours","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:20.358483536Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"bbbb2d52-80c9-4b65-b207-4ecfed07b5b2","path":"/oidc/auth"},"message":"Starting request bbbb2d52-80c9-4b65-b207-4ecfed07b5b2 on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:20.361887202Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"bbbb2d52-80c9-4b65-b207-4ecfed07b5b2","duration":"3.44419ms","path":"/oidc/auth"},"message":"Finished request bbbb2d52-80c9-4b65-b207-4ecfed07b5b2 in %!f(time.Duration=3444190) seconds on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:20.500892598Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"62373ff8-e667-4612-8c8b-1b3d93c9f4c2","path":"/oidc/auth"},"message":"Starting request 62373ff8-e667-4612-8c8b-1b3d93c9f4c2 on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:20.501685806Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"a0efae03-bf3a-4ad5-86fa-b96b15c7d9a9","path":"/oidc/auth"},"message":"Starting request a0efae03-bf3a-4ad5-86fa-b96b15c7d9a9 on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:20.503937818Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"62373ff8-e667-4612-8c8b-1b3d93c9f4c2","duration":"3.071024ms","path":"/oidc/auth"},"message":"Finished request 62373ff8-e667-4612-8c8b-1b3d93c9f4c2 in %!f(time.Duration=3071024) seconds on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:20.505142361Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"a0efae03-bf3a-4ad5-86fa-b96b15c7d9a9","duration":"3.496621ms","path":"/oidc/auth"},"message":"Finished request a0efae03-bf3a-4ad5-86fa-b96b15c7d9a9 in %!f(time.Duration=3496621) seconds on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:20.60123366Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"84a74e1a-a85b-4f8d-bec0-d65b87c10f9f","path":"/oidc/auth"},"message":"Starting request 84a74e1a-a85b-4f8d-bec0-d65b87c10f9f on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:20.601468233Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"98e12913-53b3-4f12-a9fb-986fbd14f4ee","path":"/oidc/auth"},"message":"Starting request 98e12913-53b3-4f12-a9fb-986fbd14f4ee on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:20.60355511Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"84a74e1a-a85b-4f8d-bec0-d65b87c10f9f","duration":"2.345327ms","path":"/oidc/auth"},"message":"Finished request 84a74e1a-a85b-4f8d-bec0-d65b87c10f9f in %!f(time.Duration=2345327) seconds on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:20.603795736Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"98e12913-53b3-4f12-a9fb-986fbd14f4ee","duration":"2.352447ms","path":"/oidc/auth"},"message":"Finished request 98e12913-53b3-4f12-a9fb-986fbd14f4ee in %!f(time.Duration=2352447) seconds on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:22.456818387Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"93958437-d5d0-41b7-8b6f-881bb978a4ca","path":"/oidc/auth"},"message":"Starting request 93958437-d5d0-41b7-8b6f-881bb978a4ca on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:22.463885171Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"93958437-d5d0-41b7-8b6f-881bb978a4ca","duration":"7.045837ms","path":"/oidc/auth"},"message":"Finished request 93958437-d5d0-41b7-8b6f-881bb978a4ca in %!f(time.Duration=7045837) seconds on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:22.469086507Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"c2a2e0d7-775c-4a84-8f14-813e5ea1d828","path":"/oidc/auth"},"message":"Starting request c2a2e0d7-775c-4a84-8f14-813e5ea1d828 on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:22.469528494Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"f59299c1-fb84-4327-8983-60868062a780","path":"/oidc/auth"},"message":"Starting request f59299c1-fb84-4327-8983-60868062a780 on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:22.469888249Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/session.go:132","pid":62},"data":{"domain":"10.20.84.191","method":"GET","origin":"http://10.20.84.191:8080","request":"/oidc/auth","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.51 Safari/537.36"},"message":"Request origin not allowed: http://10.20.84.191:8080","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:22.470151852Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/session.go:132","pid":62},"data":{"method":"GET","origin":"http://10.20.84.191:8080","request":"/oidc/auth","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.51 Safari/537.36","domain":"10.20.84.191"},"message":"Request origin not allowed: http://10.20.84.191:8080","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:22.470920303Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"c2a2e0d7-775c-4a84-8f14-813e5ea1d828","duration":"1.880337ms","path":"/oidc/auth"},"message":"Finished request c2a2e0d7-775c-4a84-8f14-813e5ea1d828 in %!f(time.Duration=1880337) seconds on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:22.471129405Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"f59299c1-fb84-4327-8983-60868062a780","duration":"1.644863ms","path":"/oidc/auth"},"message":"Finished request f59299c1-fb84-4327-8983-60868062a780 in %!f(time.Duration=1644863) seconds on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:22.568181812Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"f8d2f8fd-a365-449b-a8fc-098098ae2247","path":"/oidc/auth"},"message":"Starting request f8d2f8fd-a365-449b-a8fc-098098ae2247 on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:22.56892691Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/session.go:132","pid":62},"data":{"method":"GET","origin":"http://10.20.84.191:8080","request":"/oidc/auth","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.51 Safari/537.36","domain":"10.20.84.191"},"message":"Request origin not allowed: http://10.20.84.191:8080","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:22.569053973Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"f8d2f8fd-a365-449b-a8fc-098098ae2247","duration":"897.659µs","path":"/oidc/auth"},"message":"Finished request f8d2f8fd-a365-449b-a8fc-098098ae2247 in %!f(time.Duration=897659) seconds on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:22.779917444Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"ce0d50bb-fede-4d81-925b-262193e492c8","path":"/oidc/auth"},"message":"Starting request ce0d50bb-fede-4d81-925b-262193e492c8 on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:22.781321736Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/session.go:132","pid":62},"data":{"domain":"10.20.84.191","method":"GET","origin":"http://10.20.84.191:8080","request":"/oidc/auth","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.51 Safari/537.36"},"message":"Request origin not allowed: http://10.20.84.191:8080","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:22.781451015Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"ce0d50bb-fede-4d81-925b-262193e492c8","duration":"1.573415ms","path":"/oidc/auth"},"message":"Finished request ce0d50bb-fede-4d81-925b-262193e492c8 in %!f(time.Duration=1573415) seconds on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:24.594079581Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"0e8e0def-751c-4219-ba5f-73339c46a721","path":"/oidc/auth"},"message":"Starting request 0e8e0def-751c-4219-ba5f-73339c46a721 on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:24.595069354Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/session.go:132","pid":62},"data":{"domain":"10.20.84.191","method":"GET","origin":"http://10.20.84.191:8080","request":"/oidc/auth","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.51 Safari/537.36"},"message":"Request origin not allowed: http://10.20.84.191:8080","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:24.595198811Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"0e8e0def-751c-4219-ba5f-73339c46a721","duration":"1.16209ms","path":"/oidc/auth"},"message":"Finished request 0e8e0def-751c-4219-ba5f-73339c46a721 in %!f(time.Duration=1162090) seconds on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:24.692135098Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"31e68d54-cf8d-4404-bf0d-d44195aed01f","path":"/oidc/auth"},"message":"Starting request 31e68d54-cf8d-4404-bf0d-d44195aed01f on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:24.692859176Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/session.go:132","pid":62},"data":{"userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.51 Safari/537.36","domain":"10.20.84.191","method":"GET","origin":"http://10.20.84.191:8080","request":"/oidc/auth"},"message":"Request origin not allowed: http://10.20.84.191:8080","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:24.692974177Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"31e68d54-cf8d-4404-bf0d-d44195aed01f","duration":"861.743µs","path":"/oidc/auth"},"message":"Finished request 31e68d54-cf8d-4404-bf0d-d44195aed01f in %!f(time.Duration=861743) seconds on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:24.799954038Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"a49f81fd-4e80-445f-9cca-25b606e8df4d","path":"/oidc/auth"},"message":"Starting request a49f81fd-4e80-445f-9cca-25b606e8df4d on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:24.800705902Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/session.go:132","pid":62},"data":{"origin":"http://10.20.84.191:8080","request":"/oidc/auth","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.51 Safari/537.36","domain":"10.20.84.191","method":"GET"},"message":"Request origin not allowed: http://10.20.84.191:8080","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:24.800825844Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"a49f81fd-4e80-445f-9cca-25b606e8df4d","duration":"899.222µs","path":"/oidc/auth"},"message":"Finished request a49f81fd-4e80-445f-9cca-25b606e8df4d in %!f(time.Duration=899222) seconds on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:24.890937595Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"063a0db1-5189-447a-888d-d131479e8081","path":"/oidc/auth"},"message":"Starting request 063a0db1-5189-447a-888d-d131479e8081 on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:24.891606698Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/session.go:132","pid":62},"data":{"domain":"10.20.84.191","method":"GET","origin":"http://10.20.84.191:8080","request":"/oidc/auth","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.51 Safari/537.36"},"message":"Request origin not allowed: http://10.20.84.191:8080","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:24.891730774Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"063a0db1-5189-447a-888d-d131479e8081","duration":"820.374µs","path":"/oidc/auth"},"message":"Finished request 063a0db1-5189-447a-888d-d131479e8081 in %!f(time.Duration=820374) seconds on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:24.99628551Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"d313a5cf-fb36-4b06-9359-7a13f5bf9b61","path":"/oidc/auth"},"message":"Starting request d313a5cf-fb36-4b06-9359-7a13f5bf9b61 on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:24.997054295Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/session.go:132","pid":62},"data":{"request":"/oidc/auth","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.51 Safari/537.36","domain":"10.20.84.191","method":"GET","origin":"http://10.20.84.191:8080"},"message":"Request origin not allowed: http://10.20.84.191:8080","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:24.997171655Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"d313a5cf-fb36-4b06-9359-7a13f5bf9b61","duration":"909.164µs","path":"/oidc/auth"},"message":"Finished request d313a5cf-fb36-4b06-9359-7a13f5bf9b61 in %!f(time.Duration=909164) seconds on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:29.357558231Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"53a7341f-8cc1-4da4-b893-6bdf189163e7","path":"/oidc/auth"},"message":"Starting request 53a7341f-8cc1-4da4-b893-6bdf189163e7 on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:29.358832827Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/session.go:132","pid":62},"data":{"domain":"10.20.84.191","method":"GET","origin":"http://10.20.84.191:8080","request":"/oidc/auth","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.51 Safari/537.36"},"message":"Request origin not allowed: http://10.20.84.191:8080","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:29.358962127Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"53a7341f-8cc1-4da4-b893-6bdf189163e7","duration":"1.446988ms","path":"/oidc/auth"},"message":"Finished request 53a7341f-8cc1-4da4-b893-6bdf189163e7 in %!f(time.Duration=1446988) seconds on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:31.518127018Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"75c439b6-e5ae-404b-bcbb-70560e8f9ef8","path":"/oidc/auth"},"message":"Starting request 75c439b6-e5ae-404b-bcbb-70560e8f9ef8 on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:31.519090761Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/session.go:132","pid":62},"data":{"domain":"10.20.84.191","method":"GET","origin":"http://10.20.84.191:8080","request":"/oidc/auth","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.51 Safari/537.36"},"message":"Request origin not allowed: http://10.20.84.191:8080","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:31.519221735Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"75c439b6-e5ae-404b-bcbb-70560e8f9ef8","duration":"1.131436ms","path":"/oidc/auth"},"message":"Finished request 75c439b6-e5ae-404b-bcbb-70560e8f9ef8 in %!f(time.Duration=1131436) seconds on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:32.269252358Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"2c7f2a37-6f4b-4057-907c-016a758fe215","path":"/oidc/auth"},"message":"Starting request 2c7f2a37-6f4b-4057-907c-016a758fe215 on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:32.270295179Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/session.go:132","pid":62},"data":{"userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.51 Safari/537.36","domain":"10.20.84.191","method":"GET","origin":"http://10.20.84.191:8080","request":"/oidc/auth"},"message":"Request origin not allowed: http://10.20.84.191:8080","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:32.270426778Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"2c7f2a37-6f4b-4057-907c-016a758fe215","duration":"1.217108ms","path":"/oidc/auth"},"message":"Finished request 2c7f2a37-6f4b-4057-907c-016a758fe215 in %!f(time.Duration=1217108) seconds on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:32.889792844Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"142f41f5-fd83-47a1-baa9-f7b75723110f","path":"/oidc/auth"},"message":"Starting request 142f41f5-fd83-47a1-baa9-f7b75723110f on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:32.89096156Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/session.go:132","pid":62},"data":{"origin":"http://10.20.84.191:8080","request":"/oidc/auth","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.51 Safari/537.36","domain":"10.20.84.191","method":"GET"},"message":"Request origin not allowed: http://10.20.84.191:8080","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:32.891089012Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"142f41f5-fd83-47a1-baa9-f7b75723110f","duration":"1.330469ms","path":"/oidc/auth"},"message":"Finished request 142f41f5-fd83-47a1-baa9-f7b75723110f in %!f(time.Duration=1330469) seconds on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:44.986698572Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"d224d16b-057b-4183-ac16-746f6f3a3c1e","path":"/oidc/auth"},"message":"Starting request d224d16b-057b-4183-ac16-746f6f3a3c1e on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:44.987796438Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/session.go:132","pid":62},"data":{"domain":"10.20.84.191","method":"GET","origin":"http://10.20.84.191:8080","request":"/oidc/auth","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.51 Safari/537.36"},"message":"Request origin not allowed: http://10.20.84.191:8080","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:11:44.987957386Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"d224d16b-057b-4183-ac16-746f6f3a3c1e","duration":"1.303019ms","path":"/oidc/auth"},"message":"Finished request d224d16b-057b-4183-ac16-746f6f3a3c1e in %!f(time.Duration=1303019) seconds on /oidc/auth","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:12:01.03855553Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"a456ff48-2da8-4c5c-9f52-9ebf2d6518e4","path":"/admin/delete_artifact_collections"},"message":"Starting request a456ff48-2da8-4c5c-9f52-9ebf2d6518e4 on /admin/delete_artifact_collections","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:12:01.039755909Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"015c9b3d-7344-4991-8812-ea3967ec785d","path":"/admin/recompute_artifact_storage_bytes"},"message":"Starting request 015c9b3d-7344-4991-8812-ea3967ec785d on /admin/recompute_artifact_storage_bytes","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:12:01.040012597Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"a456ff48-2da8-4c5c-9f52-9ebf2d6518e4","duration":"1.487975ms","path":"/admin/delete_artifact_collections"},"message":"Finished request a456ff48-2da8-4c5c-9f52-9ebf2d6518e4 in %!f(time.Duration=1487975) seconds on /admin/delete_artifact_collections","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:12:01.040460343Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"015c9b3d-7344-4991-8812-ea3967ec785d","duration":"721.479µs","path":"/admin/recompute_artifact_storage_bytes"},"message":"Finished request 015c9b3d-7344-4991-8812-ea3967ec785d in %!f(time.Duration=721479) seconds on /admin/recompute_artifact_storage_bytes","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:12:01.041981195Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"e341f8da-a218-4ff3-8a44-c3675d1c81fd","path":"/admin/garbage_collect_artifacts"},"message":"Starting request e341f8da-a218-4ff3-8a44-c3675d1c81fd on /admin/garbage_collect_artifacts","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:12:01.042558945Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"fb07392f-f8ae-4472-91e7-b43569d8e646","path":"/admin/early_stop"},"message":"Starting request fb07392f-f8ae-4472-91e7-b43569d8e646 on /admin/early_stop","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:12:01.042849671Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"e341f8da-a218-4ff3-8a44-c3675d1c81fd","duration":"880.622µs","path":"/admin/garbage_collect_artifacts"},"message":"Finished request e341f8da-a218-4ff3-8a44-c3675d1c81fd in %!f(time.Duration=880622) seconds on /admin/garbage_collect_artifacts","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:12:01.043147367Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"278c4f64-1522-4921-ab01-2766e1f9f392","path":"/admin/clean_run_storage_stats"},"message":"Starting request 278c4f64-1522-4921-ab01-2766e1f9f392 on /admin/clean_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:12:01.043319426Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"fb07392f-f8ae-4472-91e7-b43569d8e646","duration":"778.321µs","path":"/admin/early_stop"},"message":"Finished request fb07392f-f8ae-4472-91e7-b43569d8e646 in %!f(time.Duration=778321) seconds on /admin/early_stop","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:12:01.043997961Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"5b1bec70-3457-4d33-a92f-860a126abb2d","path":"/admin/delete_projects"},"message":"Starting request 5b1bec70-3457-4d33-a92f-860a126abb2d on /admin/delete_projects","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:12:01.044060158Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"a04c0642-fd63-4cd0-a356-ef4408be1478","path":"/admin/build_artifact_deadlists"},"message":"Starting request a04c0642-fd63-4cd0-a356-ef4408be1478 on /admin/build_artifact_deadlists","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:12:01.044112501Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"95268ef6-6041-47de-bc5d-9d795be2f5aa","path":"/admin/backfill_run_storage_stats"},"message":"Starting request 95268ef6-6041-47de-bc5d-9d795be2f5aa on /admin/backfill_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:12:01.04422489Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"ab988cc7-30b4-4f50-8e8a-94e069137f58","path":"/admin/backfill_run_compute_hours"},"message":"Starting request ab988cc7-30b4-4f50-8e8a-94e069137f58 on /admin/backfill_run_compute_hours","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:12:01.045076107Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"a04c0642-fd63-4cd0-a356-ef4408be1478","duration":"1.022691ms","path":"/admin/build_artifact_deadlists"},"message":"Finished request a04c0642-fd63-4cd0-a356-ef4408be1478 in %!f(time.Duration=1022691) seconds on /admin/build_artifact_deadlists","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:12:01.045259708Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"5b1bec70-3457-4d33-a92f-860a126abb2d","duration":"1.282783ms","path":"/admin/delete_projects"},"message":"Finished request 5b1bec70-3457-4d33-a92f-860a126abb2d in %!f(time.Duration=1282783) seconds on /admin/delete_projects","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:12:01.045721805Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"8db60eac-60f9-4004-9808-b6e42ce7256f","path":"/admin/clean_artifact_collection_storage_stats"},"message":"Starting request 8db60eac-60f9-4004-9808-b6e42ce7256f on /admin/clean_artifact_collection_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:12:01.046457225Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"8db60eac-60f9-4004-9808-b6e42ce7256f","duration":"760.798µs","path":"/admin/clean_artifact_collection_storage_stats"},"message":"Finished request 8db60eac-60f9-4004-9808-b6e42ce7256f in %!f(time.Duration=760798) seconds on /admin/clean_artifact_collection_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:12:01.047785027Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"95268ef6-6041-47de-bc5d-9d795be2f5aa","duration":"3.698585ms","path":"/admin/backfill_run_storage_stats"},"message":"Finished request 95268ef6-6041-47de-bc5d-9d795be2f5aa in %!f(time.Duration=3698585) seconds on /admin/backfill_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:12:01.15348247Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"278c4f64-1522-4921-ab01-2766e1f9f392","duration":"110.343338ms","path":"/admin/clean_run_storage_stats"},"message":"Finished request 278c4f64-1522-4921-ab01-2766e1f9f392 in %!f(time.Duration=110343338) seconds on /admin/clean_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:12:01.175930258Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"ab988cc7-30b4-4f50-8e8a-94e069137f58","duration":"131.703637ms","path":"/admin/backfill_run_compute_hours"},"message":"Finished request ab988cc7-30b4-4f50-8e8a-94e069137f58 in %!f(time.Duration=131703637) seconds on /admin/backfill_run_compute_hours","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:13:00.10575591Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"a784d5b1-5e99-493e-a79c-42214cf65059","path":"/admin/garbage_collect_artifacts"},"message":"Starting request a784d5b1-5e99-493e-a79c-42214cf65059 on /admin/garbage_collect_artifacts","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:13:00.106434318Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"3c1612c7-4b30-435b-8f51-777eeff753f5","path":"/admin/early_stop"},"message":"Starting request 3c1612c7-4b30-435b-8f51-777eeff753f5 on /admin/early_stop","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:13:00.106596883Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"a6d60774-d21e-46de-804f-fb756edb635b","path":"/admin/backfill_run_compute_hours"},"message":"Starting request a6d60774-d21e-46de-804f-fb756edb635b on /admin/backfill_run_compute_hours","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:13:00.113714028Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"63a4219b-9cf6-4ef6-9336-dac2fd2573ac","path":"/admin/delete_artifact_collections"},"message":"Starting request 63a4219b-9cf6-4ef6-9336-dac2fd2573ac on /admin/delete_artifact_collections","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:13:00.113760755Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"3c1612c7-4b30-435b-8f51-777eeff753f5","duration":"7.342913ms","path":"/admin/early_stop"},"message":"Finished request 3c1612c7-4b30-435b-8f51-777eeff753f5 in %!f(time.Duration=7342913) seconds on /admin/early_stop","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:13:00.118291404Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"a784d5b1-5e99-493e-a79c-42214cf65059","duration":"12.565816ms","path":"/admin/garbage_collect_artifacts"},"message":"Finished request a784d5b1-5e99-493e-a79c-42214cf65059 in %!f(time.Duration=12565816) seconds on /admin/garbage_collect_artifacts","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:13:00.120210375Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"63a4219b-9cf6-4ef6-9336-dac2fd2573ac","duration":"6.50553ms","path":"/admin/delete_artifact_collections"},"message":"Finished request 63a4219b-9cf6-4ef6-9336-dac2fd2573ac in %!f(time.Duration=6505530) seconds on /admin/delete_artifact_collections","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:13:00.12578421Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"92a280cf-4a21-4e71-9638-be7573718801","path":"/admin/delete_projects"},"message":"Starting request 92a280cf-4a21-4e71-9638-be7573718801 on /admin/delete_projects","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:13:00.126392086Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"3591b7eb-0add-4d61-af3a-c2f4bd65cb4e","path":"/admin/backfill_run_storage_stats"},"message":"Starting request 3591b7eb-0add-4d61-af3a-c2f4bd65cb4e on /admin/backfill_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:13:00.130403684Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"9ad9237d-bd49-4099-adc7-f4fcefe945dc","path":"/admin/build_artifact_deadlists"},"message":"Starting request 9ad9237d-bd49-4099-adc7-f4fcefe945dc on /admin/build_artifact_deadlists","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:13:00.134039262Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"9ad9237d-bd49-4099-adc7-f4fcefe945dc","duration":"3.647657ms","path":"/admin/build_artifact_deadlists"},"message":"Finished request 9ad9237d-bd49-4099-adc7-f4fcefe945dc in %!f(time.Duration=3647657) seconds on /admin/build_artifact_deadlists","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:13:00.134508398Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"92a280cf-4a21-4e71-9638-be7573718801","duration":"8.732465ms","path":"/admin/delete_projects"},"message":"Finished request 92a280cf-4a21-4e71-9638-be7573718801 in %!f(time.Duration=8732465) seconds on /admin/delete_projects","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:13:00.136418253Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"6394f280-0525-4acd-88e1-4ef7abe8c758","path":"/admin/clean_artifact_collection_storage_stats"},"message":"Starting request 6394f280-0525-4acd-88e1-4ef7abe8c758 on /admin/clean_artifact_collection_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:13:00.137069856Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"6394f280-0525-4acd-88e1-4ef7abe8c758","duration":"663.664µs","path":"/admin/clean_artifact_collection_storage_stats"},"message":"Finished request 6394f280-0525-4acd-88e1-4ef7abe8c758 in %!f(time.Duration=663664) seconds on /admin/clean_artifact_collection_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:13:00.141051375Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"0b318e5c-c6bd-4a27-b3aa-5d20e4fb8756","path":"/admin/recompute_artifact_storage_bytes"},"message":"Starting request 0b318e5c-c6bd-4a27-b3aa-5d20e4fb8756 on /admin/recompute_artifact_storage_bytes","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:13:00.141042825Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"3591b7eb-0add-4d61-af3a-c2f4bd65cb4e","duration":"14.651687ms","path":"/admin/backfill_run_storage_stats"},"message":"Finished request 3591b7eb-0add-4d61-af3a-c2f4bd65cb4e in %!f(time.Duration=14651687) seconds on /admin/backfill_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:13:00.141698203Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"0b318e5c-c6bd-4a27-b3aa-5d20e4fb8756","duration":"655.251µs","path":"/admin/recompute_artifact_storage_bytes"},"message":"Finished request 0b318e5c-c6bd-4a27-b3aa-5d20e4fb8756 in %!f(time.Duration=655251) seconds on /admin/recompute_artifact_storage_bytes","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:13:00.144156204Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:100","pid":62},"data":{"requestID":"d6d9b5cb-76f6-427f-a943-4bff7a42cf9b","path":"/admin/clean_run_storage_stats"},"message":"Starting request d6d9b5cb-76f6-427f-a943-4bff7a42cf9b on /admin/clean_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:13:00.276385125Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"d6d9b5cb-76f6-427f-a943-4bff7a42cf9b","duration":"132.230427ms","path":"/admin/clean_run_storage_stats"},"message":"Finished request d6d9b5cb-76f6-427f-a943-4bff7a42cf9b in %!f(time.Duration=132230427) seconds on /admin/clean_run_storage_stats","logging.googleapis.com/trace":""}
{"level":"INFO","time":"2022-03-15T16:13:00.303202412Z","info":{"program":"gorilla","source":"mnt/ramdisk/core/services/gorilla/logging.go:104","pid":62},"data":{"requestID":"a6d60774-d21e-46de-804f-fb756edb635b","duration":"196.608074ms","path":"/admin/backfill_run_compute_hours"},"message":"Finished request a6d60774-d21e-46de-804f-fb756edb635b in %!f(time.Duration=196608074) seconds on /admin/backfill_run_compute_hours","logging.googleapis.com/trace":""}
vanpelt commented 2 years ago

@Lionelsy if you want it to listen on port 8096 you need to configure it with the --port option. The host environment variable just tells the server how users will access the service. Most customers put the service behind DNS with a load balancer that terminates SSL, i.e. -e HOST=https://wandb.mydomaon.com. For this to work on port 8096 assuming you're accessing it from this IP address:

wandb local --port 8960 -e HOST=http://10.20.84.191:8960

I'm curious, why aren't you just using default port 8080?

Lionelsy commented 2 years ago

It works! Thanks for your rapid reply. Because some ports are reserved in our lab server. lol.

Guitaricet commented 2 years ago

Thank you for help! What could I do if my server is not accessible via IP? For example, if it is inside a (pretty complicated and secured) Kubernetes cluster and I only access it via kubectl exec or kubectl port-forward

@littlerookie just add -e HOST=http://server_ip:8080 to your docker run command.

MBakirWB commented 2 years ago

@Guitaricet , to get a shell to a running container of wandb, use kubectl exec -it wandb-pod-name bash.

Guitaricet commented 2 years ago

@MBakirWB thank you, but my question was about specifying HOST environment variable.

MBakirWB commented 2 years ago

Hi @Guitaricet ,the default HOST value should be fine assuming you are port-forwarding to 8080. Port forwarding just exposes some remote host on a users local host, i.e. 127.0.0.1. The default HOST is http://127.0.0.1:8080/

Guitaricet commented 2 years ago

Thank you @MBakirWB, that's good to know. I wasn't comfortable forwarding the app to my local 8080, so I changed it to a custom port (let's say 8899). If there is no implications of that, I will try it out.

rachid-drago commented 1 year ago

Adding Host environment variable will fix the problem docker run --rm -dit -v wandb:/vol -p 8099:8080 --name wandb-local -e HOST=http://YourIP:8099 wandb/local

celeboom commented 8 months ago

the same problem when I enter the license. but my log displays “request origin not allowed: http://127.0.0.1:8080” Version is 0.50.3

stomir commented 8 months ago

the same problem when I enter the license. but my log displays “request origin not allowed: http://127.0.0.1:8080” Version is 0.50.3 The solution seems to be accessing wandb via localhost:8080 (rather than 127.0.0.1)

tao-shen commented 7 months ago

Ahh, I see what's happening. When you're accessing the instance not from 127.0.0.1:8080 you'll need to tell the instance how you'll be accessing it. You can do this by specifying the HOST environment variable on boot, i.e.

docker stop wandb-local
wandb local -e HOST=http://10.13.71.162:8083

Or by setting the Host value on the settings page from 127.0.0.1 and saving the settings.

solved! many thanks!