Closed lynnlangit closed 4 days ago
Does running on 3 images mean one job with 3 tasks run in parallel? Or submitting 3 jobs with 1 each and seeing what happens.
Does running on 3 images mean one job with 3 tasks run in parallel? Or submitting 3 jobs with 1 each and seeing what happens.
Try running 1 job with 3 tasks with parallelism set to 3. Expected behavior would be to spin up 3 GCE VMs (depends on VM size specified, i.e. n2-standard-8, n2-standard-16.... and number of cores specified)
Blocked on: #202
Benchmark result for 140M pixels using spot cpu + gpu. (n1-standard-8 + 1 T4 gpu)
Job creation time: 2024-05-23 1:35:40 PM Job start time: 2024-05-23 1:39:37 PM Job finish time: 2024-05-23 1:46:59 PM Total time: 11min 19s
Time spent in container:
Total time: 349.75s Preprocess time: 34.14 Inference time: 103.84 Postprocess time: 176.05
Total time on Vertex AI: 427s (!!)
Why 80s slower on the vertex notebook?
This is long since complete, in various forms 📖 → 📓
Continue baselining using my GCP infra: