search

Azure / batch-shipyard

Simplify HPC and Batch workloads on Azure
MIT License
277 stars 121 forks source link

task freeze on "active" state #251

Closed veonua closed 5 years ago

veonua commented 5 years ago

Problem Description

../batch-shipyard-3.6.1-cli-linux-x86_64 jobs add --tail stdout.txt creates the task but it never gets to running state

Batch Shipyard Version

3.6.1-cli-linux-x86_64

Steps to Reproduce

usually it happens when I reuse pool name (removing old one from azure portal) but basically I just create a new pool ../batch-shipyard-3.6.1-cli-linux-x86_64 pool add and start the task ../batch-shipyard-3.6.1-cli-linux-x86_64 jobs add --tail stdout.txt

Expected Results

task gets prepared and running

Actual Results

task stays in the active state (waiting to start) while node is idle

Additonal Comments

not sure how to debug this, outputs look ok

veonua commented 5 years ago

ok it seems app needs better validation if pool removed properly, because even

../batch-shipyard-3.6.1-cli-linux-x86_64 pool del

left pool in the Azure dashboard

alfpark commented 5 years ago

Pools are not instantaneously deleted.

veonua commented 5 years ago

there must be some tests about pool integrity, otherwise user just gets random errors