agent scheduler causes hang due to unsafe path handling

[vladmandic]

in scripts/task_scheduler.py method get_checkpoint_choices gets executed on server startup, but it can cause infinite loop if checkpoint is an absolute path (starting with /) since condition checkpoint_dir != "" will always be true.

common scenario where this happens is if entire models dir is set to be on a different mount, for example starting sdnext with --models-dir /mnt/f/models

quick workaround is

while checkpoint_dir != "" and checkpoint_dir != "/":

but in general, assuming that any parent path in absolute path can be a checkpoint dir is not correct.

[vladmandic]

ping as this is a massive issue as it brings core app down, not just agent scheduler. cc @artventuredev @tungnguyensipher

[artventuredev]

Hi @vladmandic, I just added a quick fix as your suggestion.

I'm doing some debugging but list_checkpoint_tiles always returns the checkpoints with model_path stripped, even when I have --models-dir points to a different dir.

[vladmandic]

thanks. and it depends, if models-dir is on a different drive, then models-path cannot be stripped and its returned as absolute (windows cannot do relative paths cross-drives).

[artventuredev]

Understood. I’ll investigate this issue further when I’m at my windows PC.