Since this might run for a while on a remote server, it might be nice to use some tool for remote monitoring.
Maybe use WandB, https://docs.wandb.ai/quickstart or something else.
***UPDATE 21.06.2023
I think that W&B is a little too heavy weight and requires setting up an account. Maybe search for easier way to do remote monitoring,
if any. I think that this can be aborted. If something breakes at Zach's side, he can just send me the logs. I feel like this is too much and not worth the time.
Since this might run for a while on a remote server, it might be nice to use some tool for remote monitoring. Maybe use WandB, https://docs.wandb.ai/quickstart or something else.
***UPDATE 21.06.2023 I think that W&B is a little too heavy weight and requires setting up an account. Maybe search for easier way to do remote monitoring, if any. I think that this can be aborted. If something breakes at Zach's side, he can just send me the logs. I feel like this is too much and not worth the time.