Above directory can be checked with tensorboard --bind_all --logdir=/path/to/log --port 8080 or tensorboard --bind_all --logdir_spec=exp1:/path/to/log1,exp2:/path/to/log2 --port 8080
outputs/{date}/{time}/weights -> output model weights
Use with cautionpgrep -f "python pretrain.py" -a will show all the training process. pkill -f "python pretrain.py" will kill them all.
tmux cheatsheet
tmux ls: Current tmux sessions
tmux new -s {session_name}: Create session with session_name
tmux attach -t {session_name}: Attach to existing session