search

BMW-InnovationLab / BMW-YOLOv4-Training-Automation

This repository allows you to get started with training a state-of-the-art Deep Learning model with little to no configuration needed! You provide your labeled dataset or label your dataset using our BMW-LabelTool-Lite and you can start the training right away and monitor it in many different ways like TensorBoard or a custom REST API and GUI. NoCode training with YOLOv4 and YOLOV3 has never been so easy.
BSD 3-Clause "New" or "Revised" License
642 stars 127 forks source link

where does the tensorflow logs files are saved #17

Closed lidorshimoni closed 4 years ago

lidorshimoni commented 4 years ago

I want to see the tensorboard after the training has finished, so I'm trying to use:

tensorboard --logdir path/to/logs where is the "path/to/logs"? the closest thing I saw was the "yolo_events.log" but I couldn't get this to work.

hadikoub commented 4 years ago

The training logs are stored inside the docker container and not exported to the host folder and therefore could not be viewed. they are stored inside ./runs

lidorshimoni commented 4 years ago

in order to use the tensorboard logs after the training has finished i change the "run_docker_linux_gpu.sh" file

# start docker
sudo docker run  --runtime=nvidia -it -e TRAIN_NAME=$container_name -e TRAIN_START_TIME="$(date '+%Y%m%d_%H:%M:%S')" $ports  -v $folder_path:/training/assets -v $(pwd)/trainings:/training/custom_training -v /etc/timezone:/etc/timezone:ro -v /etc/localtime:/etc/localtime:ro --name $container_name darknet_yolov4_gpu:1 ; 

# copy tensorboard logs
mkdir -p tensorboard_logs/$container_name
docker cp $container_name:/training/runs tensorboard_logs/$container_name

# clean container
sudo docker container rm $container_name