ranahanocka / MeshCNN

Convolutional Neural Network for 3D meshes in PyTorch
MIT License
1.61k stars 320 forks source link

Proper TensorBoard Usage #147

Open ghost opened 1 year ago

ghost commented 1 year ago

TensorBoard Usage

Hello, can someone help me properly use tensorboard to monitor the training process of human_seg? I am new to tensorboard so I apologize if this is a dumb question.

I am running python3.8-dev and python3.8-venv

Steps to reproduce error:

create python environment:

python3.8 -m venv PythonEnv/mesh
source !$/bin/activate

clone repository:

git clone --recurse-submodules https://github.com/ranahanocka/MeshCNN
cd MeshCNN

install dependencies:

pip install --upgrade pip
pip install cython==0.29.33
pip install numpy==1.23
pip install torchvision==0.14.1
pip install tensorboard

collect dataset:

source ./scripts/human_seg/get_data.sh

begin training:

source ./scripts/human_seg/train.sh

However, when I run tensorboard with:

tensorboard --host 0.0.0.0 --logdir=runs

I get the following error:

Screen Shot 2023-02-11 at 12 35 07 PM

And the result from bash:

tensorboard --host 0.0.0.0 --logdir=runs
2023-02-11 13:12:07.977485: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-02-11 13:12:08.505366: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory
2023-02-11 13:12:08.505415: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory
2023-02-11 13:12:08.505423: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
2023-02-11 13:12:08.902349: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-02-11 13:12:08.902530: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-02-11 13:12:08.921833: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudnn.so.8'; dlerror: libcudnn.so.8: cannot open shared object file: No such file or directory
2023-02-11 13:12:08.921861: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1934] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...

NOTE: Using experimental fast data loading logic. To disable, pass
    "--load_fast=false" and report issues on GitHub. More details:
    https://github.com/tensorflow/tensorboard/issues/4784

TensorBoard 2.11.2 at http://0.0.0.0:6006/ (Press CTRL+C to quit)

Can someone help?

Thank you!