search

Linaqruf / kohya-trainer

Adapted from https://note.com/kohya_ss/n/nbf7ce8d80f29 for easier cloning
Apache License 2.0
1.86k stars 306 forks source link

TensorBoard Issues when trying to log #129

Open duskfallcrew opened 1 year ago

duskfallcrew commented 1 year ago

/content/kohya-trainer ERROR: Failed to launch TensorBoard (exited with 1). Contents of stderr: 2023-03-09 21:26:58.994358: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered 2023-03-09 21:27:00.755778: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib64-nvidia 2023-03-09 21:27:00.755978: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib64-nvidia 2023-03-09 21:27:00.756008: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.

NOTE: Using experimental fast data loading logic. To disable, pass "--load_fast=false" and report issues on GitHub. More details: https://github.com/tensorflow/tensorboard/issues/4784

Traceback (most recent call last): File "/usr/local/bin/tensorboard", line 8, in sys.exit(run_main()) File "/usr/local/lib/python3.9/dist-packages/tensorboard/main.py", line 46, in run_main app.run(tensorboard.main, flags_parser=tensorboard.configure) File "/usr/local/lib/python3.9/dist-packages/absl/app.py", line 308, in run _run_main(main, args) File "/usr/local/lib/python3.9/dist-packages/absl/app.py", line 254, in _run_main sys.exit(main(argv)) File "/usr/local/lib/python3.9/dist-packages/tensorboard/program.py", line 276, in main return runner(self.flags) or 0 File "/usr/local/lib/python3.9/dist-packages/tensorboard/program.py", line 292, in _run_serve_subcommand server = self._make_server() File "/usr/local/lib/python3.9/dist-packages/tensorboard/program.py", line 467, in _make_server app = application.TensorBoardWSGIApp( File "/usr/local/lib/python3.9/dist-packages/tensorboard/backend/application.py", line 139, in TensorBoardWSGIApp return TensorBoardWSGI( File "/usr/local/lib/python3.9/dist-packages/tensorboard/backend/application.py", line 252, in init raise ValueError( ValueError: Duplicate plugins for name projector