japie1235813
commented
2 years ago Without being able to reproduce the case myself I can only think about couples of guess: The error occurs in this line when tf tried to load the check points
- check your
checkpointfile, it containsmodel.ckpt-178based on the error message - check
model.ckpt-178exists in /ckpt/..: the last one I see is 'model.ckpt-0.data-00000-of-00001' from your log. If there is no such model name, probably the name is somewhere not set in your training script - check the tutorial and see if anything wrong with the usage of
.writehttps://www.tensorflow.org/io/tutorials/azure#read_and_write_files_to_azure_storage_with_tensorflow
ben-omji
Environment information (required)
Diagnostics
Diagnostics output
`````` --- check: autoidentify INFO: diagnose_tensorboard.py version e43767ef2b648d0d5d57c00f38ccbd38390e38da --- check: general INFO: sys.version_info: sys.version_info(major=3, minor=8, micro=10, releaselevel='final', serial=0) INFO: os.name: posix INFO: os.uname(): posix.uname_result(sysname='Linux', nodename='a601721015eb', release='5.4.0-74-generic', version='#83-Ubuntu SMP Sat May 8 02:35:39 UTC 2021', machine='x86_64') INFO: sys.getwindowsversion(): N/A --- check: package_management INFO: has conda-meta: False INFO: $VIRTUAL_ENV: None --- check: installed_packages INFO: installed: tensorboard==2.8.0 WARNING: no installation among: ['tensorflow', 'tensorflow-gpu', 'tf-nightly', 'tf-nightly-2.0-preview', 'tf-nightly-gpu', 'tf-nightly-gpu-2.0-preview'] INFO: installed: tf-estimator-nightly==2.8.0.dev2021122109 INFO: installed: tensorboard-data-server==0.6.1 --- check: tensorboard_python_version INFO: tensorboard.version.VERSION: '2.8.0' --- check: tensorflow_python_version INFO: tensorflow.__version__: '2.8.0' INFO: tensorflow.__git_version__: 'v2.8.0-rc1-32-g3f878cff5b6' --- check: tensorboard_data_server_version INFO: data server binary: '/usr/local/lib/python3.8/dist-packages/tensorboard_data_server/bin/server' INFO: data server binary version: b'rustboard 0.6.1' --- check: tensorboard_binary_path INFO: which tensorboard: b'/usr/local/bin/tensorboard\n' --- check: addrinfos socket.has_ipv6 = True socket.AF_UNSPEC =Issue description
I can get list of model ckeckpoint directory with tensorflow gfile and tensorflow-io.
But When I try to set the Azure blob storage path to logdir of tensorboard, I always get "No dashboards are active for the current data set."
Reproduction steps
I downloaded tensorflow-io with pip and set accesskey on env var
Then I check the connection with storage through python script.
The connection is looks good, so I try to log the blob directory with tensorboard.
I got the log below, and face to "No dashboards are active for the current data set." page.
Do you guys have any idea for this problem?
Thanks in advance.