Open RobbeSneyders opened 6 years ago
The metadata_path
property should be relative to the log directory. Does specifying a path such as ../not-logdir/first_dict
work?
Specifying a relative path does not work either.
With the following config:
embeddings {
tensor_name: "scope/first_tensor_name:0"
metadata_path: "../not-logdir/first_dict"
}
Tensorboard shows the following error message:
Error fetching metadata "gs://bucket/logdir/../not-logdir/first_dict" not found, or is not a file
Which can be expected, since os.path.join()
can't handle relative paths.
I was looking into os.path.normpath()
to collapse the relative path, but unfortunately it also removes double slashes:
>>> dict_path = os.path.join('gs://bucket/logdir', '../not-logdir/first_dict')
# 'gs://bucket/logdir/../not-logdir/first_dict'
>>> os.path.normpath(dict_path)
# 'gs:/bucket/not-logdir/first_dict'
Hmm, is there a chance you could move your mappings into the log directory? Or within a subdirectory of the log directory?
It is possible to copy the mappings into the log directory, however I think it would be valuable for Tensorboard to support other cases. Especially since this only seems to be a requirement for GCS, which I suspect is one of the main file systems used with tensorboard.
Does someone have a solution for metadata and embedding checkpoints in subfolders in the meantime?
I want to do the same thing because i want to compare embeddings of different training runs which are stored in subfolders of course.
Even with the TensorBoard callback i was not able to specify a correct metadata path.
The documentation is not clear and a link for details in the documation is not reachable anymore.
I really would like to use the embedding projector with metadata and multiple runs.
Here is an issue i created a few days ago with all the links to the tensorflow documentation:
https://github.com/tensorflow/tensorflow/issues/47565 ( TensorBoard Embedding Projector Metadata filepath format not clear )
I would really appreciate any small hint. Thank you.
System specifications
Problem specification
I am using a projector_config.pbtxt file to link my embedding tensors to their metadata. The config file is structured like this:
When I load the projector in tensorboard, I get the following error message:
Tensorboard uses my absolute path as a relative path.
Source code
The metadata path is fetched here and constructed to an absolute path here:
This snippet shows that the path should only be made absolute if it is not an absolute path already. The
os.path.isabs()
function however fails to detect if a GCP storage path is absolute.Is there another way to specify a metadata path outside of the logdir on GCP storage or should this be fixed?
The relevant code seems to be unchanged between my version 0.1.8 and the current version 1.6.0.