mikeizbicki / cmc-csci181-deeplearning

deep learning course materials
15 stars 6 forks source link

TensorBoard not working, log directory shows logs #51

Open clayton-north opened 4 years ago

clayton-north commented 4 years ago

image I'm having trouble getting my tensorboard to show anything, the log directory is filling it up with logs. I followed the instructions in part 3, here is (part of) my code:

if args.embed:
    category_embedding = torch.cat([model.fc_class.weight, torch.unsqueeze(model.fc_class.bias,dim=1)],dim=1)
    if args.log_dir is None:
        raise ValueError("no log directory")
    writer = SummaryWriter(log_dir=args.log_dir)
    writer.add_embedding(
        mat = category_embedding,
        metadata = all_categories,
        tag = 'category embedding'
    )

and my terminal is showing no errors when I run:

python3 names.py --train --data=corona.multilang100.jsonl.gz  --model=bert  --learning_rate=1e-3 --batch_size=4 --input_length=64 --data_format=headlines --embed 
device= cpu
len(vocabulary)= 76
len(lines_category)= 106767
len(all_categories)= 100
100 1% (112.79 sec) 15.1957 Lin Shi Xiu Xiao haBen Dang niYi Wei nonaiXin Xing koronaDui Ce nanoka?(Hu Na Xian Zhi ) / www.lepoint.fr ✗ (news.yahoo.co.jp)

and:

tensorboard dev upload --logdir logs
TensorFlow installation not found - running with reduced feature set.
TensorBoard.dev now supports the "graphs" plugin. To upload data for this plugin, please upgrade to TensorBoard 2.2.1 or newer.
Upload started and will continue reading any new data as it's added
to the logdir. To stop uploading, press Ctrl-C.
View your TensorBoard live at: https://tensorboard.dev/experiment/xwLtbfesQpyBCe94Ckyw3w/

I'm not sure how to go about solving this issue, any suggestions ?

mikeizbicki commented 4 years ago

I believe that tensorboard.dev does not support displaying the embeddings for some reason, and you have to do that on a local tensorboard. (That's why I'm asking for a screenshot instead of a link to tensorboard.dev in the submission.)

clayton-north commented 4 years ago

I'm having a similar problem with my local tensorboard

image

I have tried removing and recreating my logs directory and am still having an issue

ademuro20 commented 4 years ago

I had an issue similar but my terminal raised an error. I added these lines of code and mine worked, doesn't hurt to try?

    import tensorflow as tf
    import tensorboard as tb
    tf.io.gfile = tb.compat.tensorflow_stub.io.gfile
mikeizbicki commented 4 years ago

@ademuro20 The error you are seeing is due to incompatible versions and I don't think it's related to @clayton-north 's error. See #40.

@clayton-north There is a dropdown menu in the top right of your screen. Inside that menu should be the embeddings option.

clayton-north commented 4 years ago

I'm not seeing it in the dropdown menu, i also was not able to get tensorboard working for part 2 either.

image

image

sarahbashir commented 4 years ago

It should be under the 'projector' tab!

clayton-north commented 4 years ago

Oh good catch! I didn't know that. I'm still having a little trouble

image

mikeizbicki commented 4 years ago

The No checkpoint was found message means that tensorboard is working correctly, but there is no data for it to load.

In the original command you gave, you had both the --embed and the --train options passed in at the same time. If you are quitting your program early, then you will never reach the embedding code. Are you 100% sure that your embed code is actually being run?

clayton-north commented 4 years ago

Thanks for the help! my embed code was never being run.