stats.log file is empty, yet weighs in at 500+ meg

[OniDaito]

Hello there. I'm trying to load a log file, saved with tensorwatch. I have a machine at the end of the internet, logging to a file. When a run is completed I upload the log file to a server, where a jupyter notebook is running. My hope is to open that log file and do all my work there where it can be shared. Eventually, I'd like to stream my results but that requires a lot more work with tunnelling and security. For now, this method would be fine.

On the AI box:

class Stats(object):
    """ We used a stats object as we have streams and all that sort of
     thing."""

    def __init__(self):
        # create a stream for logging
        self.watching = {}
        self.observing = {}

    def on(self, savedir, port):
        self.watcher = tw.Watcher(filename=savedir + "/stats.log", port=port)
        # self.watcher.make_notebook(savedir + "/stats.ipynb")

    def watch(self, obj, name):
        """ Add something to be watched via tensorwatch. """
        if name not in self.watching.keys():
            self.watching[name] = {
                "object": obj,
                "stream": self.watcher.create_stream(name=name)}

    def observe(self, obj, name):
        """ Add something to be observed with tensorwatch. This is apparently
        low cost and lazy and I suspect only happens when we actually look
        at things live. Could be very handy though. """
        if name not in self.observing.keys():
            kw = {name: obj}
            self.observing[name] = obj
            self.watcher.observe(**kw)

    def update(self, idx):
        """ Update all our streams with the current idx value. """
        for name in self.watching.keys():
            dd = self.watching[name]
            dd['stream'].write((idx, dd['object']))

I use watch to add objects such as tensors with a label. Update is called when I start writing to the stream

On the jupyter notebook side I have this:

import tensorwatch as tw
import pickle
import sys
import torch

client = tw.WatcherClient(port=None, filename="stats.log")
in_e = client.open_stream("in_e")
out_e = client.open_stream("out_e")

text_vis = tw.Visualizer(out_e, vis_type='text')

The text_vis loads (with a bit of extra coddling to convert from GPU to CPU tensors) but it's empty! 0 entries.

Is there something I'm doing wrong here? Ideally, I'd like to see a tutorial on loading from a file or interrogating these files so I can manipulate the data stored in there.

I'm using Python 3.7 with pytorch 1.5.0

Cheers Ben