"exception loading monitor file in PATH: 't'"

[maciejjaskowski]

When using Monitor I am sometimes getting results x.y.monitor.csv:

# {"t_start": 1545757370.154182, "env_id": "PongNoFrameskip-v4"} 
r,l,t-21.0,792,63.464505
-21.0,819,96.574202
(...)

Note the missing endline character after r,l,t

E.g. when using monitor.load_results some 40 monitor files are correctly read before a failure on this one: /home/user/logs/pong-v4-6/0.16.monitor.csv

[maciejjaskowski]

I didn't invest into understanding why this happens but this simple script works around this issue by rewriting csv files so that the r,l,t-21.0,792,63.464505 beomes

r,l,t
-21.0,792,63.464505

def fix_csv(fname):
    with open(fname, 'rt') as fh:
                first_sharp_line = fh.readline()
                if not first_sharp_line:
                    return
                assert first_sharp_line[0] == '#'
                col_names = fh.readline()

                if col_names != 'r,l,t':
                    first_csv_line = col_names[len('r,l,t'):]
                    all_lines = [first_sharp_line, 'r,l,t\n', first_csv_line] + fh.readlines()
                    with open(fname, 'wt') as fh:
                                fh.write("".join(all_lines))

dir = YOUR_DIR
monitor_files = glob(osp.join(dir, "*monitor.csv"))
if not monitor_files:
    raise LoadMonitorResultsError("no monitor files of the form *%s found in %s" % (Monitor.EXT, dir))

for fname in monitor_files:
    fix_csv(fname)

[pzhokhov]

missing characters in the Monitor file are usually a sign of a race condition of some sort (i.e. there is actually more than one env wrapped with a monitor that tries to write to the same file). Could you provide more information on when does it occur?

[rasoolfa]

It happens to me too. Sometimes endline character after r,l,t is not there or r,l,t are not there as well. It doesn't happen all the times, so you guess about race condition might be right.