Breakend / experiment-impact-tracker

MIT License
273 stars 31 forks source link

Pandas whitespace error fix #14

Open amorisot opened 4 years ago

amorisot commented 4 years ago

Before training starts I get this error: pandas.errors.ParserError: Expected 8 fields in line 7, saw 9. Error could possibly be due to quotes being ignored when a multi-char delimiter is used.

It's due to the space in chrome 00type=g line below, which is what you obtain after you process the output of nvidia-smi in the get_nvidia_gpu_power function.

    gpu        pid  type    sm   mem   enc   dec   command
    0       1351     G     0     0     0     0   Xorg
    0       1441     G     0     0     0     0   gnome0shell
    0       1973     G     0     0     0     0   Xorg
    0       2090     G     0     0     0     0   gnome0shell
    0      83636     C     0     0     0     0   python
    0     105873     G     0     0     0     0   chrome 00type=g
    0       1807     C     4     1     0     0   python3
    0       2090     G     4     0     0     0   gnome0shell
    0       1807     C     0     0     0     0   python3
    0       2090     G     6     0     0     0   gnome0shell
    0       2090     G     2     0     0     0   gnome0shell

When running that table through pd.read_csv(StringIO(out_str_final), delim_whitespace=True), delim_whitespace turns chrome 00type=g into 2 columns.

To fix, before doing out_str_final = out_str_final.replace("-", "0"), do out_str_final = out_str_final.replace(" --", "").

cifkao commented 3 years ago

Fixes #36.