Closed Ademord closed 1 year ago
So i managed to get a script that merges the manually downloaded CSVs into a single csv, for anyone who would need something like this:
import pandas as pd
import glob
from run_name_getter import *
import matplotlib.pyplot as plt
files = glob.glob("movingForward/*.csv")
if len(files) == 0: raise BaseException("no files")
step_column = pd.read_csv(files[0], usecols=['Step'])
all_csvs = [pd.read_csv(p, usecols=['Value']) for p in files]
for i in range(len(all_csvs)):
csv = all_csvs[i]
run_name = get_run_name(files[i])
csv.columns = [run_name]
metric_name = get_metric_name(files[0])
all_csvs = [step_column] + all_csvs
df = pd.concat(all_csvs, axis=1)
df.plot(x="Step", alpha=0.5, title=metric_name)
plt.show()
the only downside now is I have to manually download each CSV for each run for each metric.
I'm no longer actively working on this tool and am not sure if this issue is still relevant. Going to close it for now. We can reopen if it's still an issue for you.
I am trying to use this aggregator, but i am getting the error
my dir structure looks like this
I am trying to aggregate all my runs (which have many variables) into a single csv so I don't have to download each one manually and then aggregate it manually... picture example attached: