Automatic upload traces to hugging-face

[recursix]

Make tools to simplify adding traces of agents to an ever growing huggingfaces dataset.

• create 2 datasets on hugging face

  • one that would be an index to be able to easily retrieve traces based on attributes similar to the dataframe when we run load_result_df
  • one that contains actual zipped traces that can be retrieved from a pointer in the index

• make code to upload a study trace by trace and easy way to group the traces by study in the index.

• legality:

  • limit adding only from the domains that are whitlisted (e.g. our benchmarks or a subset of them)
  • based on which LLM and which benchmarks attribute a specific license to it.

[recursix]

we can leverage the exp_args.exp_id (a uuid) as a unique reference for each trace