Open bhavul opened 9 months ago
Hi @bhavul , I would like to work on this issue. Anything i should know before I start working?
hi @bhavul is this issue fixed?
@Pritam-hakingmaster this is a good issue to get started with - the basic concatenation is implemented using hugginface datasets (see line 29 in gato/tasks/text_task.py)
The individual metrics per dataset is an open challenge that no one has taken on yet, worth doing
I will try my best.
For text task, when we would have multiple datasets, concatenation strategy could be moved to a more sophisticated logic by using huggingface concatenation.
Further, we may wish to change the evaluation loop to also give metrics individual to each dataset besides the average.
_Originally posted by @daniellawson9999 in https://github.com/ManifoldRG/NEKO/pull/1#discussion_r1299509872_