Closed sidharthrajaram closed 1 month ago
For anyone encountering the same issue, it probably varies quite a bit depending on your dataset, but I had to implement custom Task
and FairseqDataset
classes and then override setup_task()
and load_dataset()
behavior in my custom Task
class.
❓ Questions and Help
Before asking:
What is your question?
I have extended FairseqDataset and created a custom dataset implementation. How do I use this custom implementation to train against some particular model architecture using
fairseq-train
?What have you tried?
Read the docs regarding Tasks and have a feeling that's the direction to go. Still curious about the extent to which the CLI tool
fairseq-train
can be used.What's your environment?
main
2.2.2
Ubuntu 20.04
pip
, source):source
3.10
CUDA 11.8