Training Dataset APIs - Githubissues

BharatSahAIyak / autotune

A comprehensive toolkit for seamless data generation and fine-tuning of NLP models, all conveniently packed into a single block.

MIT License

9 stars 5 forks source link

Training Dataset APIs #114

Open sooraj1002 opened 4 months ago

sooraj1002 commented 4 months ago

in addition to updating the HF dataset when training, there should be a dedicated API which updates the dataset to the HF repo without triggering the training

KDwevedi commented 4 months ago

@sooraj1002 Let's simply manage training sets as their own entities and have proper APIs for interactions with training sets.

/training-dataset APIs
- PATCH: Updating Specific Rows
- PUT: Updating the Whole Dataset (would be useful for correcting divergent datasets as well)
- POST: Add a new training Dataset
- GET: GET a training dataset by ID, Name, or TaskID
training-dataset/sync syncs the current cached dataset to HF.

Checks:

[ ] Add configuration to training dataset definitions for data validation, i.e. allow user to specify acceptable values for columns
- [ ] Allow user to specify acceptable values like an enum (coming from the context of query classifier.)