Combine latest changes of the Q&A dataset and Stackoverflow dataset to the Merged Q&A dataset - Githubissues

amosproj / amos2024ss08-cloud-native-llm

MIT License

6 stars 1 forks source link

Combine latest changes of the Q&A dataset and Stackoverflow dataset to the Merged Q&A dataset #101

Open dominic0df opened 3 days ago

dominic0df commented 3 days ago

User story

As a data engineer
I want to combine all our Q&A data sources into one big data source
So that it can directly used as a whole for trianing

Acceptance criteria

The latest data of the Stackoverflow dataset is merged into the Merged dataset on Huggingface
The latest data of the Q&A dataset is merged into the Merged dataset on Huggingface

Definition of done (DoD)

Bill of Materials in the planning document has been updated
All feature branches have been merged and closed
New feature code has been documented
Potential new licenses have been checked
All GitHub Actions are passing
The requirement.txt is updated

DoD general criteria

Feature has been fully implemented
Feature has been merged into the mainline
All acceptance criteria were met
Product owner approved features
All tests are passing
Developers agreed to release