Identify Repositories for the Kubernetes LLM dataset - Githubissues

amosproj / amos2024ss08-cloud-native-llm

MIT License

7 stars 1 forks source link

Identify Repositories for the Kubernetes LLM dataset #11

Closed dominic0df closed 4 months ago

dominic0df commented 5 months ago

User story

As Data Engineer
I want to identify repositories for my dataset
so that the data could be automatically extracted to a dataset

Acceptance criteria

[x] The data of CNCF repositories is identified
[ ] Therefore Key URLs are able to be manuelly configured (f.e. in a yml-File)
[ ] Outgoing of key URLs, relevant repositories are automatically identified
- [ ] documentation sites (.pdf, .md)-Files are identified
- [ ] landscape (.yml, .yaml)-Files are identified
[ ] The result of the search (URLs) is stored, so that in a previous step data could be automatically retrieved from the webpages

DoD general criteria

Feature has been fully implemented
Feature has been merged into the mainline
All acceptance criteria were met
Product owner approved features
All tests are passing
Developers agreed to release

julioc-p commented 4 months ago

Added script to make file with the URLs requested. Did not upload to Github due to size of the file (50MB)

julioc-p commented 4 months ago

File uploaded to Hugging Face. Download Link

dominic0df commented 4 months ago

actual size: 5