Data prep for LLM application builders

mlabonne / llm-course

Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks.

https://mlabonne.github.io/blog/

Apache License 2.0

36.46k stars 3.82k forks source link

Data prep for LLM application builders #72

Open Bytes-Explorer opened 3 months ago

Bytes-Explorer commented 3 months ago

Hello! This is really cool collection.

I want to propose adding https://github.com/IBM/data-prep-kit as tools for quick data preparation as it may help LLM app developers started quickly. Our team developed these tools for our internal usage, but we believe these are widely applicable and have open sourced them.

I can raise a PR, if we think this is meaningful.

mlabonne commented 3 months ago

Hi, thanks for your suggestion, this is super relevant. It would be even better to showcase it in the LLM datasets (https://github.com/mlabonne/llm-datasets) repo I made with other tools. I'd be happy to accept a PR there if you're interested