speakleash / speakleash-instruct-creator

Generate instructions datasets for the fine-tuning purposes.
3 stars 5 forks source link

New repo structure #54

Closed IgorTest19 closed 1 month ago

IgorTest19 commented 4 months ago

Due to the creation of new training/tuning datasets, repostitory requires a new, clear and readable structure.

Proposed new directory structure:

speakleash_tuning_sets/
├───data_generators
│   ├───conversations
│   │   ├───automated
│   │   └───manual
│   ├───functions
│   │   ├───automated
│   │   └───manual
│   └───instructions
│       ├───automated
│       └───manual
├───data_samples
│   ├───conversations
│   ├───functions
│   └───instructions
├───utils/
├───merge_files.py
├───generate_samples.py
├───requirements.txt
⋮
pawelkiszczak commented 4 months ago

Directory structure seems well thought out and should work the way it's intended to. I'd approve it to be implemented.

IgorTest19 commented 4 months ago

After consultation, a differebt structure was proposed to facilitate the addition of new types as subdirectories.

speakleash_tuning_sets/
├───instructions
│   ├───samples
│   ├───automated
│   └───manual 
├───conversations
│   ├───samples
│   ├───automated
│   └───manual
├───functions
│   ├───samples
│   ├───automated
│   └───manual
├───utils/
├───merge_files.py
├───generate_samples.py
├───requirements.txt
pawelkiszczak commented 4 months ago

Approved