Can I use lora or adapter to fine-tune some non-instruction set data?

[hello-eternity]

Can I use lora or adapter to fine-tune some non-instruction set data? What script should I use to process the data and run fine-tuning?

[hello-eternity]

Just as RedPajama dataset, but I want use lora to finetune

[rasbt]

In theory, I think this should work, but in practice, it would require slightly different data preparation and handling. That's because data prep scripts like dolly.py prepare a batch of examples with the respective fields (see the generate_prompt function at the bottom). And scripts like finetune/lora.py use a get_batch function to return a batch consisting of training examples vs blocks for raw text as the data loaders in pretrain/redpajama.py.

Curious what others think (CC @lantiga @carmocca ) but imho we should probably think about a restructure so that instead of

finetune/
    full.py
    lora.py
...

we have

finetune/
   supervised/
       full.py
       lora.py
       ...
   unlabeled/
       lora.py
       adapter.py
       ...
   rlhf/
       rejection_sampling.py
       ppo.py
       ...
...

[carmocca]

I wouldn't think about restructuring before we need it, if the scripts are messy.