how to finetune with 'system information'

[mhyeonsoo]

Hello,

I am training with my custom dataset, and have a question there. What I wanted to make is assistance that can recommend me a proper mode of device depending on my conversation.

Before inserting q/a pairs, I want to let model know about the general information of 'how to use' the device. I tried to insert like below.

SYSTEM:
    There are 4 options in the mode
    - mode1
    - mode2
    - mode3
    - mode4

   you need to generate 'json' format using USER input with the proper mode.
   Desired output format is below.
   {
        'mode': [selection of mode]
        'comments': [your response]
    }

USER: example1
ASSISTANCE: response1

USER: example2
ASSISTANCE: response2

USER: example3
ASSISTANCE: response3

But it seems like the model doesn't know about the initial information about the device.

Is there any specific format like 'USER' and 'ASSITANCE' for teaching the information as well?

Thanks,

[lxe]

Try more samples (like over 300) and more epochs. Set the learning rate higher to slightly overfit it.