search

tianyi-lab / Cherry_LLM

[NAACL'24] Self-data filtering of LLM instruction-tuning data using a novel perplexity-based difficulty score, without using any other models
306 stars 21 forks source link

How to transform a json file into fastchat format? #23

Closed daidaiershidi closed 1 month ago

daidaiershidi commented 1 month ago

Thank for your great work!

The current format is:

[
    {
        "instruction": "This is instruction",
        "input": "This is input",
        "output": "This is output"
    },...
]

The format of fastchat is:

[
  {
    "id": "identity_0",
    "conversations": [
      {
        "from": "human",
        "value": "Who are you?"
      },
      {
        "from": "gpt",
        "value": "I am Vicuna, a language model trained by researchers from Large Model Systems Organization (LMSYS)."
      },...
    ]
  },...
]

So, how to transform a json file into fastchat format?

daidaiershidi commented 1 month ago

I understand.