pytorch / torchtune

PyTorch native finetuning library
https://pytorch.org/torchtune/main/
BSD 3-Clause "New" or "Revised" License
4.3k stars 430 forks source link

Include converter from OpenAI message format #1373

Closed joecummings closed 1 month ago

RdoubleA commented 2 months ago

This is now JSONToMessages, or are you referring to a different format?

joecummings commented 2 months ago

This is the OpenAI reference. We should match this exactly otherwise people cannot use it OOTB.

    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "What'\''s in this image?"
          },
          {
            "type": "image_url",
            "image_url": {
              "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"
            }
          }
        ]
      }