Importing data from Chatbot-UI to Smart-Chatbot-UI

[johnbrownlow]

I have a bunch of conversations on Chatbot-UI that I would like to migrate to Smart-Chatbot-UI. However, the json doesn't want to import. I've compared the formats and tried manually editing them but again I couldn't seem to get it to work. Is there a straightforward way of achieving this? I probably only need to do it once.

[johan456789]

Same here. What have you tried? I tried adding "temperature": 0.5, to all items in history and "folderId": null to all items in prompts but there are still errors.

EDIT:

I have written a script fix-json.py to fix the format and figured out how to import them (at least for my use case). Run the script: python fix-json.py <your_chat_export_json>:

import json
import sys

filename = sys.argv[1]
with open(filename) as f:
    data = json.load(f)

data['history'] = data['history'][:100]  # TODO: change this to get import 100 chats at a time
print(f'There are total {len(data["history"])} chats.')
for item in data['history']:
    item['temperature'] = 0.5

    # Iterate through the 'history' and fix the 'model' property
    if item['model']['name'] == "GPT-4":
        item['model']['maxLength'] = 24000
        item['model']['tokenLimit'] = 8000
    elif item['model']['name'] == "GPT-3.5":
        item['model']['maxLength'] = 12000
        item['model']['tokenLimit'] = 4000

# data['prompts'] = []
print(f'There are total {len(data["prompts"])} prompts.')
for prompt in data['prompts']:
    prompt['folderId'] = None

with open('fixed.json', 'w') as f:
    json.dump(data, f, indent=2)

It seems it only supports importing a little more than 100 chats at a time. So I import 100 chats at a time. Change the line I commented with TODO to do that. Import the fixed.json until all chats are imported and then I think you're good to go!

[johan456789]

To fix this from the source, I believe we should:

1. set a default value in the import code when properties are missing.
2. fix the 100 or so chat import limit. I am not sure but I suspect it might be related to this. The error shown is:

[johan456789]

Somehow after the import, this error keeps popping up whenever the chatbot finishes an answer. Kinda annoying.

I'll probably use https://github.com/Shopify/chatbot-ui for now, until this TRPCClientError issue is solved.