danny-avila / LibreChat

Enhanced ChatGPT Clone: Features Anthropic, AWS, OpenAI, Assistants API, Azure, Groq, o1, GPT-4o, Mistral, OpenRouter, Vertex AI, Gemini, Artifacts, AI model switching, message search, langchain, DALL-E-3, ChatGPT Plugins, OpenAI Functions, Secure Multi-User System, Presets, completely open-source for self-hosting. Actively in public development.
https://librechat.ai/
MIT License
19.28k stars 3.22k forks source link

Enhancement: Import/export from ChatGPT + backup (and restore) all conversations for LibreChat #1660

Open darkBuddha opened 10 months ago

darkBuddha commented 10 months ago

What features would you like to see added?

I guess most people that start to use LibreChat used ChatGPT before. A fair amount of them will switch fully to LibreChat and not use ChatGPT again.

As ChatGPT allows to export all chats into a nice JSON, maybe it would be a cool feature to be able to import these timestamped JSON messages into LibreChat, to not lose the history?

Also, the ChatGPT chats are not private and used for training. So there is an incentive to delete the Chats from ChatGPT.

More details

I think this would decrease the friction of leaving ChatGPT for LibreChat and thus increase users.

Which components are impacted by your request?

No response

Pictures

No response

Code of Conduct

danny-avila commented 6 months ago

The import portion of this is now merged!

darkBuddha commented 6 months ago

Bless you ๐Ÿ™๐Ÿป

aswny commented 6 months ago

Thank you so much. ๐Ÿ™๐Ÿป

Are you also working on adding an option to export all conversations from LibreChat?

Currently I use LibreChat on a work machine but I eventually want to export all conversations to personal device.

darkBuddha commented 6 months ago

being able to open single conversations and share them by link would also be nice... no exports needed then.

aswny commented 6 months ago

@darkBuddha single conversations can still be shared, right? In JSON/Text formats at least. I'm looking for a bulk export to transfer all chats from one machine to another.

darkBuddha commented 6 months ago

this is too much work, it would be easier/better to be able to share a link to a conversation (like in ChatGPT)

darkBuddha commented 6 months ago

HI @danny-avila, I just tested it, it works!! 15 MB ChatGPT export from a few months ago, first try. Amazing, thank you! Meilisearch also very fast, good choice!๐Ÿ‘๐Ÿป

darkBuddha commented 6 months ago

@danny-avila on first try, nothing happened, so i tried again with console open. then it worked and i saw the "conversation import successful". now i saw, that the messages were imported twice. do you have an idea, how i can get rid of this, e.g. in mongo express?

danny-avila commented 6 months ago

@danny-avila on first try, nothing happened, so i tried again with console open. then it worked and i saw the "conversation import successful". now i saw, that the messages were imported twice. do you have an idea, how i can get rid of this, e.g. in mongo express?

So it had started already the first time, thatโ€™s why it got imported twice. Maybe an info toast saying it was importing couldโ€™ve helped prevent this

It would be a bit difficult to โ€œundoโ€ but will keep that in mind as an improvement

sam1am commented 6 months ago

Is there any kinda hacky way to export conversations in the meantime? Key feature for me is open access to my own gathered data and conversations. Thanks.

danny-avila commented 6 months ago

Is there any kinda hacky way to export conversations in the meantime? Key feature for me is open access to my own gathered data and conversations. Thanks.

You can use mongodump for this now

OverLordGoldDragon commented 5 months ago

way to export conversations in the meantime

My approach, based on MongoDB Compass:

1) LibreChat > conversations > Aggregations

2) + CREATE NEW, check </> TEXT, copy/paste below snippet

expand ```shell [ { $lookup: { from: "messages", localField: "messages", foreignField: "_id", as: "messageDetails" } }, { $project: { _id: 1, conversationId: 1, user: 1, createdAt: 1, updatedAt: 1, title: 1, messageDetails: 1 } } ] ``` ![image](https://github.com/danny-avila/LibreChat/assets/16495490/064244c0-9a65-47e5-a1c6-7e0789a6dfd1)

3) Run + Export

This yields a .json. It has lots of irrelevant info. I converted it to .md in Python as follows (made by GPT-4o + Claude 3.5 Sonnet):

expand ```python # -*- coding: utf-8 -*- import json from datetime import datetime # Load JSON data from a file path = "LibreChat.conversations.json" with open(path, 'r', encoding='utf-8') as file: data = json.load(file) def convert_date(date_str): return datetime.strptime( date_str, "%Y-%m-%dT%H:%M:%S.%fZ").strftime("%Y-%m-%d %H:%M:%S") def json_to_markdown(conversation): markdown = f"# Conversation: {conversation['title']}\n\n" markdown += f"Conversation ID: {conversation['conversationId']}\n" markdown += f"User ID: {conversation['user']}\n" markdown += f"Created At: {convert_date(conversation['createdAt']['$date'])}\n" markdown += f"Updated At: {convert_date(conversation['updatedAt']['$date'])}\n" markdown += "\n## Messages\n" for msg in conversation['messageDetails']: markdown += f"### Message from {msg['sender']} at {convert_date(msg['createdAt']['$date'])}\n" markdown += f"{msg['text']}\n\n" return markdown # Process all conversations all_markdown = "" for conversation in data: all_markdown += json_to_markdown(conversation) + "\n\n---\n\n" # Print the first few lines of the output print(all_markdown[:500]) # Write all conversations to a single markdown file with open("all_conversations.md", "w", encoding='utf-8') as file: file.write(all_markdown) print("Conversion complete. Check 'all_conversations.md' for the result.") ```

This can be viewed in whichever markdown viewer, including Github comments. I've not tested whether the .json can be imported back into LibreChat.

cendenta commented 4 months ago

I've found ChatGPT Exporter the best exporter of any of the various systems I've tried, with some nice customizability. Since LibreChat is very similar to ChatGPT, I had (mild) hopes that the TamperMonkey script would work just by adding the LibreChat URLs into the config (no dice, unfortunately). Thought it was worth mentioning for considering potential compatibility, or at least inspiration for LibreChat export.

mayurjobanputra commented 1 week ago

Thank you so much. ๐Ÿ™๐Ÿป

Are you also working on adding an option to export all conversations from LibreChat?

Currently I use LibreChat on a work machine but I eventually want to export all conversations to personal device.

I have this issue as well. Started using it on Railway but not run locally now.