[WIP] Refactor: do not save as folder tree

jtpio / trello-full-backup

Python script to backup everything from Trello: boards, lists, cards and attachments

MIT License

123 stars 36 forks source link

[WIP] Refactor: do not save as folder tree #17

Open jtpio opened 6 years ago

jtpio commented 6 years ago

This is the beginning of a major refactor to the script.

The idea behind that change is to make the backup process as simple as possible and as error-prone as possible. The long term goal is to be able to run such script on hosted provider such as AWS Lambda, making use of the recurring scheduling features.

Instead of saving to disk by name (card name, list name or board name), the entire board is saved at once (containing all the information).

Steps:

[x] Download and save attachments
[ ] Add back incremental backup (backing up to the save folder does not need to override existing content)

OliPlus commented 6 years ago

Is there a possibility to have both? I really like the folder tree approach for being human readable even in case Trello dies tomorrow. :) A single file is good for an automated backup process but I think will lose the human readbility.

jtpio commented 6 years ago

Having it human readable was the original motivation for the tree structure :+1:

The tree structure comes with a few drawbacks however:

many files are created during the process (one per card at least)
potentially very long card names
problems with locales and special characters

Maybe we could consider building a lightweight web UI using the full raw json files as a separate component to isolate the two concerns? (backing up the data and visualizing it)

OliPlus commented 6 years ago

But if we build a lightweight web UI you loose the advantage to run the export on the desktop and be able to use it w/o any additional software (web UI). For the server integration the web UI is a great idea in my eyes.

OliPlus commented 6 years ago

OK, I thought about it a little bit on my way home. Would it be possible to have two modes?

Desktop Mode which creates a folder structure as it does today? Make sit easy to run the script and have all Trello info offline available.
Server Mode which creates a single file. For this we'd need to make some kind of viewer application which could possibly by a single page JS app? JS is pretty good in handling the JSON.

jtpio commented 6 years ago

We could definitely have two modes:

The first layer would output full raw JSON and attachments (if requested)
The second would be built on top of the first layer and provide the tree structure

Or only one mode to dump the raw JSON files, and use a tool like wekan (open source Trello-like kanban) to visualize the data in a more natural way than browsing folders and files on disk. I quickly tried wekan and:

it's pretty simple to get running: docker compose up when using https://github.com/wekan/wekan-mongodb
it has support for importing Trello board from JSON, recreating the UI (with labels, attachments and so on) automatically!

OliPlus commented 6 years ago

Sorry for my late response, got busy here.

I like the idea with wekan, but keep in mind there is users in this world who do not play with Docker, etc.. So still having a mode which writes the tree structure as of today has a value in my eyes. It's basically a one script deployment and nothing else is needed.