Other sync options

[firebone-be]

Is your feature request related to a problem? Please describe. I really love how MicroPad works but I find the sync option limited. I have several (paying) online services and would like it if I can use one of those to sync notebooks.

Describe the solution you'd like Options to sync via (onedrive, google drive, owncloud, nextcloud, webdav, ...)

Describe alternatives you've considered Exporting but that would mean I would be constantly be exporting and importing. Also tried other markdown apps like Joplin etc but I find MicroPad more to my liking except for the sync options.

Additional context Add any other context or screenshots about the feature request here.

[NickGeek]

Hi, I'm glad to hear that you like how MicroPad works. The export/import can be made faster through the use of keyboard shortcuts like Ctrl+S and Ctrl+O.

What features do you find limited about MicroSync?

[firebone-be]

Hey @NickGeek

I find it limiting that I would have to pay (al be it a small amount) when I already have several cloud storage facilities at my disposal (some paying). I would like to be able to use onedrive of google drive or whatever I want to sync my notepads :)

[NickGeek]

That's completely fair, although it is free for a lot of notebooks. I actually used OneDrive as my syncing system when I was first making MicroPad and didn't have any sync built in.

I'm not sure what the best way to go about adding competing syncing services would be. MicroSync doesn't actually store the notebook as a normal NPX file, it stores the structure, text, and metadata as a JSON object and splits out assets into their own files, which is why the syncing is so much faster (only the assets that change get synced).

I think for services like OneDrive you'd need to sync the full NPX file. I'm open to a PR to support that although you'd need to be careful because doing a notebook export is a fairly expensive operation so I'd probably wrap it in a web worker to avoid impacting the UI's performance.

[NickGeek]

Alternatively I'm happy to give you 6 months of free MicroSync Pro if you need it. You've had some good feature requests and I appreciate it.

[martinkoutecky]

@NickGeek hi -- I'm in a similar boat as @firebone-be, except wanting to use Syncthing rather than some payed synchronization service (like Google, OneDrive etc.). If MicroPad wants to be as open as possible, surely it can expose the directory it synchronizes using MicroSync, and then I could sync it using syncthing easily? I still lose benefits of centralized sync (e.g. no sync if a devices with changes turns off before another device turns on) but that's on me.

I would like to use micropad as a research notebook where I would store PDFs of papers in it and also drawings, and I don't want to pay and don't want to make my coauthors pay - that's why I'd like to avoid MicroSync, even though I think it's cool that you provide it.

So maybe my question is - what am I missing, why isn't it easy to just expose some directory with data to be synced using any other service?

[NickGeek]

@martinkoutecky The issue is that MicroPad notepads aren't actually stored as NPX files in a directory. They're stored in an IndexedDB database on your computer in a light-weight flattened JSON format.

When you save a notepad with Ctrl+S, or Ctrl+Shift+S (for export all) it generates all the NPX files from the light-weight JSON format in IndexedDB.

I've looked into exposing a directory before but the performance was terrible due to the constant back-and-forth between NPX and the runtime format. MicroSync doesn't just use some hidden directory of NPX files, it stores another custom JSON format that the client can cheaply convert between with assets stores separately with UUID references. The way it works enables a differential sync where only the binary assets that have changed have to get uploaded/downloaded.

MicroSync is only able to work because it was specifically designed to work with MicroPad (and MicroPad with it).

I still completely support you wanting to use another service, and you can trivially save all of your notepads to a folder with Ctrl+Shift+S (or Cmd+Shift+S on Mac) and then open (Ctrl+O/Cmd+O) that ZIP in MicroPad to do so. This is how I used to use MicroPad before MicroSync existed and it worked fairly well.

If you have any ideas on how it could be made easier to share notebooks outside of MicroSync I'd be keen to hear them but I don't see any easy way to just expose a directory because MicroPad doesn't actually ever have some directory filled with notebooks.

[martinkoutecky]

@NickGeek thanks for the great response! I gathered so much from your initial response. Probably there is something obvious that eludes me -- why couldn't I sync the internal custom JSON format with the asset stores etc.? Basically I thought about just syncing the .micropad directory but probably this would break because of something in electron (two instances of the app running off of one .micropad dir). Is it difficult to expose the data in the custom format?

Or even what would help me is some autosave option - say that I'm OK with MicroPad autosaving to NBX every 5 minutes. I have been warned. Would that be possible?

Thanks for this interesting tool!

[NickGeek]

@martinkoutecky Because you'd need to know which assets to sync each time. Syncing the directory itself might work but yeah I can't guarantee it does because it's not meant to be.

Autosave is doable although MicroPad would need some form of setting panel added to it to support that alongside integration with native OS FS APIs. It's doable but a non-small chunk of work. If someone wants to pick it up and work on it I can help, but it's not just something that could be done in an hour or so.

I hope you still find the tool useful. Ctrl+S and Ctrl+O to a shared directory is still a surprisingly effective strategy that's worked for a long time and I think a lot of people overlook it when that's the whole reason MicroPad supports it.

[martinkoutecky]

@NickGeek bottom-line for me is, maybe I'll give Ctrl-S, Ctrl-O a try.

I'm still wondering - when you say "you'd need to know which assets to sync each time" - are you saying that there are assets whose timestamp or hash changes BUT they don't need to be synced? Otherwise I'd say that any half-decent sync app (like syncthing) only ever synchronizes changes. So basically if the internal format only changes when I change something in micropad (this seems obvious?) AND if there is some proportionality of change (e.g. if I fix a typo, only few files change and only a little), then everything should work without passing any domain-specific info to the sync app.

I won't be able to dedicate any work to this unfortunately - when it comes to this i'm just a user - but maybe a further clarification regarding what I wrote about may be helpful to someone else some time in the future. Anyway, thanks for your work.

[NickGeek]

@martinkoutecky The issue is that MicroPad would still need to write the asset files to disk on every sync to let your sync service determine whether or not to sync it. With MicroSync there's a hand-shake like process where the client and the server negotiate the missing assets to figure it out before anything needs to get loaded into memory/written to disk (as opposed to before something gets sent over a network).

[martinkoutecky]

@NickGeek I see, OK, makes sense, thanks for the explanation.