oxly uses the Dropbox API to auto-merge Dropbox file revisions with a git-like cli.
Though much of this README is geared toward Orgzly/Emacs usage many of the features can be used for general Dropbox revision groking.
You can edit and save the same file simultaneously on two Dropbox clients -- usually Emacs/laptop and Orgzly/mobile -- and then later run oxly on laptop to view/diff/merge/push revisions.
oxly is most useful when you try to Orgzly Sync and it fails with the following error message
Both local and remote notebook have been modified.The oxly merge cmd uses diff3(1) and will try to auto-merge. If it can't auto-merge all hunks the conflicts can be resolved by hand with the Emacs ediff-merge-with-ancestor cmd (nice UI) or $EDITOR diff3-output (not so nice UI).
My use case is two Dropbox clients (Emacs/Unix, Ogzly/Android) so more/other clients not tested but maybe can be done carefully and two at a time. Also see Caveats/Gotchas below.
Used dailyish by the developer (w/2 Dropbox clients, Emacs laptop and Orgzly mobile) but that's total usage asfaik -- more users are welcome -- any bug/issue/suggestion/question post it at https://github.com/gaak99/oxly/issues).
You probably want to try master HEAD before fetching a release.
oxly does no Deletes via Dropbox API and all edits/merges are saved as a new revision, so should be low risk to give it a try. And note if a mismerge is saved you can easily revert to the revision you want, see Caveats/Gotchas below.
It's been over a year since any big changes so seems pretty solid at least for my usage -- when I'm in note-taking-mode I use it almost daily and works good. My only complaint is diff3 seems to not auto-merge as frequently as I hoped. But I've gotten good at emacs ediff to resolve conflicts so not a big problem.
Every time you edit/save or copy over an existing file (citation needed) a new revision is quietly made by Dropbox. And Dropbox will save them for 1 month (free) or 1 year (paid). And as a long time casual Dropbox user this was news to me recently.
And my fave org-mode mobile app Orgzly supports Dropbox but not git(1) (yet) so I needed a way to merge notes that are modified on both laptop and mobile.
And if you squint hard enough Dropbox's auto-versioning looks like lightweight commits and maybe we can simulate a (limited) DVCS here enough to be useful.
On Dropbox we keep a small&simple filename=content_hash kv db called the ancdb. The content_hash is the official Dropbox one.
oxly clone/merge/push will (pseudocode):
# fpath is file path being merged
fa = dropbox_download(revs[latest]) # latest from Orgzly
fb = dropbox_download(revs[latest_rev-1]) # latest from Emacs
fanc = dropbox_download(ancdb_get(fpath))
rt = diff3 -m fa fanc fb #> fout
if rt == 0: # no conflicts
pass
elseif rt == 1:
# hand edit fout or 3-way ediff
dropbox_upload(fout)
ancdb_set(fpath); dropbox_upload(ancdb)On Orgzly (when regular Sync fails) select Force Save.
On laptop run oxmerge (wrapper around oxly). If auto-merge aka diff3(1) does not resolve all conflicts, resolve them by hand.
On Orgzly run Sync.
$ oxly --help
$ oxly sub-cmd --helpgit clone https://github.com/gaak99/oxly.git
export SUDO=sudo # set for your env
cd oxly && $SUDO python setup.py install
export MYBIN=/usr/local/bin # set for your env
$SUDO cp oxly/scripts/oxmerge.sh $MYBIN/oxmerge
$SUDO chmod 755 $MYBIN/oxmergeCreate a Dropbox API app (w/full access to files and types) from Dropbox app console
<https://www.dropbox.com/developers/apps>
and generate an access token for yourself.
And add it to ~/.oxlyconfig. Note no quotes needed around $token.
[misc]
auth_token=$tokenMake sure Orgzly has a clean Sync of file.
Run oxly cmds to init file in the ancestor db on laptop something like this:
$ mkdir /tmp/myoxlyrepo ; cd /tmp/myoxlyrepo
$ oxly clone --init-ancdb dropbox://orgzly/foo.orgMake edits and save same file as needed on Emacs and Orgzly like usual.
Save file shared via Dropbox on laptop/Emacs (~/Dropbox) as needed.
On mobile/Orgzly save (locally) the same note as needed.
When ready to sync/merge, on Orgzly select Sync notes on Orgzly main menu.
If the sync fails and the Orgzly error msg says it's modified both local and remote -- this is the case we need oxly -- then Force Save (long press on note) on Orgzly.
The forced save is safe cuz the prev edits will be saved by Dropbox as seperate revisions.
But once you do this don't make any more changes (via Emacs/Orgzly/etc) to the file as it may cause problems with the merge. See section Caveats/Gotchas below.
Now the 2 most recent revisions -- one each from Emacs and Orgzly -- in Dropbox are ready to be merged with oxly:
Run oxly cmds via oxmerge script on laptop something like this:
$ cd /tmp/myoxlyrepo
$ oxmerge dropbox://orgzly/foo.org2a. If oxmerge finished with no conflicts -- YAAAY -- goto step 3 below.
2b. If oxmerge finished with conflicts -- BOOOO -- choose one of the options output to resolve the conflict(s).
Sync (Force Load not necessary) to load merged/latest revision from Dropbox. This should be done before any other changes are saved to Dropbox.Congrats your file is merged.
oxmerge dropbox://orgzly/misc-notes-spring17.org
oxly, version 0.9.21
Cloning dropbox://orgzly/misc-notes-spring17.org into /tmp/oxnotes ...
Moving/saving old /tmp/oxnotes/.oxly/.tmp to /tmp/oxnotes/.oxly/.old/oxlytmp.10636 ... done.
Downloading metadata of 100 latest revisions on Dropbox ... done.
Checking 2 latest revisions in Dropbox...
downloading rev 33880446decd data ... done.
downloading rev 33870446decd data ... done.
Checking ancestor db ... already downloaded.
Checking ancestor rev data ...
downloading rev 33670446decd data ... done.
Viewing metadata latest 2 revisions (cached locally) ...
33880446decd 26242 2017-04-24 01:54:16 EDT-0400 427013b2
33870446decd 28816 2017-04-24 01:50:32 EDT-0400 b97299f0
Viewing metadata least latest 2 revisions (cached locally) ...
32cf0446decd 20509 2017-04-19 11:41:03 EDT-0400 bcba0f1d
32ce0446decd 20504 2017-04-19 11:38:50 EDT-0400 4591b454
Merging latest 2 revisions data ...
No conflicts found. File fully merged locally in orgzly/misc-notes-spring17.org
Pushing merged revision data ...
Uploading staged orgzly/misc-notes-spring17.org to Dropbox as /orgzly/misc-notes-spring17.org ... done.
Uploading ancestor db orgzly/_oxly_ancestor_pickledb.json to Dropbox ... done.
Please select Sync (regular, Forced not necessary) note on Orgzly now.
It should be done before any other changes are saved to this file on Dropbox/Emacs/Orgzly.oxly cmd --help.clone phase.push it will run ancdb_push. If the the data file upload succeeds but ancdb upload fails, you can run ancdb_push by hand.;; don't start another frame
;; this is done by default in preluse
(setq ediff-window-setup-function 'ediff-setup-windows-plain)
;; put windows side by side
(setq ediff-split-window-function (quote split-window-horizontally))
;; revert windows on exit - needs winner mode
(winner-mode)
(add-hook 'ediff-after-quit-hook-internal 'winner-undo)
(add-hook 'ediff-prepare-buffer-hook #'show-all)oxmerge url at a timeIf the file/hash key not found in ancdb or hash seems incorrect you can't do usual merge but we can 2-way merge2and reset the file/hash key:
# Note this assumes last Orgzly/Orgzly versions are current/current-1 revs in Dropbox.
# If not, see `log` cmd and use --rev `merge2` options.
$ oxly clone dropbox://orgzly/foo.org # get latest ancdb
$ oxly merge2 orgzly/foo.org # merge by hand w/emacsclient
$ oxly push --add orgzly/foo.org # will reset ancdbor if you don't need to merge now but want to reset ancdb for this file:
$ oxly clone --init-ancdb dropbox://orgzly/foo.org # get latest ancdb$ oxly clone dropbox://orgzly/foo.org
$ oxly log --oneline orgzly/foo.org #find rev needed
# oxly cat and diff handy here
$ oxly cat --rev $rev orgzly/foo.org > orgzly/foo.org
$ oxly push --no-dry-run --add orgzly/foo.org
# view/check it
$ oxly clone dropbox://orgzly/foo.org
$ oxly cat orgzly/foo.orgpush it will run ancdb_push. If the the data file upload succeeds but ancdb upload fails, you can run ancdb_push by hand.oxly is not git -- no real commits, no branches, single user, etc. But as far as a poor-man's DVCS goes, oxly can be useful when git is not avail. Oxly just implements enough of a subset of git to support a basic clone-merge-add-push flow (and a few others to view the revisions and merged file). New files in wd/index not supported.
My use case is Emacs on a Unix laptop and Android Orgzly on mobile phone so far only only that config has much real world use. More clients should be viable as long as two at a time are merged/pushed in a careful manner (don't save any non-oxly changes to Dropbox while this is being done). There's no locking done here so the user has to be careful and follow the procedure above.
Only handles a single file on Dropbox as remote repo (might be expanded to a dir tree in future).
$ export PYTHONPATH=/tmp/pypath
$ mkdir /tmp/pypath && python setup.py develop --install-dir /tmp/pypath
# If you bumpup the version of dev pkg and have a prev version installed globally,
# you will probably have to uninstall global version to test dev version.
$ oxly --version
oxly, version 0.10.10
$ sudo python -m pip uninstall oxly
$ /tmp/pypath/oxly --version
oxly, version 0.10.11
# note valid Dropbox auth token needed in ~/.oxlyconfig
$ PATH=/tmp/pypath:$PATH bash oxly/tests/run-tests.shhttps://github.com/dropbox/dropbox-api-content-hasher/blob/master/License.txt
MIT. See LICENSE file for full text.
None.
Copyright (c) 2016 Glenn Barry (gmail: gaak99)
http://www.orgzly.com/help#Both-local-and-remote-notebook-have-been-modified
https://github.com/dropbox/dropbox-api-content-hasher.git
https://www.gnu.org/software/emacs/manual/html_node/ediff/
http://blog.plasticscm.com/2010/11/live-to-merge-merge-to-live.html?m=1
https://cloudrail.com/compare-consistency-models-of-cloud-storage-services/
The hackers behind Dropbox, Orgzly, emacs/org-mode/ediff, Python/Click, git/github/git-remote-dropbox, and others I'm probably forgetting.