Not Detecting Changes - Perhaps need Env

[benyanke]

When starting unison instances from a python script (using subprocess.popen w/ shell=false), I am unable to detect any changes. However, when running the same commands in my shell directly, or with shell=true, it works.

I suspect this may be something to do with the environment, but I'm not sure. I am not using any config files, all options are being specified directly as arguments. One such command and it's debug logs:

/usr/bin/unison /mnt/lan/pcnart ssh://aws-artshare-sync//mnt/local/pcnart -path="Art Department/M00000000 - M00000999" -copyquoterem=true -fastcheck -ignorecase -auto -copyonconflict -prefer=newer -perms=0 -dontchmod=true -retry=10 -repeat=5 -log=true -logfile=/tmp/biglog -dumbtty -auto -batch -ignorelocks -ignorearchives -times=true -watch=false -servercmd=/usr/bin/unison -debug=all

Logs:

Sleeping for 5 seconds...

Looking for changes
[ui] temp: Globals.paths = "Art Department/M00001000 - M00001999"
[update] Setting archive for //pcnartsync//mnt/lan/pcnart
[server: update] Setting archive for //ip-10-100-1-247//mnt/local/pcnart
[update] findOnRoot //ip-10-100-1-247//mnt/local/pcnart
[update] findOnRoot /mnt/lan/pcnart
[update] findLocal /mnt/lan/pcnart ("Art Department/M00001000 - M00001999")
[pred] ignore '"Art Department/M00001000 - M00001999"' = false
[update] buildUpdateRec: /mnt/lan/pcnart/\"Art Department/M00001000 - M00001999\"
[update]   buildUpdate -> Absent and no archive
[update] Setting archive for //pcnartsync//mnt/lan/pcnart

  Waiting for changes from server
[server: update] findLocal /mnt/local/pcnart ("Art Department/M00001000 - M00001999")
[server: pred] ignore '"Art Department/M00001000 - M00001999"' = false
[server: update] buildUpdateRec: /mnt/local/pcnart/\"Art Department/M00001000 - M00001999\"
[server: update]   buildUpdate -> Absent and no archive
[server: update] Setting archive for //ip-10-100-1-247//mnt/local/pcnart
Reconciling changes
[recon] reconcileAll
[recon] reconcile: 0 results
[update] Marking 0 paths equal
Nothing to do: replicas have not changed since last sync.

Any ideas?

[brabalan]

I think it's a problem of string interpretation (the "" are removed by the shell, but here they seem to be preserved and the path synchronized seem to be literally "Art Department/M00001000 - M00001999"). Can you try with a path that does not need to be quoted?

[benyanke]

Thank you so much! I got it working by simply removing the quotes. Turns out the subprocess.popen module doesn't require any quotes or escaped spaces, because it passes array entries as arguments directly, so there's no issue of spaces being confused for the end of the argument.

[benyanke]

In case anyone is curious, I just finished v1.0.0 of UnisonCTRL, my unison wrapper script for handling large datasets which I mentioned above.

I'm currently using it on my 1.9 TB, 1.1 million file dataset to achieve sub-minute sync latency.

benyanke/unisonctrl

[brabalan]

Nice!