Zsh completion

[zmwangx]

I just started using ACD and found this tool very helpful (thanks! I would have to write one myself otherwise since the Mac client is totally crap and web interface isn't very good either...). In an effort to learn the subcommands and their options, I wrote a complete Zsh completion definition that encompasses all subcommands, global and per-command options, as well as individual arguments.

The source file is here: https://github.com/zmwangx/zsh-completions/blob/master/src/_acd_cli. In case something happens to the file or the repo in the future, here's a gist with the current definition (but it's unlikely that I'll update the gist): https://gist.github.com/ef40f29afad361b5a3fd. To use it one needs to put it somewhere on the $fpath and load the new completion system with compinit (which is probably already done for you if you have Prezto, OMZ etc. installed).

---

12/06/2015 Update. The one-file version has been archived here or here and is no longer developed.

I have implemented support for external actions (functions/commands of the type acdcli-action), but to facilitate completion of those I had to split the definition up. The completions (as well shell wrapper with external actions support) are available as a Prezto module: https://github.com/zmwangx/prezto/tree/master/modules/acdcli.

If you don't use Prezto and don't care about defining custom external actions, *just grab the files _acdcli and `_acdclifrom [this directory](https://github.com/zmwangx/prezto/tree/master/modules/acdcli/functions), and put all of them somewhere on your$fpath`**.

---

Note that I've looked at the Bash completion in #127. I didn't want to maintain compatibility with Bash because I never bothered to learn the Bash completion system. Zsh does have bashcompinit that allegedly supports bash-styled completion definitions, but I've no idea how well it works. I'm not even sure if Bash's completion system is good enough to handle a fairly complex situation like this. Therefore, Zsh-only.

I know my completion def is not perfect:

• Most troublesome: completing remote paths. Not only is it slow (which I can't help — maybe it's just my heavily-loaded plattered disk, but it's more likely the API response time, which I haven't measured and likely never will need to, thanks to this project), but it has trouble completing weird paths (like ones with quotes in them), or when there are funny things in the argument to be completed, say a shell parameter. That's because the word I receive for completion is a verbatim copy of what's in the ZLE buffer, so I could get word1\ word2, '1'\''2', $HOME/bin, etc. Currently my strategy is to strip all quoting (', ", \), which should work reasonably well in the great majority of cases; and in the worst case you just don't get a match and have to type manually. I think it's okay though. After all I'm not in the position to write _path_files, and I can't. Maybe I don't need to (I'm not so familiar with zshcompsys), so any help is appreciated.

  By the way, I looked at the Bash completion in #127 and the problem above seems to be ignored completely, whereas my simple strategy still covers most real world cases.

---

2015/12/08 Update. Performance should no longer be an issue even on slower machines, since I have disabled DB integrity checks for acd_cli calls.

---

• acd_cli upload has an unfortunate synopsis

  acd_cli [options] upload [options] path [path ...] parent

  One will immediately realize that it's impossible to know which argument is the parent (remote), since before you send the command with accept-line, any argument could potentially be the last one. I'm making a compromise here by assuming two positional arguments only, path and parent, because there are loads of ways to copy local paths. In case one wants to complete multiple local paths, the trick is to move the cursor to before the first path argument.

  No longer an issue. See https://github.com/yadayada/acd_cli/issues/207#issuecomment-161792865.

So the two problems stated above are the central ones. There might be other smaller issues that I've forgotten about. Anyway, the source file itself is fairly well-documented. Hope it helps some people.

Edit: It goes without saying that comments, suggestions, bug reports and patches are welcome.

[zmwangx]

Ah, I knew I forgot something, and I did. For the restore action, I decided to not complete the node IDs, since it's very easy to have loads of crap in Trash, and retrieving them all would presumably be very expensive.

However, on second thought, we already have the list-trash action exactly for this kind of situation, and it's not hard to parse, so why not. I'm now thinking of setting a timeout for all external calls to acd_cli (which could be controlled by a zstyle), and the call would be killed when it reaches the timeout, thus cutting down the wait time without user actively interrupting. Might implement this tomorrow, or another day. Too tired tonight.

[zmwangx]

Timeout and trashed node completion now implemented on the master branch (same link as before).

[AlexCzar]

@zmwangx , hi I've tried to use zsh-completions, but I don't see it working. I'm using oh-my-zsh, so I installed it via git into custom/plugins and added it to plugins directive: plugins=(git,some-other-plugins,zsh-completions)

That said, here are my two cents on the upload issue you have: I don't really know details of zsh completion implementation but it seams to me you could implement completions of path[ path ...] parent by suggesting remote paths and local paths simultaneously, e.g. adding virtual classifier, like this:

$ acd_cli upload my→
  r: /myDir1
  r: /myDir2
  l: myLocalDir1
  l: myLocalDir2

where l: sands for local and r: for remote.

[zmwangx]

I haven't got time to test with OMZ now, but could you let me know what happens with the second positional argument of acd_cli upload (should be remote only)? And what about other commands, e.g., download, overwrite — can you let me know their behavior? Also, what if you apply the following changes?

-            ':local path:_files'
-            ':remote parent folder:__acd_dirs'
+            '1:local path:_files'
+            '2:remote parent folder:__acd_dirs'

[zmwangx]

Ah wait a second, so the example is your suggestion? Okay, then ignore my previous comment, and please define "I don't see it working". You might want to remove your zcompdump and retry. @AlexCzar

P.S. Regarding the suggestion: Sounds reasonable. I'll consider that.

[zmwangx]

@AlexCzar Also, how exactly are you installing the file? Please specify its full path, and show me your $fpath. I don't see a plugin called zsh-completion in OMZ, by the way.

Note that the file has to be called _acd_cli (or at least something that begins with a _) to be inspected by compinit, so if you name it blah.plugin.zsh then it most definitely won't work.

[zmwangx]

Revised upload completion behavior in https://github.com/zmwangx/zsh-completions/commit/273799cf03ac8c306a9fea738f7847106aa67360: the first positional argument still completes to local paths only; from the second argument onward, both local paths and remote paths (dironly) are offered, under different tags locals and remotes. remotes takes precedence.

[yadayada]

Most troublesome: completing remote paths. Not only is it slow (which I can't help — maybe it's just my heavily-loaded plattered disk, but it's more likely the API response time, which I haven't measured and likely never will need to, thanks to this project)

The ls command is purely local, but it still takes a few deciseconds to complete. I'll rewrite the look-up in C as a shell script and compare the times.

[zmwangx]

@yadayada

It still takes a few deciseconds to complete.

Hmm, it usually takes at least two seconds and up to ten or even fifteen seconds for me. And my local db is only 151M at the moment, because I've only been using ACD for a few days and am still migrating loads of stuff into it... Do you think I have a serious performance problem that needs some tracing and debugging? By the way, I'm running a mid-2012 13'' nrMBP with an old plattered hard drive; could disk IO be my bottleneck?

[yadayada]

I cannot gauge whether your times are in a normal range. But I think a considerable portion is caused by acd_cli pulling in a lot of dependencies, you using magnetic storage in 2015, and SQLAlchemy and Python just generally being slow.

You can get a rough estimate of the possible speed-up by comparing the ls speed of acdfuse_c vs the one of acd_cli's FUSE module.

[zmwangx]

In fact, that horrible speed is the speed when it works at all. Sometimes ls just gets completely stuck and I have to kill. This happens about one third of the time it seems, and I just listed four times and got stuck 4/4. Even ^C won't respond. I waited out the fourth time, and behold:

> acdcli -v ls /
15-12-04 14:06:19.541 [INFO] [acd_cli] - Plugin leaf classes: TestPlugin
15-12-04 14:06:19.569 [INFO] [acd_cli] - TestPlugin attached.
15-12-04 14:06:19.570 [INFO] [acdcli.cache.db] - Initializing cache with path "/Users/zmwang/Library/Caches/acd_cli".
15-12-04 14:09:32.311 [INFO] [acdcli.cache.db] - Cache is initialized.
15-12-04 14:09:32.353 [INFO] [acdcli.cache.db] - DB schema version is 1.
15-12-04 14:09:32.612 [INFO] [acd_cli] - Last sync at 2015-12-04 21:01:17.682719.
15-12-04 14:09:32.612 [INFO] [acd_cli] - Resolved "/" to "<redacted>"
[<redacted>] [A] Arq Backup Data/
[<redacted>] [A] archive/
[<redacted>] [A] aud/
[<redacted>] [A] book/
[<redacted>] [A] vid/

Three minutes. There's certainly something fishy going on.

[zmwangx]

Just profiled a rather normal call that lasted for about 5s.

> time python3 -m cProfile -s tottime acd_cli.py ls / >|acd_profile.log
python3 -m cProfile -s tottime acd_cli.py ls / >| acd_profile.log  2.89s user 2.00s system 96% cpu 5.044 total

Results here: https://gist.github.com/anonymous/481abac144680087edff.

It seems that most of the time is spent on {method 'execute' of 'sqlite3.Cursor' objects} (tottime 4.069s).

[yadayada]

I'd like to know what times you get when doing the same with the sqlite3 CLI

cache_dir=`python3 -c 'import appdirs; print(appdirs.user_cache_dir("acd_cli"))'`
time sqlite3 $cache_dir/nodes.db 'SELECT name FROM nodes WHERE id IN (SELECT child FROM parentage WHERE parent == (SELECT id FROM nodes WHERE name is NULL and type == "folder")) AND status == "AVAILABLE" ORDER BY name;'

It's about 0.60s vs ~0.01s for me.

[zmwangx]

sqlite3 ~/Library/Caches/acd_cli/nodes.db   0.05s user 0.04s system 97% cpu 0.091 total

Not too bad. (I mean, definitely much worse than 0.01s, but no where near 0.6s.)

[zmwangx]

A bit of follow-up on performance: I just ran acd_cli through pdb and noticed that almost all the grinding time (or hangs) is spent on db integrity check. Disabling integrity check (--check=none) helps tremendously. I'll update the completion defs to not perform integrity checks.

[zmwangx]

Closing this since there's no further action.