`get` does not work on folders

[slzarate]

Neither ./dbxcli get xxx/* nor ./dbxcli get xxx/ work for downloading all files from a folder.

The error message is: Error: unexpected end of JSON input

Is there something I'm doing wrong, or has this feature not been implemented yet? Is there an easy way to download all files in a folder?

[lfaraone]

Thanks for the report @slzarate!

This was implemented in #22, but appears to be broken.

[diwakergupta]

Thanks @slzarate and @lfaraone !

@waits do you want to take a crack at this?

[ubergarm]

This is the only function that I needed, but it doesn't work on any of the half dozen release binaries that I tried.

So as a tribute to Guido who works at Dropbox, I figured I'd make a minimal CLI around dropbox-sdk-python so as to goad you golang crowd into fixing it. ;)

GITHUB ubergarm/pydbxcli DOCKER ubergarm/pydbxcli

I will admit that an ~7MiB golang binary is indeed smaller and more portable than packing a ~200 line python script inside an ~20MiB compressed Alpine Docker image w/ a Python interpreter.

Thanks guys for putting up with my teasing and keep up the good work!

[robphoenix]

I don't think this has been implemented yet. The functionality added in #22 is to be able to specify a destination folder, as in download file text.txt into the directory here:

dbxcli get text.txt here

Rather than downloading a directory.

I've had a brief look into it this morning, and unless I'm mistaken, you can't download a directory via the API. So, to implement this I assume you would have to get the metadata of the item, check the .tag field in the response to see if it's file or folder, and if it's a folder recursively download each file within it. Maybe someone with more knowledge of the API, @diwakergupta, could correct me if I'm mistaken or missing something.

@ubergarm I'd be interested to hear your understanding of this. I did look through your Python code but didn't manage to work out if this is what was going on.

[ubergarm]

@robphoenix

Ahh, thanks for clarifying my misunderstanding. Recursively downloading a directory and all its contents / files is the use case I desire.

The official dropbox-sdk-python has a built in function to recursively list all files/folders e.g. dbx.files_list_folder(path='', recursive=True)

My code just paginates through all the cursor results and:

1. For each remote folder found create a local folder
2. For each remote file download it into the correct local folder
3. Persist the remote timestamp metadata to the local filesystem.

Looking at the dropbox-sdk-go-unofficial function ListFolder() it may be the place to add a recursive flag or similar?

[diwakergupta]

@ubergarm ListFolder already supports the recursive flag, see ListFolderArg

So first step would be to add a --recursive option to dbxcli ls. Then you can pipe the output into dbxcli get. Would that work for you?

cc/ @robphoenix

[slzarate]

@diwakergupta FWIW, the workaround I ended up writing was effectively a recursive dbxcli ls that fed into dbxcli get. Glad this is getting some attention!

[robphoenix]

Thanks @ubergarm I suspected that's what was going on.

@diwakergupta It looks like that would work for sure, I intend on spending some time tomorrow looking into implementing this and hope to submit a PR for review soon. Thanks for #74 as well, really helps pointing the way.

[robphoenix]

Just wanted to mention that I've not had the time I had hoped to look into implementing this. It's still something I'd like to do, but it's not currently a priority I'm afraid. I don't want to stop someone else from solving this.

[debuti]

+1

[feklee]

FYI I wrote a set of wrappers around dbxcli:

https://github.com/feklee/dbx-tools

The goal is making it easy to maintain a local mirror of selected Dropbox files and directories. Supported in particular are recursive get and put. I use dbx-tools from within Termux, a terminal emulator for Android. (In fact, I developed dbx-tools on my phone.)

[kshitij10496]

Hey guys! 👋 Thanks a lot for the awesome work on the Go SDK and the CLI. 🤗 I stumbled on this issue while trying to solve the same problem.

For me, the method of piping results using: $ dbxcli ls --recurse | dbxcli get still results in an error. Am I doing something wrong here?

Also, would it not be a good idea if we could facilitate the user to download the folder as a zip file instead? The Dropbox API Documentation, as well as the Go SDK, have an endpoint implemented for this use-case.

It would be wonderful if we can integrate both the ways to download file(s) via the CLI into a single command get. Let me know what you guys think about this. I would be more than happy to fix this issue and open a PR. 😄

[shadiakiki1986]

Not sure how other people have used piping with dbxcli ls and get, but this is what worked for me

dbxcli ls -l "path/to/folder/to/download"|awk -F"ago" '{print $2}'|sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//'|xargs -I{} dbxcli get {} .

Versions

> dbxcli version
dbxcli version: v2.1.1
SDK version: 4.5.0
Spec version: a1d5111

Pseudocode:

• use the -l flag for ls (long format)
• split on "ago"
• trimming leading/trailing spaces
• piping into get as the first argument

If a -1 flag (one entry per line, like ls -1) were available, then the splitting and trimming wouldn't be needed.

The above command is wrapped into a shell script in this gist for simple installation to /usr/local/bin/dbxcli-getdir

[toddca]

Is anyone piping dbxcli ls to get using the windows command line? Looking for an example of how to get this working.

[debuti]

Since this dont seem to be fixed anytime soon I wrote a little py script to get the job done recursively. Let me know your thoughs

https://gist.github.com/debuti/5887c126811eeae1bf9451e73a7b8fd8

[rofrol]

I am using rclone to copy all files from dropbox:

rclone copy dropbox:/ .

[rofrol]

I got errors when using rclone. Something like these:

2019/05/31 16:01:00 ERROR : projects/javascript/date/doc/Writing_A_Daytime_Server_In_Node_js_pliki/shCore.js: Failed to copy: failed to open source object: path/restricted_content/...
2019-05-31 16:19:20 ERROR : projects/g2/gopath/.git/objects/12/5883d3cb94901419f70602a09d79830d8cef7b: Failed to copy: failed to open source object: unexpected end of JSON input

It worked with the original dropbox sync tool.

[EricKeenan]

Implimenting a recursive feature would be nice! Is there any functionality with glob queries?

[aharwood2]

Since this dont seem to be fixed anytime soon I wrote a little py script to get the job done recursively. Let me know your thoughs

https://gist.github.com/debuti/5887c126811eeae1bf9451e73a7b8fd8

As this uses subprocess.run this will only work in Python 3.5+. I used this workaround for anyone interested. In addition, you need to change the line proc.stdout.decode to proc.[1].decode as the workaround returns a tuple.

[asanakoy]

Still not working? How to download an entire folder?

[shadiakiki1986]

dbxcli-extras has a getr command based on the code by @debuti above