Curl like client to test tus API implementation?

[olalonde]

Is there any command line client to upload a file to a tus server? Would be useful for testing implementations.

[kvz]

Hi Oli, currently there's no curl based client. We did have some simple demo code, but it dates from 2013 and has been deprecated and removed from the tusd repository: https://github.com/tus/tusd/blob/f67ce78c4d75e86afc400c8e102542f61e3cf312/scripts/demo-alphabet.sh - it's not compatible with tus 1.0.

It might be possible to use it as a starting point for some new demo code, that we could perhaps store in a wiki or on the site somewhere. Alternatively, the tus-js-client has been recently modified to work with Node.js as well as (the already supported) browsers. It only exposes tus as an SDK, but it shouldn't be too hard requireing it in a Node.js cli utility.

Then there is a Ruby client in the works by the girls and guys from Vimeo (@alexanderbez specifically), perhaps this could also be used for your purposes, depending if you have/want/like Ruby as a dependency (this goes for Node as well of course). Our own @ifedapoolarewaju is contemplating building one in Python, same story here.

For completeness sake, there are also some more ideas for implementations here https://github.com/tus/tus-resumable-upload-protocol/issues/67 - so maybe we need to add the curl/bash idea.

Let me know if any of these routes seem interesting to you, and also if you're in the position to help us get this set up. Depending on your feedback I'd also like @Acconut to weigh in on this, but I personally feel that a cli tool or examples to test and play with a tus(d) server could aid in adoption, and a more pleasant onboarding experience for developers. I think the curl route makes it very easy for developers of all languages to change things around and play with it, so adding something like that (aside from all the other clients that are also happening) would have a +1 from me.

[Acconut]

I personally feel that a cli tool or examples to test and play with a tus(d) server could aid in adoption, and a more pleasant onboarding experience for developers.

I agree. This also may help understanding the basic steps a client library does while uploading a file.

I think the curl route makes it very easy for developers of all languages to change things around and play with it

From my personal experience, using curl directly to speak to the tus server is not pleasant. You basically always have to remember to include the correct headers which makes the command quite verbose, e.g.:

curl -X PATCH -i -H "Tus-Resumable: 1.0.0" -H "Upload-Offset: 0" -H "application/offset+octet-stream" https://master.tus.io/files/9b25954262f74b232656d62ca5721bf4 -d @file.ext

What I would prefer is a zero-dependency binary which you can simply download somewhere and execute without worrying about installed runtimes or interpreters. Sadly, this is not easily achievable with the current client libraries written in Java, JavaScript and Python. I started a pet project using the Crystal language (https://crystal-lang.org/) but never released it due to immaturity and since it relied on a custom fork of the runtime. It may be worth investing in this direction.

[olalonde]

A Go library would be cool. And it could be wrapped into a no-dependencies CLI.

[kvz]

+1 For a Go client. I'd also be interested to see a Rust client (this isn't either or afaic). At the same time, I think there still is a usecase for maintained curl snippets. I think there's enough people that will copypaste them into bash scripts, creating 10minute abstractions, just to get a feel for the tech and do some prototyping. With a black-box binary, they might think they still don't understand tus, we'd also be making choices for them in terms of interface that might not necessarily be what they are after

[Acconut]

A Go library may be a good way to go.

I'd also be interested to see a Rust client

I am not an expert on the current status of Rust but if you want to consume HTTP in any form you are in kind of dilemma. It's standard library does not feature a HTTP interface (neither server nor client) and the most usable community projects should still not be considered stable (e.g. see https://github.com/hyperium/hyper#hyper). Therefore Rust may not be the best option if you want to produce a working project soonish.

I think there's enough people that will copypaste them into bash scripts

I do agree but I would not recommend them for people how want to get started in the most easiest way. It would also be nice to have a tutorial for understanding the protocol which could feature some curl statements for helping to understand the basics.

[fentas]

I wrote a quick bash script (cURL).

#!/usr/bin/env bash
# ref:
# - https://tus.io/blog/2015/11/16/tus.1.0
# - https://github.com/tus/tus.io/issues/96

: {TUSD:=https://tusd/files/}

if [ ! -f "${1}" ]; then
  echo -e "\n\033[1;31m✘\033[0m First argument needs to be a file.\n"
  exit 1
fi

file=${1}
filename=$(basename "${file}" | base64)
filesize="$(wc -c <"${file}")"

# Apparently 'Location: ..' is terminated by CRLF. grep and awk faithfully
# preserve the line ending, and the shell's $() substitution strips off the
# final LF leaving you with a string that just ends with a CR.
#
# When the CR is printed, the cursor moves to the beginning of the line and
# whatever gets printed next overwrites what was there.
# ... | tr -d '\015'
location=$(curl \
  -s -I \
  -X POST \
  -H "Tus-Resumable: 1.0.0" \
  -H "Content-Length: 0" \
  -H "Upload-Length: ${filesize}" \
  -H "Upload-Metadata: name ${filename}" \
  ${TUSD} | grep 'Location:' | awk '{print $2}' | tr -d '\015')

if [ -n "${location}" ]; then
  curl \
    -X PATCH \
    -H "Tus-Resumable: 1.0.0" \
    -H "Upload-Offset: 0" \
    -H "Content-Length: ${filesize}" \
    -H "Content-Type: application/offset+octet-stream" \
    --data-binary "@${file}" \
    "${location}" -v
else
  echo -e "\n\033[1;31m✘\033[0m File creation failed..\n"
  exit 1
fi

Just use it like

./upload.sh ~/Downloads/some.zip

*NOTE no resume + all data at once, but it works

:+1: For go client!

[kvz]

Nice! Would you be able to add a resume too? Just as an example, I think this might help to clarify the protocol to people a lot.

Sent from mobile, pardon the brevity.

On 29 Jun 2017, at 19:29, Jan Guth notifications@github.com wrote:

I wrote a quick bash script (cURL).

!/usr/bin/env bash

ref:

- https://tus.io/blog/2015/11/16/tus.1.0

- https://github.com/tus/tus.io/issues/96

: {TUSD:=https://tusd/files/}

if [ ! -f "${1}" ]; then echo -e "\n\033[1;31m✘\033[0m First argument needs to be a file.\n" exit 1 fi

file=${1} filename=$(basename "${file}" | base64) filesize="$(wc -c <"${file}")"

Apparently 'Location: ..' is terminated by CRLF. grep and awk faithfully

preserve the line ending, and the shell's $() substitution strips off the

final LF leaving you with a string that just ends with a CR.

#

When the CR is printed, the cursor moves to the beginning of the line and

whatever gets printed next overwrites what was there.

... | tr -d '\015'

location=$(curl \ -s -I \ -X POST \ -H "Tus-Resumable: 1.0.0" \ -H "Content-Length: 0" \ -H "Upload-Length: ${filesize}" \ -H "Upload-Metadata: name ${filename}" \ ${TUSD} | grep 'Location:' | awk '{print $2}' | tr -d '\015')

if [ -n "${location}" ]; then curl \ -X PATCH \ -H "Tus-Resumable: 1.0.0" \ -H "Upload-Offset: 0" \ -H "Content-Length: ${filesize}" \ -H "Content-Type: application/offset+octet-stream" \ --data-binary "@${file}" \ "${location}" -v else echo -e "\n\033[1;31m✘\033[0m File creation failed..\n" exit 1 fi Just use it like

./upload.sh ~/Downloads/some.zip *NOTE no resume + all data at once, but it works

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[fentas]

:sparkles: I'll try to script something in the next days.

[fentas]

:hammer_and_wrench: Here is the promised follow-up https://github.com/fentas/tus-shell-client

It can split- and upload files (resumable). Also, it should be able to upload in parallel, but it is not properly tested. The rest will follow.

[olalonde]

@fentas great. Btw, I think TUS_CHUNCK_SIZE should be TUS_CHUNK_SIZE.