kvz
commented
8 years ago topics
I was thinking maybe we could do a FAQ based on those interview questions we did a while back. We'd strip out my name and such ofc. Pasting it below.
I agree with the other things you mentioned.
Which problem does ‘tus’ hope to solve?
It has happened to everyone,
Van Zonneveld explains,
You are trying to upload a fairly large file, but suddenly you enter a tunnel. Or maybe you're just coming home and switch from 4G to Wi-Fi. Because your connection drops for just a moment, the upload is aborted. When the connection is restored, you can start all the way from the beginning – if you even bother.
So, ‘tus’ can improve network connections?
Sadly not, no. What we can do, however, is resume the upload as soon as the connection is restored. That way, only the remaining bytes will have to be sent. When this process is automated, the end user might not even notice that the connection was lost. The upload completes in roughly the same time.
How did you come up with this idea?
Our company handles uploads from end users all across the world. They are often on the move and sometimes have terrible Wi-Fi, or they live in a place where network connections are generally unreliable.
We were receiving an increasing amount of complaints about failing uploads, while our servers in the data center were stable. This increase turned out to be a result of video files becoming ever larger, while network connections lagged behind for many people.
Since we cannot improve all the network connections of the world, we tried to find a solution that would hold up, even under the most dire of network conditions. That is when we started looking into ‘resumable file uploads’.
Resumable uploads, didn’t that already exist?
A few large companies, such as Google and Dropbox, had implemented resumable uploads. Sadly their solutions were commercially oriented, not aimed at web applications, and often only served their own ‘use case’. The available solutions are scattered. They are incompatible because they all speak a different language. They are not broadly applicable.
Imagine HTTP did not exist and that each company maintained its own web protocol. Websites running on Windows servers could then only be viewed with Internet Explorer. That is not how an open internet is supposed to work. There was already an open standard for reliably downloading web pages and files. Resumability for downloads has been embedded since HTTP/1.1. However, nothing had been set up for reliable uploading.
Why did you choose the open source route?
We are a small company. By going open source, we ultimately received cooperation from developers at Google, Vimeo, Yahoo, Node.js and ZeroMQ. We could have kept it to ourselves, but would the protocol have turned out to be as good? I don’t think so.
Aren’t you afraid your competitors will use this?
It would be a way for them to improve their service without much effort, so that is a risk. That being said, the protocol benefits greatly from widespread adoption, even by our competitors. In the end, it would result in uploading becoming better for everyone. We hope that enough potential customers will remain who don’t want to host servers themselves and will prefer to outsource their uploads. If that turns out to be not the case, we will have still gained experience and, of course, a great solution for our existing customers.
How does ‘tus’ compare to (S)FTP or BitTorrent?
Those are indeed also protocols for exchanging files. ‘tus’ has some overlap with them, but not completely. The main advantage of ‘tus’ is that it is based on HTTP. Because of this, it works in browsers, without the need for large libraries or additional firewall permissions (in an airport, for example, web browsing is possible, but you can forget about any FTP traffic). Moreover, it can be easily combined with existing infrastructure, such as HTTP load balancers, cachers, proxies and firewalls in the data center. Developing and expanding ‘tus’ is also relatively simple, because of excellent HTTP support in programming languages and command-line utilities.
Can you tell us something else about how ‘tus’ works?
‘tus’ is essentially a collection of rules and agreements. A document describing how uploading could work. It can only be of any value when many developers start to abide by these rules. Only then will a language emerge that is commonly spoken by apps and servers, and will everyone have reliable uploads. We are helping this along a bit by making example implementations available in every popular programming language.
Are there any other advantages to ‘tus’, besides reliability?
Speed. ‘tus’ is small at its core and offers optional extensions. One of these extensions is ‘concat’, which is able to break up files into smaller pieces and then send those simultaneously. This will help a great deal in combating network inefficiency. A few large internet companies, for example, use ‘tus’ in their data centers to send files of 100GB, thereby making full use of their available network capacity. Until now, this could only be done by using commercial solutions, such as Aspera.
Can you already name some early adopters?
We are very proud to announce that Vimeo will be using ‘tus’ for all file uploads. Not only from browser and mobile to the platform, but also between their servers in the data center.
Why didn’t you take the plane to Berlin?
Haha. Good question. We will be on this train for another five hours, while the flight to Berlin only takes about an hour. But in the end, you will lose twice as much time on transportation, waiting and safety checks. Time that I rather spend writing a new blog post. Or doing an interview with your magazine, for that matter. And the seats are actually quite comfortable, don’t you think?
Is there anything else you would like to tell our readers?
Thanks for making it all the way to the end! Feel free to have a look on tus.io, if this issue interests you. There are always things you can do to contribute.
StefanWallin
The tus.io website does not contain a lot of useful information for first-time visitors to get a good idea what the entire tus project is about. The only proper resources are the short introduction texts on the landing page and the list of implementation. Of course, there is also the specification but I would not consider it a document which is supposed to teach you what the tus protocol is, if you have no idea what this is all about. However, that is no shame since this is not its purpose. Therefore I propose a list of changes which are directed towards helping newcomers become members of the tus community:
I would be very pleased to hear feedback about these points from anyone or if someone has another idea on how we can improve the website.