doesn't upload

[francoisdop]

Thank you for the package.

i follow carefully all the instructions. Started tus server, but absolutely no upload occurs.

I am testing it on MAMP / OSX / NGINX properly pointed the endpoint. But no upload at all happened. I tried to proxy_pass the nginx server with no success.

What am I doing wrong?

Thank you for your help

[avvertix]

Hi, for OSX we currently have limited support. To use the integration you need to copy the linux hooks (under the hook/linux directory) to the macos folder hooks/macos. At that point the tusd executable should communicate with php.

[francoisdop]

thanks for your quick reply.

You mean renaming the folder from linux to macos ... or creating a new folder somewhere else ?

Maybe I did't set up properly nginx. do I need to set up a proxy ? and if yes how?

Thanks

[avvertix]

You mean renaming the folder from linux to macos ... or creating a new folder somewhere else

Sorry, I checked and I forgot that macos is considered linux, so the Linux hooks should work. One important thing is that you must check if those shell scripts are executable. If not a chmod +x on each file should do the job.

Maybe I did't set up properly nginx. do I need to set up a proxy ? and if yes how?

If you are using NGINX to proxy the requests to the tusd server, you need to do two things

1. configure NGINX to proxy the requests (copied from https://github.com/k-box/k-box/blob/master/docker/nginx-default.conf#L59-L77)

    location ~* /tus-uploads {

    # Disable request and response buffering
        proxy_request_buffering  off;
        proxy_buffering          off;
        proxy_http_version       1.1;

        # Add X-Forwarded-* headers
        proxy_set_header Host $http_host;
        proxy_set_header X-Forwarded-Host $http_host;
        proxy_set_header X-Forwarded-Proto $http_x_forwarded_proto;

        proxy_set_header         Upgrade $http_upgrade;
        proxy_set_header         Connection "upgrade";

        client_max_body_size     0;

        proxy_pass http://127.0.0.1:1080;  
       # I'm assuming tusd will listen on port 1080      
    }

2. Configure the Laravel integration to know that requests are proxies. This can be done via environment variables, as in the code block below, or by publishing the configuration file php artisan vendor:publish --tag=tusupload-config

TUSUPLOAD_USE_PROXY=true
TUSUPLOAD_HOST=0.0.0.0
TUSUPLOAD_HTTP_PATH=/tus-uploads/
TUSUPLOAD_URL=${APP_URL}tus-uploads/

where ${APP_URL} is the domain on which the Laravel application listen, e.g. http://localhost:8000/. Here I'm assuming that from the outside you want tus to be available on/tus-uploads/` path. One quick point: slash at the end of the path are important

[francoisdop]

Ok thanks again for this very quick and detailed answer.

Still confused :

• What is the difference between TUSUPLOAD_HTTP_PATH and TUSUPLOAD_URL ?

• on the client side does the endpoint has to be exactly the same than the TUSUPLOAD_HTTP_PATH or is the endpoint only for authorization check (in this case how the js script knows where to send the file?) ? :

var endpoint = '/tus-uploads/' var uploader = new window.TusUploader({autoUpload: true, endpoint: endpoint});

• if tus server is listening to another port how to make the js client tusUpload know about it?

Thanks!

[avvertix]

What is the difference between TUSUPLOAD_HTTP_PATH and TUSUPLOAD_URL ?

TUSUPLOAD_HTTP_PATH is the path to which the tusd server listen on. It matches 1 to 1 to the --base-path tusd argument. This is also the path you want NGINX to proxy to the tusd server.

TUSUPLOAD_URL is the absolute url that the Laravel integration will use if running behind a proxy

on the client side does the endpoint has to be exactly the same than the TUSUPLOAD_HTTP_PATH or is the endpoint only for authorization check (in this case how the js script knows where to send the file?) ?

The javascript client wants the endpoint that OneOffTech\TusUpload\Tus::routes() is creating. In particular it is /uploadjobs (not configurable at the moment). Probably this step is not well documented in the readme, sorry.

That endpoint will authorize the upload based on the defined Gate upload-via-tus and, if authorized, will return to the library the necessary information to contact the tusd server for starting the upload.

if tus server is listening to another port how to make the js client tusUpload know about it?

There is an environment variable that let you configure so, it is called TUSUPLOAD_PORT. If you did publish the configuration file set the port key.

[avvertix]

The general workflow can be summarized like so

1. the js calls the /uploadjobs endpoint to verify that the authenticated user has the rights to perform the upload (this is handled within Laravel, no tusd involvement here)
2. if authorized it returns the url to the tusd service (if behind proxy it returns the TUSUPLOAD_URL)
3. the library now uses the tus protocol and communicate to the obtained endpoint
4. tusd runs the hook to confirm that the upload was authorized and to report the status of the upload

[francoisdop]

Ok thanks.

I understand now properly the workflow and set up everything properly, but no upload occurs!

in my .env : TUSUPLOAD_USE_PROXY=true TUSUPLOAD_HOST=127.0.0.1 TUSUPLOAD_PORT=1084 TUSUPLOAD_HTTP_PATH=/tus-uploads/ TUSUPLOAD_URL=http://localhost:8888/blog/tus-uploads/

after i started tusd server : [tusd] Using '/Applications/MAMP/_BLOG/hooks/linux' for hooks [tusd] Using '/Applications/MAMP/_BLOG/storage/app/uploads' as directory storage. [tusd] Using 0.00MB as maximum size. [tusd] Using 127.0.0.1:1084 as address to listen. [tusd] Using /tus-uploads/ as the base path.

my nginx set-up: location /blog/tus-uploads { proxy_set_header Host $http_host; #all the other headers .... proxy_pass http://127.0.0.1:1084; }

and the json response of /uploadjobs:

filename: "file" location: "http://localhost:8888/blog/tus-uploads/" request_id: "cjunywzlu00003h5lkvorsudb" size: 20 upload_token: "GJU0ClMyBsQVgImfpnbobywrbkAXDt7TAF1cjunywzlu00003h5lkvorsudb"

(it properly inserted the upload in the database)

But then nothing happens !

PS: i tested the proxy that works well. So my guess it has to be something with the hooks (which I made executables), but I have no clue how to debug this

Thank you so much for you time and help by the way...

[avvertix]

based on your information I guess it is related to the hooks or to NGINX not passing the requests to the tusd executable.

I would check first the tusd log. It should be printed on stderr and, after an upload request is received, should look like (I tried it on our K-Box instance, which is using this package)

[tusd] event="UploadCreated" id="3e771b511796b0d2e8db62e005ed71e2" size="15632" url="http://localhost:8000/tus-uploads/3e771b511796b0d2e8db62e005ed71e2"
[tusd] event="ResponseOutgoing" status="201" method="POST" path=""
[tusd] event="HookInvocationStart" type="pre-create" id=""
[tusd] event="HookInvocationFinish" type="pre-create" id=""
[tusd] event="HookInvocationStart" type="post-create" id="3e771b511796b0d2e8db62e005ed71e2"
[tusd] event="HookInvocationFinish" type="post-create" id="3e771b511796b0d2e8db62e005ed71e2"
[tusd] event="RequestIncoming" method="PATCH" path="3e771b511796b0d2e8db62e005ed71e2"

[tusd] event="ChunkWriteStart" id="3e771b511796b0d2e8db62e005ed71e2" maxSize="15632" offset="0"

[tusd] event="ChunkWriteComplete" id="3e771b511796b0d2e8db62e005ed71e2" bytesWritten="15632"
[tusd] event="ResponseOutgoing" status="204" method="PATCH" path="3e771b511796b0d2e8db62e005ed71e2"

[tusd] event="HookInvocationStart" type="post-receive" id="3e771b511796b0d2e8db62e005ed71e2"
[tusd] event="UploadFinished" id="3e771b511796b0d2e8db62e005ed71e2" size="15632"

[tusd] event="HookInvocationStart" type="post-finish" id="3e771b511796b0d2e8db62e005ed71e2"

if the pre-create hook fails the upload is denied.

In any case you can also disable the hooks to see if the upload then succeed (at least from the tusd perspective).

php artisan tus:start --no-hooks

Laravel will not know about the upload, but at least you can see if the tus storage will contain two files, one of them should be ending with .bin extension and contain the file you uploaded.

If nothing is reported in the tusd log and running tusd without hook works, then the problem is between javascript and NGINX.

On the NGINX side, I thought your application was immediately under localhost, but apparently you are also hosting it under a folder. I have to admin I never tried with this kind of setup.

This is the configuration you shared in the comment

location /blog/tus-uploads {
proxy_set_header Host $http_host;
#all the other headers .... 
proxy_pass http://127.0.0.1:1084;
}

And I think a possible problem could reside in the proxy_pass directive because you have a folder path before /tus-uploads, which is the default path for tusd. In that configuration tusd expects http calls to

http://127.0.0.1:1084/tus-uploads/

but you have another parent folder in the path, which is /blog. If I remember correctly NGINX passes the full path, so tusd receives this kind of requests

http://127.0.0.1:1084/blog/tus-uploads/

I'm not sure that tusd accepts /blog/tus-uploads/ as the base path, but you can try to set it up in that way using TUSUPLOAD_HTTP_PATH=/blog/tus-uploads/. If that work we have discovered an interesting option, otherwise the possible solution could be rewrite in NGINX the path received, or make the Laravel application directly under localhost, without an additional folder.

[francoisdop]

ok, I am trying to narrow down the issue (with my folder 'blog') :

in Nginx: location /blog/tus-uploads { proxy_pass http://127.0.0.1:1201; } (PS: each time i restart the tusd server i need to change the port otherwise it says its already in use).

to make a ultra basic test i have on my webpage : axios.patch('\tus-uploads');

if i don't start tus server, i have a 502 gateway error from nginx (normal proxying to nothing)

if I start tus server (with no hooks!), I can see that nginx redirect properly to tusd server:

on my console i can see (again with no hooks): [tusd] event="RequestIncoming" method="PATCH" path="" [tusd] event="ResponseOutgoing" status="412" method="PATCH" path="" error="unsupported version"

and on chrome debug: Tus-Resumable: 1.0.0 Access-Control-Expose-Headers: Upload-Offset, Location, Upload-Length, Tus-Version, Tus-Resumable, Tus-Max-Size, Tus-Extension, Upload-Metadata

but that work only if i set : TUSUPLOAD_HTTP_PATH=/blog/tus-uploads/

so the folder 'blog' is not an issue.

The only thing I can think of causing the problem is the hooks (not Nginx).

in the pre-create hook i can see: php ../../../../../artisan tus:hook -vvv pre-create "${payload}"

I tried to modify this to : php ../../artisan tus:hook -vvv pre-create "${payload}"

as the relative path to artisan should be only 2 folders up, but with no success.

Thanks you

[avvertix]

each time i restart the tusd server i need to change the port otherwise it says its already in use

Probably you have lot of spare tusd processes running, maybe they are not getting killed properly

The only thing I can think of causing the problem is the hooks

Following your test seems indeed a problem in the hooks. Unfortunately I don't have access to a mac book with MacOS, so I need to figure out how to try to replicate your problem. Maybe just try to create a shell script to replace a default hook to see if it is called

[avvertix]

as the relative path to artisan should be only 2 folders up, but with no success.

tusd set the current working directory to the path where the hooks reside. During normal execution ./vendor/oneofftech/laravel-tus-upload/hooks/linux/, that's why artisan is 5 folders up

[avvertix]

if I start tus server (with no hooks!),

I think is time to start tusd with another hooks directory or with changed hooks. I'd suggest to change the pre-create hook file to just output what it receives on standard input.

#!/bin/bash

pwd
payload=$(</dev/stdin)
if [ -z "$payload" ]; then
  echo "Error: no payload provided"
  exit 1
fi
echo "${payload}"

I believe that it should highlight the fact that hooks are not executed on macos. Maybe you should also try to download the latest tusd from https://github.com/tus/tusd/releases/tag/0.11.0 to verify if the same problem happens on newer version of the tusd server.

I'm experiencing the same scenario on Windows tus/tusd#267, so maybe there is a common problem.

[francoisdop]

Thanks for your answers and your care!

Updates:

I finally discover what was the problem! It was on the javascript client: Because of my complex vue.js set up i made a stupid mistake, instead of sending var file = e.target.files[0], i was sending var file = e.target. I thought it was the hooks because there were no errors output in the console. I discover it only when i changed tusuploader.js in assets\js (before compiling) line 158:

onError: handleUploadError.bind(this), to onError: function(error) {console.log("Failed because: " + error)},

only then i could see the error from the tus-client that i was not sending the file properly. For some reason your own implementation of handleUploadError doesn't propagate the inner error of tus-client-js.

Now the pre-create hook work !!

But I had to change : php ../../../../../artisan tus:hook -vvv pre-create "${payload}" to php ../../artisan tus:hook -vvv pre-create "${payload}"

as when i start the tusd server it output : [tusd] Using '/Applications/MAMP/_BLOG/hooks/linux' for hooks

(and not : ./vendor/oneofftech/laravel-tus-upload/hooks/linux/)

Now the file do upload (testing with .jpg image) so the hooks work. The error i have now: The uploaded file lost his extension becoming a .bin file (instead of a .jpg)

and in the console a lot of this errors:

• ERROR output ----------------- [tusd] event="HookInvocationStart" type="post-receive" id="b576607c4d4123c4468b6b7762e1c553"

or

• ERROR output ----------------- [tusd] event="HookInvocationFinish" type="post-receive" id="b576607c4d4123c4468b6b7762e1c553"

[avvertix]

I finally discover what was the problem! It was on the javascript client:

Great, I didn't consider it as a probable location of the problem.

only then i could see the error from the tus-client that i was not sending the file properly. For some reason your own implementation of handleUploadError doesn't propagate the inner error of tus-client-js.

It emits an upload.failed event, as the error handler is the same for all uploads and an upload can fail in an async way. You could listen to those events by calling the on method on a Tusuploader instance.

var uploader = new window.TusUploader(options: { /*...*/ });

uploader.on("upload.failed", function(data){
  // data is an Object with the following keys { upload: {}, type: 'upload.failed', error: {} }
});

Probably I should document it more.

But I had to change : php ../../../../../artisan tus:hook -vvv pre-create "${payload}" to php ../../artisan tus:hook -vvv pre-create "${payload}" as ... [tusd] Using '/Applications/MAMP/_BLOG/hooks/linux' for hooks

I was considering the hooks to not be changed, but you're right the path can change. Probably on this side make sense to enable a certain level of customization.

The uploaded file lost his extension becoming a .bin file (instead of a .jpg)

That is actually not an error, but this is how tus works. The file is transferred without the original extension and the mime type is only stored in the metadata. Please keep in mind that the mime type is added by the browser and therefore is better to check it twice once the upload is completed.

For example renaming the file and moving to a different folder could be done in the TusUploadCompleted event. To do so generate an event listener in your Laravel application for the OneOffTech\TusUpload\Events\TusUploadCompleted event. In the handle event you then have access to a OneOffTech\TusUpload\TusUpload instance

public function handle(TusUploadCompleted$event)
{
        // Access the TusUpload using $event->upload...
}

You could get the mime type sent by the client on a TusUpload instance by using the mimetype property. In addition you get all information you sent additionally as part of the metadata and also the path on disk, by using the path() method.

Consider tus as a way to upload files and not a storage location. Like in a form based upload you move the file from the temporary directory, also in case of tus uploads you should move them to the storage you want to use and name them accordingly to your naming convention.

and in the console a lot of this errors: ERROR output ----------------- [tusd] event="HookInvocationStart" type="post-receive" id="b576607c4d4123c4468b6b7762e1c553" or

That's not an error. Tusd version 0.8 (which is the current bundled within the package) output all information on standard error. The latest version (0.11) correctly prints them on standard output.

[francoisdop]

ok got it! thanks a lot.

Last (hopefully) questions:

• to update tus to 0.11, i just have to copy and replace the binary exe to the bin folder ? (i guess for mac osx it should be the tusd_linux_amd64.tar.gz or tusd_linux_arm.tar.gz ?)

• to implement a progress bar, i need to use the tusUploaderProgress event ? and somehow return a json response of the offset ?

• Is there a better way to restart the tusd server ? at the moment, I need to use ctrl-Z, then find all the processes (ps -ax | grep tusd-mac), then : kill -9

• will Uppy library (https://uppy.io/) will work with this package ?

[avvertix]

to update tus to 0.11, i just have to copy and replace the binary exe to the bin folder ? (i guess for mac osx it should be the tusd_linux_amd64.tar.gz or tusd_linux_arm.tar.gz ?)

I'm working on that in the pull request #13, but in general the rule is all amd64 and for Mac the Darwin version.

Is there a better way to restart the tusd server ? at the moment, I need to use ctrl-Z, then find all the processes (ps -ax | grep tusd-mac), then : kill -9

If I remember correctly Ctrl+Z pauses the execution of a command. I normally use Ctrl+C to kill it. Otherwise you need to enable the pcntl PHP extension which handles better the unix signals.

As part of #13 I made a small change to the shutdown workflow to reduce a timeout before which Symfony Process will send a kill event to the controlled process.

Not sure if it solves the problem, but would be very helpful if you can test that branch on OSX.

to implement a progress bar, i need to use the tusUploaderProgress event ? and somehow return a json response of the offset ?

You can listen on the upload.progress event of the TusUploader JS object. In the payload passed as first argument to the callback function you will get all the details

var uploader = new window.TusUploader(options: { /*...*/ });

uploader.on("upload.progress", function(data){
  // data is an Object with the following keys { upload: {}, type: 'upload.failed', percentage: float, total: int, transferred: int }
});

The event is raised only after the server confirm that the chunk is transferred.

will Uppy library (https://uppy.io/) will work with this package ?

Well, I guess it will not work without changes, because in this package the upload is processed directly by the tus-client only after checking that the logged-in user has the rights to do so. If I remember correctly Uppy connects directly to the tusd server after the file is selected. I didn't check the changes to Uppy recently, so my conclusion is just a guess.

[francoisdop]

ok perfect thanks.

I will test the new branch as soon as I have a bit of time.