Upload completes, but no response

[sujayjaju]

Hi

I have a cron task that uploads the video using the PHP Api. But for some reason, I am getting a response back 1 out of 3 times - though the videos do complete uploading.

The videos are ~ 150 MB and take less than a minute to upload. But the process stays stuck waiting for response. There is no error. Its just stuck.

Any ideas on how I can debug?

Thanks.

[sujayjaju]

The issue seems to stem from the response handling under: perform_upload_tus() I see a 409 response from the server with the error message mismatched offset

Any idea how to fix or what could be causing it?

[sujayjaju]

Figured the issue out: the first call fails with the error: Error:Unable to complete request. [Operation timed out after 30000 milliseconds with 0 bytes received]

All subsequent calls then keep failing. I fixed it by adding the following curl options under perform_upload_tus(): CURLOPT_TIMEOUT => 240,

Can you add that one line or should I setup a PR?

[erunion]

I'd recommend setting CURLOPT_TIMEOUT to 30 seconds (to match the $CURL_DEFAULTS in the SDK. I'll reach out to our upload team to see if we had any issues with tus uploads this morning.

Feel free to set up a PR.

[frutality]

Hi!

Any news on this? It seems I'm experiencing same problem. No response from API, but empty video is present. Happens only when I'm trying to upload big files (400 mb). I'm using Laravel wrapper and simple code:

        Log::info('started to vimeo'); // got this in log
        $uri = Vimeo::upload($file->path(), [
            'name' => 'Video',                
        ]);

        $response = Vimeo::request($uri);
        Log::info('finished to vimeo'); // but not this

Also maybe related with https://github.com/vimeo/vimeo.php/issues/167?

I set up nginx' "fastcgi_read_timeout" and php's "max_execution_time" to 1 hour, so I'm almost sure it's not timeout problem.

[frutality]

I should add, that this happens with my developer "free" account. Right now, with same file, everything seems fine at production when uploading to PLUS account. I'll do more tests with different and bigger files.

[frutality]

One more update. Free account has nothing with that. Tried to upload ~900mb mp4 to PLUS account, same problem.

[frutality]

Another update. Just tried to upload file with GuzzleHttp Client instead of official Vimeo client. Small files uploaded without problems. Big files causes error "cURL error 55: SSL_write() returned SYSCALL, errno = 32".

I have OpenSSL 1.1.0f 25 May 2017. I read old version could cause something like that, but mine is fresh.

[erunion]

So you're getting no response at all from the API, or the video is being created but it's stuck in a processing state?

[frutality]

If using Vimeo Client - no response, no error at all. Video is created (I can see it in my vimeo dashboard), but it's empty (0:00).

If using GuzzleHttp Client - I got curl error. I found something curious about it:

The output you see is from lib/ssluse.c in libcurl's source code and the "errno" mentioned there is not the libcurl error code but the actual errno variable at that time.

Therefore, I don't see actual error code. Trying to find it out today.

[frutality]

Ditched GizzleHttp, decided to try with clean curl from command line. And got interesting results.

Cmd is curl -v -X PATCH -H 'Tus-Resumable: 1.0.0' -H 'Upload-Offset: 0' -H 'Content-Type: application/offset+octet-stream' --data-binary '@/home/alex/Downloads/videos/example.flv' https://files.tus.vimeo.com/files/vimeo-prod-src-tus-eu/xxxx

1. Production server did not have curl installed until today. Fixed it, executed command and got

   We are completely uploaded and fine

   • GnuTLS recv error (-110): The TLS connection was non-properly terminated.

I understand this is not important error since all data was transferred. But it's been more than hour, and vimeo is still saying "Your video is uploading. When your upload is complete, we’ll start converting your file". Full curl log https://pastebin.com/4zJmw0qw

2. On development machine got other error

   GnuTLS recv error (-54): Error in the pull function.

I guess it is my, local problem, since I'm running Ubuntu inside of virtual machine with bridged network. May be related. But still confused, small files uploaded with no problems. Full log https://pastebin.com/yTWtgr7i

[frutality]

Another update. We are using PULL approach now instead of TUS. Good news: we got response from API.

Bad news, however, now we got "Invalid file" after uploading to API. But if I upload exactly same file from web-browser everything is fine. How that could be? I opened ticket #3702104.

[nerdbeere]

I sometimes experience a similar issue while uploading videos. For some reason sometimes I get a 408 Request Timeout from the server in which case perform_upload_tus runs into a infinity loop because that case is not handled at all. I still wonder why the server closes the connection.

One other thing I noticed is that the verification head requests don't show up in charles (which I used to look into that).

[kezek]

Can confirm , had this issue since v2.x got released.

[nerdbeere]

Can confirm that this issue has to do with big files and it's a huge issue for us. We can't use the PULL approach as the videos that we are uploading are behind a firewall.

I'd be really pleased if someone from the team could look into this because at this point we have to consider other services.

[sujayjaju]

@nerdbeere Setting CURLOPT_TIMEOUT => 240 under perform_upload_tus() fixed it for me

[erunion]

Sorry for radio silence on this, I've been swamped this week. Someone will look into it today or tomorrow.

[mbaumgartl]

Modifying CURLOPT_TIMEOUT may not work depending on the filesize or the speed of the internet connection.

The timeout handling is broken in the current version of the client. If the request is canceled due to the curl timeout, you will end up in an endless loop. This happens because request is used instead of _request. The first method prepends "https://api.vimeo.com" automatically and thereby produces an invalid url. This will throw a VimeoRequestException and the upload starts over.

I looked into the issue last week and fixed it in 232997d6c6. I didn't file a pull request because I'm still not happy about the error handling. I think the client can still run into an endless loop. But without knowing the possible server errors, it's hard to improve the upload.

[nerdbeere]

@nerdbeere Setting CURLOPT_TIMEOUT => 240 under perform_upload_tus() fixed it for me

I set CURLOPT_TIMEOUT to 480 now (from 120) and wrote a supervisor process that spawns the php process to upload and kills it after 10 minutes which works for our import use-case for now but I'd highly appreciate this getting fixed as it will be cause for more headaches in the long run.

@mbaumgartl There must be at least a catch all where all other potential errors are handled otherwise it will just stay in the loop

[erunion]

I don't have very high confidence in the Tus work I originally did for v2.0 so I'm rewriting it to use https://github.com/ankitpokhrel/tus-php right now instead. Our PHP is the only SDK of ours that has custom written Tus work, and I really want to get us away from that.

Since this library requires PHP >=7.1, we're bumping up the library requirement to that and dropping our existing support for >=5.3 <7.1.

Should have a functional PR in a couple hours!

[erunion]

Ok, I have a functional branch and PR here: https://github.com/vimeo/vimeo.php/pull/186

[erunion]

Ok, the fix has been tagged to 3.0.0-RC1. Please let me know if that resolves the problems you've been having and I'll make 3.0.0 official.

[nerdbeere]

Ok, the fix has been tagged to 3.0.0-RC1. Please let me know if that resolves the problems you've been having and I'll make 3.0.0 official.

Good to see that things are going forward here. Unfortunately our infrastructure is still on 7.0 so I can't use it for the moment.

[erunion]

3.0 has been released.

[selimhallac]

https://github.com/vimeo/vimeo.php/issues/167#issuecomment-492694061 Apply the method here. It works :)