search

wistia / wistia-uploader

A simple ruby command line upload client for Wistia.
MIT License
10 stars 3 forks source link

Hangs on uploading #1

Open morganestes opened 11 years ago

morganestes commented 11 years ago

Each time I try to use the uploader, it seems to hang - or at least no progress is indicated via the progress meter.

I'm running rubygems 1.8.23 and ruby 1.9.3 on OS X 10.7.5 (Lion).

Here's the output I get when I try to upload:

Ruby/Extensions: Binding#eval is already defined; not overwriting Ruby/Extensions: Enumerable#none? is already defined; not overwriting Ruby/Extensions: Enumerable#one? is already defined; not overwriting Ruby/Extensions: Integer#even? is already defined; not overwriting Ruby/Extensions: Integer#odd? is already defined; not overwriting Ruby/Extensions: Symbol#to_proc is already defined; not overwriting Found ~/.wistia.conf, loading defaults! Uploading 'Sq2Sq_Myth07_Fixing_Slice.mov'... 0% [> ] - 0 bytes

chroniton commented 11 years ago

Hola, sorry to hear about your difficulties with the uploader. I'm investigating the issue now, but it would help to know a bit more about the environment your running it in.

Gem versions (wistia-uploader uses multipart-post and json in particular), the shell command you executed, and the actual size of the target file might be useful.

BTW, I managed to track down a OS X 10.7 machine to test on and had no difficulties running the wistia-uploader from an rvm sandbox. If you haven't already, I'd recommend giving that a shot.

mmmmmrob commented 10 years ago

I have this same problem:

OS X 10.8.5 /Users/rob/.rvm/rubies/ruby-2.0.0-p247/bin/ruby /Users/rob/.rvm/rubies/ruby-2.0.0-p247/bin/gem

Example command:

wistia-uploader --json -k ${API_KEY} -p ${PROJECT_HASH} -n "${DISPLAY_NAME}" -f "${VIDEO}" > "${VIDEO}.upload.json"

the output looks like a curl output progress bar and sits static at 0.

Doing an upload via the command line curl I also get a problem that appears to be curl waiting for a 100 Continue from upload.wistia.com.

The issue may be down to curl versions?

curl 7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8y zlib/1.2.5 Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smtp smtps telnet tftp Features: AsynchDNS GSS-Negotiate IPv6 Largefile NTLM NTLM_WB SSL libz

Curl output:

mmmmmrob commented 10 years ago

Working directly with cUrl suggests that the issue is around the HTTP/1.1 100 Continue. The following command works in OS X 10.8.5

curl -i \
  -H "Expect:" \
  -F \"api_password=${API_KEY}\" \
  -F \"name=${DISPLAY_NAME}\" \
  -F \"project_id=${PROJECT_HASH}\" \
  -F file=@${VIDEO} \
  -o \"${VIDEO}.json\" \
  https://upload.wistia.com/

Note the -H that sets Expect to an empty header. This forces curl and the server to not use continue and the upload progresses.

It would be great to fix this in the wistia-uploader.

freerobby commented 10 years ago

@mmmmmrob Thanks for reopening this bad boy and for the curl sleuthing! Something is definitely not right here, at least on Ruby 2.0.0.

I believe the issue you're seeing is actually separate from what @morganestes posted a while back (which I believe was since fixed when we added Ruby 1.9 support several months ago -- sorry this ticket never got closed out).

I'm able to reproduce something similar to what you're seeing with some, but not all files. Certain files seem to cause it to hang mid-upload for me, and when they do, it hangs at the exact same spot every time for a given file (not at position 0, as you're seeing, however). I observe this behavior only on Ruby 2.0; when I downgrade to 1.9 it works as advertised for me.

If you have a moment, could you try running this on Ruby 1.9.3 and see if you still have the problem? Obviously we will want to get this running on Ruby 2.0 regardless, but it'd be helpful for me in narrowing down whether we're seeing the same issue or whether there is an additional problem I'm not picking up locally. I'm running OS X 10.9, so it's possible that has some impact, though I suspect not given that we are both running via rvm.

morganestes commented 10 years ago

@chroniton Man, I'm so embarrassed I never responded to your message back to me! I got pinged when this ticket updated and it reminded me to go back and look.

@freerobby Since I initially reported the problem, I've upgraded to OS X 10.9 Mavericks, which made me reinstall the wistia-uploader gem, along with multipart-post, json, and bundler. Once those all got working I tried the upload a file, which is stuck at 0%. It's a 3.3M WMV file that's already in my account in a project.

$ wistia-uploader v1.1.wmv
Found ~/.wistia.conf, loading defaults!
Uploading local file: 'v1.1.wmv'...
   0% [>                                                  ] - 0 bytes

When I break out of it, here's the output:

/usr/local/Cellar/ruby/2.0.0-p353/lib/ruby/gems/2.0.0/gems/wistia-uploader-1.0.0/bin/wistia-uploader:161:in `sleep': Interrupt
    from /usr/local/Cellar/ruby/2.0.0-p353/lib/ruby/gems/2.0.0/gems/wistia-uploader-1.0.0/bin/wistia-uploader:161:in `block in <top (required)>'
    from /usr/local/Cellar/ruby/2.0.0-p353/lib/ruby/gems/2.0.0/gems/wistia-uploader-1.0.0/bin/wistia-uploader:121:in `each'
    from /usr/local/Cellar/ruby/2.0.0-p353/lib/ruby/gems/2.0.0/gems/wistia-uploader-1.0.0/bin/wistia-uploader:121:in `<top (required)>'
    from /usr/local/opt/ruby/bin/wistia-uploader:23:in `load'
    from /usr/local/opt/ruby/bin/wistia-uploader:23:in `<main>'

I'm using the homebrew version of Ruby.

$ gem environment
RubyGems Environment:
  - RUBYGEMS VERSION: 2.0.14
  - RUBY VERSION: 2.0.0 (2013-11-22 patchlevel 353) [x86_64-darwin13.0.0]
  - INSTALLATION DIRECTORY: /usr/local/Cellar/ruby/2.0.0-p353/lib/ruby/gems/2.0.0
  - RUBY EXECUTABLE: /usr/local/Cellar/ruby/2.0.0-p353/bin/ruby
  - EXECUTABLE DIRECTORY: /usr/local/Cellar/ruby/2.0.0-p353/bin
  - RUBYGEMS PLATFORMS:
    - ruby
    - x86_64-darwin-13
  - GEM PATHS:
     - /usr/local/Cellar/ruby/2.0.0-p353/lib/ruby/gems/2.0.0
     - /Users/morganestes/.gem/ruby/2.0.0
  - GEM CONFIGURATION:
     - :update_sources => true
     - :verbose => true
     - :backtrace => false
     - :bulk_threshold => 1000
  - REMOTE SOURCES:
     - https://rubygems.org/

Gems installed:

$ gem list

*** LOCAL GEMS ***

actionpack (4.0.1, 4.0.0)
activemodel (4.0.1, 4.0.0, 3.2.13)
activeresource (4.0.0, 3.2.13)
activesupport (4.0.1, 4.0.0, 3.2.13)
addressable (2.3.5, 2.3.4, 2.2.8)
af (0.3.18.12)
archive-tar-minitar (0.5.2)
async_sinatra (1.1.0, 0.5.0)
atomic (1.1.14, 1.1.13, 1.1.12, 1.1.10, 1.1.9)
bigdecimal (1.2.3, 1.2.2, 1.2.1, 1.2.0)
blankslate (3.1.2, 2.1.2.4)
builder (3.2.2, 3.2.0, 3.1.4, 3.0.4)
bundler (1.3.5)
caldecott (0.0.5)
capybara (2.2.0, 2.1.0)
childprocess (0.3.9)
chunky_png (1.2.9, 1.2.8, 1.1.2)
classifier (1.3.3)
cmdparse (2.0.5)
colorator (0.1)
colorize (0.6.0, 0.5.8)
commander (4.1.5, 4.1.4, 4.1.3)
compass (0.12.2)
cookiejar (0.3.0)
crack (0.4.1)
css-spriter (1.0.1)
csscss (1.3.2, 1.3.1, 1.1.0)
directory_watcher (1.5.1, 1.4.1)
em-http-request (1.1.1, 1.1.0, 0.3.0)
em-socksify (0.3.0)
em-websocket (0.5.0, 0.3.8)
erubis (2.7.0)
escape_utils (1.0.0, 0.3.2)
eventmachine (1.0.3)
fast-stemmer (1.0.2)
ffi (1.9.3, 1.9.2, 1.9.0, 1.8.1)
fssm (0.2.10)
gist (4.1.3, 4.1.2, 4.1.1, 4.0.3, 4.0.0)
handbrake (0.4.0)
highline (1.6.20, 1.6.19, 1.6.18)
hpricot (0.8.6)
http_parser.rb (0.6.0.beta.2, 0.5.3)
httpclient (2.3.4.1)
i18n (0.6.5, 0.6.4, 0.6.1)
interact (0.5.2, 0.4.8)
io-console (0.4.2)
json (1.8.1, 1.8.0, 1.7.7, 1.6.8)
json_pure (1.8.1, 1.8.0, 1.6.8)
juicer (1.1.1, 1.1.0, 1.0.22, 1.0.21, 1.0.20)
kilt (0.5.1)
kramdown (1.2.0, 1.1.0, 1.0.2, 1.0.1, 0.14.2)
liquid (2.6.0, 2.5.4, 2.5.3, 2.5.2, 2.5.1, 2.5.0)
maruku (0.7.0, 0.6.1)
mime-types (2.0, 1.25, 1.24, 1.23)
mini_portile (0.5.2, 0.5.1, 0.5.0)
minitest (5.0.8, 5.0.7, 5.0.6, 5.0.5, 5.0.4, 5.0.3, 5.0.2, 5.0.1, 4.7.4, 4.7.3, 4.3.2)
multi_json (1.8.2, 1.8.1, 1.8.0, 1.7.9, 1.7.8, 1.7.7, 1.7.6, 1.7.5, 1.7.4, 1.7.3, 1.7.2)
multipart-post (1.2.0)
mustache (0.99.5, 0.99.4)
net-ssh (2.7.0)
net-ssh-gateway (1.2.0)
net-ssh-multi (1.2.0)
nokogiri (1.6.0, 1.5.9)
open4 (1.3.0)
parslet (1.5.0)
posix-spawn (0.3.6)
psych (2.0.2, 2.0.1, 2.0.0)
pygmentize (0.0.3)
pygments.rb (0.5.4, 0.5.2, 0.5.1, 0.5.0, 0.4.2)
rack (1.5.2)
rack-protection (1.5.1, 1.5.0)
rack-test (0.6.2)
rails-observers (0.1.2, 0.1.1)
railties (4.0.1, 4.0.0)
rake (10.1.0, 10.0.4, 0.9.6)
rb-readline (0.5.0, 0.4.2)
rdiscount (2.1.7, 2.1.6)
rdoc (4.0.1, 4.0.0)
redcarpet (3.0.0, 2.2.2)
rest-client (1.6.7)
rhc (1.17.6, 1.16.9, 1.15.6, 1.14.7)
ronn (0.7.3)
rubygems-update (2.1.11, 2.1.10, 2.1.9, 2.1.8, 2.1.6, 2.1.5, 2.1.4, 2.1.3, 2.1.0, 2.0.7, 2.0.6, 2.0.5, 2.0.4, 2.0.3)
rubytree (0.8.3)
rubyzip (1.1.0, 1.0.0, 0.9.9)
rufus-scheduler (3.0.2, 3.0.1, 3.0.0, 2.0.24, 2.0.23)
safe_yaml (0.9.7, 0.9.5, 0.9.4, 0.9.3, 0.9.2, 0.9.1, 0.7.1)
sass (3.2.12)
selenium-webdriver (2.37.0, 2.35.1, 2.35.0, 2.34.0, 2.33.0, 2.32.1)
sinatra (1.4.4, 1.4.3)
sqlite3 (1.3.8)
structured_warnings (0.1.4)
syntax (1.0.0)
terminal-table (1.4.5)
test-unit (2.5.5, 2.5.4, 2.0.0.0)
thor (0.18.1)
thread_safe (0.1.3, 0.1.2, 0.1.0)
tilt (2.0.0, 1.4.1)
tzinfo (1.1.0, 1.0.1, 0.3.37)
uuidtools (2.1.4)
wbench (0.4.0, 0.3.7, 0.3.6)
websocket (1.1.2, 1.1.1, 1.1.0, 1.0.7)
wistia-api (0.2.3)
wistia-uploader (1.0.0, 0.1.4)
xpath (2.0.0)
yajl-ruby (1.1.0)
freerobby commented 10 years ago

@morganestes Thanks for all that debug info. Sadly I'm still having trouble reproducing even with your gem versions for json/multi_json/multipart-post. I'm starting to wonder if it might even be a backend problem of some sort (seems unlikely, but I'm perplexed). Is it possible for you to send me the v1.1.wmv file that you're using so I can look into that? robby@wistia.com if you prefer to send it privately. We will get to the bottom of this!

morganestes commented 10 years ago

Thanks @freerobby. Video's on its way via email.

freerobby commented 10 years ago

@morganestes Thanks for that. I was able to reproduce a similar issue with that particular file on Ruby 2.0.0 (for me it hung at 10% rather that 0%, much the same as what I reproduced from what @mmmmmrob posted). I'm not sure if this will solve your problem, as you were already using multipart-post 1.2, which is the fixed version for Ruby 2.0. However I wonder if it's possible there was a chained dependency that didn't get picked up somewhere when we only required 1.5+.

Do you mind upgrading to the latest wistia-uploader gem that I just released (v1.1) and giving it a try?

morganestes commented 10 years ago

I tried it again, this time with the built-in ruby, and am still getting it stuck on 0%.

ruby 2.0.0p247 (2013-06-27 revision 41674) [universal.x86_64-darwin13]

bundler (1.3.5)
CFPropertyList (2.2.0)
json (1.8.1)
libxml-ruby (2.6.0)
multipart-post (1.2.0)
nokogiri (1.5.6)
sqlite3 (1.3.7)
wistia-uploader (1.1.0)
mmmmmrob commented 10 years ago

I find it a little too coincidental that both the wistia uploader and curl on the command line get stuck at 0% with videos that have uploaded successfully through the web UI (and with curl if the HTTP/1.1 Expect header is nulled)

This makes me think exploring what the wistia server is doing might be fruitful — the curl command line suggests that the wistia server is failing to send a 100 Continue response which would then trigger the upload of the file.

That might prove not to be it, but looks a likely candidate from where I'm sat.

morganestes commented 10 years ago

I gave it another shot with mixed results:

I installed rvm, along with ruby 1.9.3, 2.0.0, and 2.1.0, then tried to upload the same file. It worked just fine on 1.9.3, stuck at 0% on 2.0.0 and 2.1.0. I don't know much about Ruby, but that's just weird.