thoughtbot / paperclip

Easy file attachment management for ActiveRecord
https://thoughtbot.com
Other
9.01k stars 2.43k forks source link

Paperclip issues when processing too much - Errno::E2BIG: Argument list too long - identify -format %wx%h #722

Closed YavorIvanov closed 12 years ago

YavorIvanov commented 12 years ago

Hi,

I'm having 2 projects. One of them is Rails 2.3.14 (ruby 1.8.7) and the other one is 3.2 (ruby 1.9.3). In both projects we are experiencing the same couple of errors.

The Rails 2 one is dealing with small images and runs fine but when running a mass job to reprocess pictures in about 400 items (each needs to reprocess about 6 pictures) it starts getting an error and needs a restart.

In the Rails 3 one, its dealing with large files and when the app runs certain amount of time it just starts to produce the error at random and crashes the app until restart is made.

That being said I want to focus on the one particular error that is popping too often and isn't triggered by a job as it crashes a perfectly good operating application on the current stable ruby and rails versions.

Errno::E2BIG: Argument list too long - identify -format %wx%h '/tmp/stream20120129-17335-xtxzs1.jpg[0]'

[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/core_ext/kernel/agnostics.rb:7:in ``'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/core_ext/kernel/agnostics.rb:7:in ``'
[GEM_ROOT]/gems/cocaine-0.2.1/lib/cocaine/command_line.rb:30:in `block in run'
[GEM_ROOT]/gems/cocaine-0.2.1/lib/cocaine/command_line.rb:51:in `with_modified_path'
[GEM_ROOT]/gems/cocaine-0.2.1/lib/cocaine/command_line.rb:28:in `run'
[GEM_ROOT]/gems/paperclip-2.5.0/lib/paperclip.rb:101:in `run'
[GEM_ROOT]/gems/paperclip-2.5.0/lib/paperclip/geometry.rb:22:in `from_file'
[GEM_ROOT]/gems/paperclip-2.5.0/lib/paperclip/thumbnail.rb:35:in `initialize'
[GEM_ROOT]/gems/paperclip-2.5.0/lib/paperclip/processor.rb:33:in `new'
[GEM_ROOT]/gems/paperclip-2.5.0/lib/paperclip/processor.rb:33:in `make'
[GEM_ROOT]/gems/paperclip-2.5.0/lib/paperclip/attachment.rb:440:in `block in post_process_style'
[GEM_ROOT]/gems/paperclip-2.5.0/lib/paperclip/attachment.rb:439:in `each'
[GEM_ROOT]/gems/paperclip-2.5.0/lib/paperclip/attachment.rb:439:in `inject'
[GEM_ROOT]/gems/paperclip-2.5.0/lib/paperclip/attachment.rb:439:in `post_process_style'
[GEM_ROOT]/gems/paperclip-2.5.0/lib/paperclip/attachment.rb:430:in `post_process_styles'
[GEM_ROOT]/gems/paperclip-2.5.0/lib/paperclip/attachment.rb:424:in `block (2 levels) in post_process'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/callbacks.rb:403:in `_run__404196128518846007__photo_post_process__2799403801532999261__callbacks'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/callbacks.rb:405:in `__run_callback'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/callbacks.rb:385:in `_run_photo_post_process_callbacks'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/callbacks.rb:81:in `run_callbacks'
[GEM_ROOT]/gems/paperclip-2.5.0/lib/paperclip/callback_compatibility.rb:54:in `run_paperclip_callbacks'
[GEM_ROOT]/gems/paperclip-2.5.0/lib/paperclip/attachment.rb:423:in `block in post_process'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/callbacks.rb:403:in `_run__404196128518846007__post_process__2799403801532999261__callbacks'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/callbacks.rb:405:in `__run_callback'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/callbacks.rb:385:in `_run_post_process_callbacks'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/callbacks.rb:81:in `run_callbacks'
[GEM_ROOT]/gems/paperclip-2.5.0/lib/paperclip/callback_compatibility.rb:54:in `run_paperclip_callbacks'
[GEM_ROOT]/gems/paperclip-2.5.0/lib/paperclip/attachment.rb:422:in `post_process'
[GEM_ROOT]/gems/paperclip-2.5.0/lib/paperclip/attachment.rb:119:in `assign'
[GEM_ROOT]/gems/paperclip-2.5.0/lib/paperclip.rb:344:in `block in has_attached_file'
[GEM_ROOT]/gems/activerecord-3.2.0/lib/active_record/attribute_assignment.rb:85:in `block in assign_attributes'
[GEM_ROOT]/gems/activerecord-3.2.0/lib/active_record/attribute_assignment.rb:78:in `each'
[GEM_ROOT]/gems/activerecord-3.2.0/lib/active_record/attribute_assignment.rb:78:in `assign_attributes'
[GEM_ROOT]/gems/activerecord-3.2.0/lib/active_record/base.rb:488:in `initialize'
app/controllers/administration/pictures_controller.rb:20:in `new'
app/controllers/administration/pictures_controller.rb:20:in `create'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/abstract_controller/base.rb:167:in `process_action'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_controller/metal/rendering.rb:10:in `process_action'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/abstract_controller/callbacks.rb:18:in `block in process_action'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/callbacks.rb:436:in `_run__1999084427942207763__process_action__2474360422919765723__callbacks'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/callbacks.rb:405:in `__run_callback'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/callbacks.rb:385:in `_run_process_action_callbacks'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/callbacks.rb:81:in `run_callbacks'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/abstract_controller/callbacks.rb:17:in `process_action'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_controller/metal/rescue.rb:29:in `process_action'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_controller/metal/instrumentation.rb:30:in `block in process_action'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/notifications.rb:123:in `block in instrument'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/notifications.rb:123:in `instrument'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_controller/metal/instrumentation.rb:29:in `process_action'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_controller/metal/params_wrapper.rb:205:in `process_action'
[GEM_ROOT]/gems/activerecord-3.2.0/lib/active_record/railties/controller_runtime.rb:18:in `process_action'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/abstract_controller/base.rb:121:in `process'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/abstract_controller/rendering.rb:45:in `process'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_controller/metal.rb:203:in `dispatch'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_controller/metal/rack_delegation.rb:14:in `dispatch'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_controller/metal.rb:246:in `block in action'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_dispatch/routing/route_set.rb:66:in `call'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_dispatch/routing/route_set.rb:66:in `dispatch'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_dispatch/routing/route_set.rb:30:in `call'
[GEM_ROOT]/gems/journey-1.0.0/lib/journey/router.rb:60:in `block in call'
[GEM_ROOT]/gems/journey-1.0.0/lib/journey/router.rb:48:in `each'
[GEM_ROOT]/gems/journey-1.0.0/lib/journey/router.rb:48:in `call'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_dispatch/routing/route_set.rb:570:in `call'
[GEM_ROOT]/gems/omniauth-1.0.2/lib/omniauth/strategy.rb:168:in `call!'
[GEM_ROOT]/gems/omniauth-1.0.2/lib/omniauth/strategy.rb:148:in `call'
[GEM_ROOT]/gems/omniauth-1.0.2/lib/omniauth/builder.rb:42:in `call'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_dispatch/middleware/best_standards_support.rb:17:in `call'
[GEM_ROOT]/gems/rack-1.4.1/lib/rack/etag.rb:23:in `call'
[GEM_ROOT]/gems/rack-1.4.1/lib/rack/conditionalget.rb:35:in `call'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_dispatch/middleware/head.rb:14:in `call'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_dispatch/middleware/params_parser.rb:21:in `call'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_dispatch/middleware/flash.rb:242:in `call'
[GEM_ROOT]/gems/rack-1.4.1/lib/rack/session/abstract/id.rb:205:in `context'
[GEM_ROOT]/gems/rack-1.4.1/lib/rack/session/abstract/id.rb:200:in `call'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_dispatch/middleware/cookies.rb:338:in `call'
[GEM_ROOT]/gems/activerecord-3.2.0/lib/active_record/query_cache.rb:64:in `call'
[GEM_ROOT]/gems/activerecord-3.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:443:in `call'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_dispatch/middleware/callbacks.rb:28:in `block in call'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/callbacks.rb:405:in `_run__2897959931721585378__call__2799403801532999261__callbacks'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/callbacks.rb:405:in `__run_callback'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/callbacks.rb:385:in `_run_call_callbacks'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/callbacks.rb:81:in `run_callbacks'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
[GEM_ROOT]/gems/rack-1.4.1/lib/rack/sendfile.rb:102:in `call'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_dispatch/middleware/remote_ip.rb:31:in `call'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_dispatch/middleware/debug_exceptions.rb:16:in `call'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_dispatch/middleware/show_exceptions.rb:56:in `call'
[GEM_ROOT]/gems/railties-3.2.0/lib/rails/rack/logger.rb:26:in `call_app'
[GEM_ROOT]/gems/railties-3.2.0/lib/rails/rack/logger.rb:16:in `call'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_dispatch/middleware/request_id.rb:22:in `call'
[GEM_ROOT]/gems/rack-1.4.1/lib/rack/methodoverride.rb:21:in `call'
[GEM_ROOT]/gems/rack-1.4.1/lib/rack/runtime.rb:17:in `call'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/cache/strategy/local_cache.rb:72:in `call'
[GEM_ROOT]/gems/rack-1.4.1/lib/rack/lock.rb:15:in `call'
[GEM_ROOT]/gems/rack-cache-1.1/lib/rack/cache/context.rb:132:in `forward'
[GEM_ROOT]/gems/rack-cache-1.1/lib/rack/cache/context.rb:139:in `pass'
[GEM_ROOT]/gems/rack-cache-1.1/lib/rack/cache/context.rb:151:in `invalidate'
[GEM_ROOT]/gems/rack-cache-1.1/lib/rack/cache/context.rb:70:in `call!'

I was unable to determine any resin behind this. I can provide more info about our configuration if necessary.

Regards, Yavor

jyurek commented 12 years ago

Can you turn on command logging? You can put this into config/initializers/paperclip.rb:

Paperclip.options[:log] = true
Paperclip.options[:log_command] = true

This will put the commands that Paperclip runs into the logfile. Would you please copy those here so I can see what's trying to be run? Thanks.

YavorIvanov commented 12 years ago

Sure. It would take sometime for the error to pop out again.

YavorIvanov commented 12 years ago

Don't know if this would speed up the process but here is the dev machine output when saving (there is no error introduced) just log output.

Started POST "/administration/pictures" for 127.0.0.1 at 2012-01-30 16:58:02 +0200
Processing by Administration::PicturesController#create as HTML
  Parameters: {"utf8"=>"✓", "authenticity_token"=>"WneWVliABnqQ6deE1nTp1uswMFVckWqSzkCIaT/l32A=", "picture"=>{"photo"=>#<ActionDispatch::Http::UploadedFile:0x007fbd9d25bc98 @original_filename="Unknown-6.png", @content_type="image/png", @headers="Content-Disposition: form-data; name=\"picture[photo]\"; filename=\"Unknown-6.png\"\r\nContent-Type: image/png\r\n", @tempfile=#<File:/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/RackMultipart20120130-76606-1o08m87>>, "rating_editor"=>"5", "tag_list"=>"icons", "album_ids"=>["2"], "title"=>"Many icons2 :)", "caption"=>"Youtube2 many icons sprite.", "geo_location"=>"", "copyright"=>"xenium", "original_width"=>"201", "original_height"=>"297", "crop_x"=>"", "crop_y"=>"", "crop_w"=>"", "crop_h"=>""}, "commit"=>"Save"}
  User Load (3.7ms)  SELECT `users`.* FROM `users` WHERE `users`.`id` = 1 AND `users`.`password` = 'a19ea622182c63ddc19bb22cde982b82' LIMIT 1
Command :: identify -format %wx%h '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm.png[0]'
Command :: convert '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm.png[0]' '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe.jpg'
Command :: identify -format %wx%h '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe.jpg[0]'
Command :: convert '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe.jpg[0]' -resize "50x" -crop "50x50+0+11" +repage -gravity Center -extent 50x50 '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe20120130-76606-tnkqhu.jpg'
Command :: identify -format %wx%h '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe.jpg[0]'
Command :: convert '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe.jpg[0]' -resize "110x" -crop "110x110+0+26" +repage -gravity Center -extent 110x110 '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe20120130-76606-eeg2z1.jpg'
Command :: identify -format %wx%h '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe.jpg[0]'
Command :: convert '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe.jpg[0]' -resize "150x" -crop "150x150+0+35" +repage -gravity Center -extent 150x150 '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe20120130-76606-19he83.jpg'
Command :: identify -format %wx%h '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe.jpg[0]'
Command :: convert '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe.jpg[0]' -resize "240x160^" -gravity Center -extent 240x160 '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe20120130-76606-12bs7gv.jpg'
Command :: identify -format %wx%h '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe.jpg[0]'
Command :: convert '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe.jpg[0]' -resize "312x234^" -gravity Center -extent 312x234 '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe20120130-76606-1tt4jz.jpg'
Command :: identify -format %wx%h '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe.jpg[0]'
Command :: convert '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe.jpg[0]' -resize "300x300^" -gravity Center -extent 300x300 '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe20120130-76606-1lvdrh0.jpg'
Command :: identify -format %wx%h '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe.jpg[0]'
Command :: convert '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe.jpg[0]' -resize "660x440^" -gravity Center -extent 660x440 '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe20120130-76606-7b31fi.jpg'
Command :: identify -format %wx%h '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe.jpg[0]'
Command :: convert '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe.jpg[0]' -resize "660x550" '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe20120130-76606-17ob78y.jpg'
Command :: identify -format %wx%h '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe.jpg[0]'
Command :: convert '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe.jpg[0]' -resize "960x550" '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe20120130-76606-41oh1x.jpg'
Command :: identify -format %wx%h '/var/folders/bg/w2mft0x51t933pzhgpvhm5800000gn/T/stream20120130-76606-4akqgm20120130-76606-kxnkoe.jpg[0]'
...
[paperclip] Saving attachments.
[paperclip] saving /Users/YavorIvanov/_Xenium/Development/Bulevard/public/attachments/pictures-photos/0000/0013/ORIGINAL-many-icons2.jpg
[paperclip] saving /Users/YavorIvanov/_Xenium/Development/Bulevard/public/attachments/pictures-photos/0000/0013/SMALL-many-icons2.jpg
[paperclip] saving /Users/YavorIvanov/_Xenium/Development/Bulevard/public/attachments/pictures-photos/0000/0013/THUMB_SMALL-many-icons2.jpg
[paperclip] saving /Users/YavorIvanov/_Xenium/Development/Bulevard/public/attachments/pictures-photos/0000/0013/THUMB_MEDIUM-many-icons2.jpg
[paperclip] saving /Users/YavorIvanov/_Xenium/Development/Bulevard/public/attachments/pictures-photos/0000/0013/MEDIUM-many-icons2.jpg
[paperclip] saving /Users/YavorIvanov/_Xenium/Development/Bulevard/public/attachments/pictures-photos/0000/0013/FEATURED-many-icons2.jpg
[paperclip] saving /Users/YavorIvanov/_Xenium/Development/Bulevard/public/attachments/pictures-photos/0000/0013/BANNER-many-icons2.jpg
[paperclip] saving /Users/YavorIvanov/_Xenium/Development/Bulevard/public/attachments/pictures-photos/0000/0013/HOMEPAGE-many-icons2.jpg
[paperclip] saving /Users/YavorIvanov/_Xenium/Development/Bulevard/public/attachments/pictures-photos/0000/0013/LARGE-many-icons2.jpg
[paperclip] saving /Users/YavorIvanov/_Xenium/Development/Bulevard/public/attachments/pictures-photos/0000/0013/FULLSCREEN-many-icons2.jpg
...
YavorIvanov commented 12 years ago

I decided that it might be relevant to also post the logic behind the image manipulations.

from picture.rb

Paperclip.interpolates :title_url do |attachment, style|
    attachment.instance.title_url
  end
  Paperclip.interpolates :style_upcase do |attachment, style|
    style_upcase = style.to_s.upcase
    return style_upcase
  end
  Paperclip.interpolates :directory_id do |attachment, style|
    id = (attachment.instance.id).to_s.rjust(8,"0")
    directory_id = id.scan(/..../)
    directory_id = directory_id.join('/')
    return directory_id
  end

  PAPERCLIP_PATH = ":rails_root/public/attachments/:class-:attachment/:directory_id/:style_upcase-:title_url.:extension"
  if Rails.env == "production"
    PAPERCLIP_URL =                  "/attachments/:class-:attachment/:directory_id/:style/:title_url.:extension"
  else
    PAPERCLIP_URL =                  "/attachments/:class-:attachment/:directory_id/:style_upcase-:title_url.:extension"
  end
  has_attached_file :photo,
                    :styles => {
                      :small        => { :format => 'jpg', :quality => 100, :geometry => '50x50#',   :processors => [:cropper] },
                      :thumb_small  => { :format => 'jpg', :quality => 100, :geometry => '110x110#', :processors => [:cropper] },
                      :thumb_medium => { :format => 'jpg', :quality => 100, :geometry => '150x150#', :processors => [:cropper] },
                      :medium       => { :format => 'jpg', :quality => 100, :geometry => '240x160^', :processors => [:cropper] },
                      :featured     => { :format => 'jpg', :quality => 100, :geometry => '312x234^', :processors => [:cropper] },
                      :banner       => { :format => 'jpg', :quality => 100, :geometry => '300x300^', :processors => [:cropper] },
                      :homepage     => { :format => 'jpg', :quality => 100, :geometry => '660x440^', :processors => [:cropper] },
                      :large        => { :format => 'jpg', :quality => 100, :geometry => '660x550' },
                      :fullscreen   => { :format => 'jpg', :quality => 100, :geometry => '960x550' },
                      :original     => { :format => 'jpg', :quality => 100, :geometry => '' }
                    },
                    :convert_options => {
                      :small        => "-gravity Center -extent 50x50  ",
                      :thumb_small  => "-gravity Center -extent 110x110",
                      :thumb_medium => "-gravity Center -extent 150x150",
                      :medium       => "-gravity Center -extent 240x160",
                      :featured     => "-gravity Center -extent 312x234",
                      :banner       => "-gravity Center -extent 300x300",
                      :homepage     => "-gravity Center -extent 660x440"
                    },
                    :path => PAPERCLIP_PATH,
                    :url  => PAPERCLIP_URL

from cropper.rb

module Paperclip

  class Cropper < Thumbnail
    def transformation_command
      if crop_command
        original_command = super
        if original_command.include?('-crop')
          original_command.delete_at(super.index('-crop') + 1)
          original_command.delete_at(super.index('-crop'))
        end
        if original_command.include?('-resize')
          crop_command('square') + original_command
        else
          crop_command + original_command
        end
      else
        super
      end
    end

    def crop_command(dimensions = nil)
      target = @attachment.instance
      if target.cropping?
        case dimensions
        when 'square'
          if target.crop_w > target.crop_h
            crop_w = target.crop_w.to_i
            crop_h = target.crop_w.to_i
            crop_x = target.crop_x.to_i
            crop_y = target.crop_y.to_i - ((target.crop_w.to_i-target.crop_h.to_i)/2).to_i
            crop_x = 0 if crop_x < 0
            crop_y = 0 if crop_y < 0
          elsif target.crop_w < target.crop_h
            crop_w = target.crop_h.to_i
            crop_h = target.crop_h.to_i
            crop_x = target.crop_x.to_i - ((target.crop_h.to_i-target.crop_w.to_i)/2).to_i
            crop_y = target.crop_y.to_i
            crop_x = 0 if crop_x < 0
            crop_y = 0 if crop_y < 0
          else
            crop_w = target.crop_w.to_i
            crop_h = target.crop_h.to_i
            crop_x = target.crop_x.to_i
            crop_y = target.crop_y.to_i
          end
          ["-crop", "#{crop_w}x#{crop_h}+#{crop_x}+#{crop_y}", "+repage"]
        else
          ["-crop", "#{target.crop_w.to_i}x#{target.crop_h.to_i}+#{target.crop_x.to_i}+#{target.crop_y.to_i}", "+repage"]
        end
      end
    end
  end

end
YavorIvanov commented 12 years ago

Error from AirBrake.

[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/core_ext/kernel/agnostics.rb:7:in ``'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/core_ext/kernel/agnostics.rb:7:in ``'
[GEM_ROOT]/gems/cocaine-0.2.1/lib/cocaine/command_line.rb:30:in `block in run'
[GEM_ROOT]/gems/cocaine-0.2.1/lib/cocaine/command_line.rb:51:in `with_modified_path'
[GEM_ROOT]/gems/cocaine-0.2.1/lib/cocaine/command_line.rb:28:in `run'
[GEM_ROOT]/gems/paperclip-2.5.0/lib/paperclip.rb:101:in `run'
[GEM_ROOT]/gems/paperclip-2.5.0/lib/paperclip/geometry.rb:22:in `from_file'
app/models/picture.rb:115:in `image_geometry'
app/views/shared/resources/_gallery.html.erb:3:in `_app_views_shared_resources__gallery_html_erb___2789380520415349251_23988900'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_view/template.rb:143:in `block in render'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/notifications.rb:125:in `instrument'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_view/template.rb:141:in `render'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_view/renderer/partial_renderer.rb:257:in `render_partial'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_view/renderer/partial_renderer.rb:230:in `block in render'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_view/renderer/abstract_renderer.rb:38:in `block in instrument'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/notifications.rb:123:in `block in instrument'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/notifications.rb:123:in `instrument'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_view/renderer/abstract_renderer.rb:38:in `instrument'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_view/renderer/partial_renderer.rb:229:in `render'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_view/renderer/renderer.rb:41:in `render_partial'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_view/renderer/renderer.rb:15:in `render'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_view/helpers/rendering_helper.rb:24:in `render'
app/views/pictures/show.html.erb:25:in `_app_views_pictures_show_html_erb__3678606835715178153_25414120'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_view/template.rb:143:in `block in render'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/notifications.rb:125:in `instrument'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_view/template.rb:141:in `render'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_view/renderer/template_renderer.rb:41:in `block (2 levels) in render_template'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_view/renderer/abstract_renderer.rb:38:in `block in instrument'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/notifications.rb:123:in `block in instrument'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/notifications.rb:123:in `instrument'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_view/renderer/abstract_renderer.rb:38:in `instrument'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_view/renderer/template_renderer.rb:40:in `block in render_template'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_view/renderer/template_renderer.rb:48:in `render_with_layout'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_view/renderer/template_renderer.rb:39:in `render_template'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_view/renderer/template_renderer.rb:12:in `render'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_view/renderer/renderer.rb:36:in `render_template'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_view/renderer/renderer.rb:17:in `render'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/abstract_controller/rendering.rb:109:in `_render_template'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_controller/metal/streaming.rb:225:in `_render_template'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/abstract_controller/rendering.rb:103:in `render_to_body'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_controller/metal/renderers.rb:28:in `render_to_body'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_controller/metal/compatibility.rb:50:in `render_to_body'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/abstract_controller/rendering.rb:88:in `render'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_controller/metal/rendering.rb:16:in `render'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_controller/metal/instrumentation.rb:40:in `block (2 levels) in render'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/core_ext/benchmark.rb:5:in `block in ms'
/opt/ruby-1.9.3-p0/lib/ruby/1.9.1/benchmark.rb:295:in `realtime'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/core_ext/benchmark.rb:5:in `ms'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_controller/metal/instrumentation.rb:40:in `block in render'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_controller/metal/instrumentation.rb:83:in `cleanup_view_runtime'
[GEM_ROOT]/gems/activerecord-3.2.0/lib/active_record/railties/controller_runtime.rb:24:in `cleanup_view_runtime'
[GEM_ROOT]/gems/thinking-sphinx-2.0.11/lib/thinking_sphinx/action_controller.rb:12:in `cleanup_view_runtime'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_controller/metal/instrumentation.rb:39:in `render'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_controller/metal/implicit_render.rb:10:in `default_render'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_controller/metal/implicit_render.rb:5:in `send_action'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/abstract_controller/base.rb:167:in `process_action'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_controller/metal/rendering.rb:10:in `process_action'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/abstract_controller/callbacks.rb:18:in `block in process_action'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/callbacks.rb:425:in `_run__260843282813735842__process_action__2500936749057328364__callbacks'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/callbacks.rb:405:in `__run_callback'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/callbacks.rb:385:in `_run_process_action_callbacks'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/callbacks.rb:81:in `run_callbacks'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/abstract_controller/callbacks.rb:17:in `process_action'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_controller/metal/rescue.rb:29:in `process_action'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_controller/metal/instrumentation.rb:30:in `block in process_action'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/notifications.rb:123:in `block in instrument'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
[GEM_ROOT]/gems/activesupport-3.2.0/lib/active_support/notifications.rb:123:in `instrument'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_controller/metal/instrumentation.rb:29:in `process_action'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_controller/metal/params_wrapper.rb:205:in `process_action'
[GEM_ROOT]/gems/activerecord-3.2.0/lib/active_record/railties/controller_runtime.rb:18:in `process_action'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/abstract_controller/base.rb:121:in `process'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/abstract_controller/rendering.rb:45:in `process'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_controller/metal.rb:203:in `dispatch'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_controller/metal/rack_delegation.rb:14:in `dispatch'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_controller/metal.rb:246:in `block in action'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_dispatch/routing/route_set.rb:66:in `call'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_dispatch/routing/route_set.rb:66:in `dispatch'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_dispatch/routing/route_set.rb:30:in `call'
[GEM_ROOT]/gems/journey-1.0.0/lib/journey/router.rb:60:in `block in call'
[GEM_ROOT]/gems/journey-1.0.0/lib/journey/router.rb:48:in `each'
[GEM_ROOT]/gems/journey-1.0.0/lib/journey/router.rb:48:in `call'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_dispatch/routing/route_set.rb:570:in `call'
[GEM_ROOT]/gems/omniauth-1.0.2/lib/omniauth/strategy.rb:168:in `call!'
[GEM_ROOT]/gems/omniauth-1.0.2/lib/omniauth/strategy.rb:148:in `call'
[GEM_ROOT]/gems/omniauth-1.0.2/lib/omniauth/builder.rb:42:in `call'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_dispatch/middleware/best_standards_support.rb:17:in `call'
[GEM_ROOT]/gems/rack-1.4.1/lib/rack/etag.rb:23:in `call'
[GEM_ROOT]/gems/rack-1.4.1/lib/rack/conditionalget.rb:25:in `call'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_dispatch/middleware/head.rb:14:in `call'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_dispatch/middleware/params_parser.rb:21:in `call'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_dispatch/middleware/flash.rb:242:in `call'
[GEM_ROOT]/gems/rack-1.4.1/lib/rack/session/abstract/id.rb:205:in `context'
[GEM_ROOT]/gems/rack-1.4.1/lib/rack/session/abstract/id.rb:200:in `call'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_dispatch/middleware/cookies.rb:338:in `call'
[GEM_ROOT]/gems/activerecord-3.2.0/lib/active_record/query_cache.rb:64:in `call'
[GEM_ROOT]/gems/activerecord-3.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:443:in `call'
[GEM_ROOT]/gems/actionpack-3.2.0/lib/action_dispatch/middleware/callbacks.rb:28:in `block in call'

Error from the production log.

ActionView::Template::Error (Argument list too long - identify -format %wx%h '/home/bulevard.bg/releases/20120130151749/public/attachments/pictures-photos/0000/4185/LARGE-fotosesiya-s-ema-uotsan.jpg[0]'):
    8:   <section>
    9:     <div id="picture-wrapper">
    10:       <div class="container"
    11:            style="width: <%=  @picture.image_geometry(:large).width.to_i %>px;
    12:                   height: <%=  @picture.image_geometry(:large).height.to_i %>px;">
    13:         <% for item in @items %>
    14:           <div class="item-in-picture"
  app/models/picture.rb:115:in `image_geometry'
  app/views/fashion/show.html.erb:11:in `_app_views_fashion_show_html_erb__4193646417233645110_25868920'
jyurek commented 12 years ago

So, the backtrace mentions Picture#image_geometry. What is that method?

From the looks of those other logs, the calls to identify seem fine, and not too long at all, so I really don't have a clear reason what might be happening yet.

YavorIvanov commented 12 years ago

Here is the image_geometry.

  def image_geometry(style = :original)
    @geometry ||= {}
    @geometry[style] ||= Paperclip::Geometry.from_file photo.path(style)
  end

The problem is battling me for like a month now :/ very very weird.

YavorIvanov commented 12 years ago

Actually a strange thing I remember discovering on another project of mine is where paperclip was confused where to find command path after certain "amount of stress" which sort of seems to be the case again.

Putting this in the initializers worked for that particular project to stop trolling errors when running too long without being restarted.

Paperclip.options[:command_path] = "/usr/local/bin"
YavorIvanov commented 12 years ago

Digging with our administrators we found out that a possible reason is a basic OS error "Argument list too long".

Here is some more info. http://www.gnu.org/software/coreutils/faq/#Argument-list-too-long http://www.in-ulm.de/~mascheck/various/argmax/#nature

It could be a number of things starting from unicorn and ruby itself to paperclip gem. We have tried to rise the OS limits up which is never a good thing. This should prevent the error from popping too quickly and deny access to our application. I'll try to keep you posted if I discover more on this issue.

YavorIvanov commented 12 years ago

FYI raising up the OS limits makes the problem occur much, much less. This isn't a proper fix though. We are going to try and see if the problem is somehow connected to ruby itself. More info to come.

YavorIvanov commented 12 years ago

Also this seem related https://github.com/thoughtbot/paperclip/issues/764

chrishomer commented 12 years ago

Confirmed here. I was running through about 1,000 3MB images to reprocess and generate 6 new thumbnails and at about # 800 or so the process died and experienced the behavior mentioned at the beginning of this "issue". Killed and started a new console to continue and all was fine after that.

jyurek commented 12 years ago

@chrishomer, did you happen to have :log_command turned on? I'm really interested to see what commands are trying to be executed that run afoul of this issue. None of the ones so far look even remotely long to have triggered this.

Domon commented 12 years ago

Hi, I'm having a similar problem too.

I can reproduce the problem in our application, by evaluating a line like the following in rails console:

Post.find_each { |post| post.image.reprocess! }

The error occurs when about 5000 to 7000 attached files are processed. After that, rerunning the line will give me the same error immediately (without completing any reprocessing job).

Errno::E2BIG: Argument list too long - identify -format %wx%h '/tmp/paperclip-reprocess20120517-13839-1ek979g[0]'
    from [GEM_ROOT]/gems/activesupport-3.1.0/lib/active_support/core_ext/kernel/agnostics.rb:7:in ``'
    from [GEM_ROOT]/gems/activesupport-3.1.0/lib/active_support/core_ext/kernel/agnostics.rb:7:in ``'
    from [GEM_ROOT]/gems/cocaine-0.2.1/lib/cocaine/command_line.rb:30:in `block in run'
    from [GEM_ROOT]/gems/cocaine-0.2.1/lib/cocaine/command_line.rb:51:in `with_modified_path'
    from [GEM_ROOT]/gems/cocaine-0.2.1/lib/cocaine/command_line.rb:28:in `run'
    from [GEM_ROOT]/gems/paperclip-2.6.0/lib/paperclip.rb:102:in `run'
    from [GEM_ROOT]/gems/paperclip-2.6.0/lib/paperclip/geometry.rb:22:in `from_file'
    from [GEM_ROOT]/gems/paperclip-2.6.0/lib/paperclip/thumbnail.rb:35:in `initialize'
    from [GEM_ROOT]/gems/paperclip-2.6.0/lib/paperclip/processor.rb:33:in `new'
    from [GEM_ROOT]/gems/paperclip-2.6.0/lib/paperclip/processor.rb:33:in `make'
    from [GEM_ROOT]/gems/paperclip-2.6.0/lib/paperclip/attachment.rb:441:in `block in post_process_style'
    from [GEM_ROOT]/gems/paperclip-2.6.0/lib/paperclip/attachment.rb:440:in `each'
    from [GEM_ROOT]/gems/paperclip-2.6.0/lib/paperclip/attachment.rb:440:in `inject'
    from [GEM_ROOT]/gems/paperclip-2.6.0/lib/paperclip/attachment.rb:440:in `post_process_style'
    from [GEM_ROOT]/gems/paperclip-2.6.0/lib/paperclip/attachment.rb:433:in `block in post_process_styles'
    from [GEM_ROOT]/gems/paperclip-2.6.0/lib/paperclip/attachment.rb:432:in `each'
... 7 levels...
    from [GEM_ROOT]/gems/activesupport-3.1.0/lib/active_support/callbacks.rb:81:in `run_callbacks'
    from [GEM_ROOT]/gems/paperclip-2.6.0/lib/paperclip/callback_compatibility.rb:54:in `run_paperclip_callbacks'
    from [GEM_ROOT]/gems/paperclip-2.6.0/lib/paperclip/attachment.rb:423:in `post_process'
    from [GEM_ROOT]/gems/paperclip-2.6.0/lib/paperclip/attachment.rb:330:in `reprocess!'
    from (irb):11:in `block in irb_binding'
    from [GEM_ROOT]/gems/activerecord-3.1.0/lib/active_record/relation/batches.rb:21:in `block (2 levels) in find_each'
    from [GEM_ROOT]/gems/activerecord-3.1.0/lib/active_record/relation/batches.rb:21:in `each'
    from [GEM_ROOT]/gems/activerecord-3.1.0/lib/active_record/relation/batches.rb:21:in `block in find_each'
    from [GEM_ROOT]/gems/activerecord-3.1.0/lib/active_record/relation/batches.rb:69:in `find_in_batches'
    from [GEM_ROOT]/gems/activerecord-3.1.0/lib/active_record/relation/batches.rb:20:in `find_each'
    from (irb):11
    from [GEM_ROOT]/gems/railties-3.1.0/lib/rails/commands/console.rb:45:in `start'
    from [GEM_ROOT]/gems/railties-3.1.0/lib/rails/commands/console.rb:8:in `start'
    from [GEM_ROOT]/gems/railties-3.1.0/lib/rails/commands.rb:40:in `<top (required)>'
    from script/rails:6:in `require'
    from script/rails:6:in `<main>'

The problem exists until I restart the rails console. And, of course, it can be reproduced in the new rails console.

trestles commented 12 years ago

same error here:

Errno::E2BIG: Argument list too long - convert '/var/folders/94/slp2488s6nvgg8qq0g0p5c0m0000gn/T/stream20120522-59995-851fnr.jpg[0]' -resize "200x200>" '/var/folders/94/slp2488s6nvgg8qq0g0p5c0m0000gn/T/stream20120522-59995-851fnr20120522-59995-1rqi6ns'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/activesupport-3.1.0/lib/active_support/core_ext/kernel/agnostics.rb:7:in ``'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/activesupport-3.1.0/lib/active_support/core_ext/kernel/agnostics.rb:7:in ``'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/cocaine-0.2.0/lib/cocaine/command_line.rb:30:in `block in run'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/cocaine-0.2.0/lib/cocaine/command_line.rb:51:in `with_modified_path'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/cocaine-0.2.0/lib/cocaine/command_line.rb:28:in `run'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip.rb:102:in `run'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/thumbnail.rb:77:in `make'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/processor.rb:33:in `make'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/attachment.rb:428:in `block (2 levels) in post_process_styles'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/attachment.rb:427:in `each'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/attachment.rb:427:in `inject'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/attachment.rb:427:in `block in post_process_styles'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/attachment.rb:423:in `each'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/attachment.rb:423:in `post_process_styles'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/attachment.rb:417:in `block (2 levels) in post_process'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/activesupport-3.1.0/lib/active_support/callbacks.rb:390:in `_run_asset_post_process_callbacks'

... 9 levels... from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/activerecord-3.1.0/lib/active_record/base.rb:1745:in `block in assign_attributes'

trestles commented 12 years ago

same error here:

Errno::E2BIG: Argument list too long - convert '/var/folders/94/slp2488s6nvgg8qq0g0p5c0m0000gn/T/stream20120522-59995-851fnr.jpg[0]' -resize "200x200>" '/var/folders/94/slp2488s6nvgg8qq0g0p5c0m0000gn/T/stream20120522-59995-851fnr20120522-59995-1rqi6ns'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/activesupport-3.1.0/lib/active_support/core_ext/kernel/agnostics.rb:7:in ``'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/activesupport-3.1.0/lib/active_support/core_ext/kernel/agnostics.rb:7:in ``'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/cocaine-0.2.0/lib/cocaine/command_line.rb:30:in `block in run'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/cocaine-0.2.0/lib/cocaine/command_line.rb:51:in `with_modified_path'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/cocaine-0.2.0/lib/cocaine/command_line.rb:28:in `run'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip.rb:102:in `run'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/thumbnail.rb:77:in `make'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/processor.rb:33:in `make'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/attachment.rb:428:in `block (2 levels) in post_process_styles'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/attachment.rb:427:in `each'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/attachment.rb:427:in `inject'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/attachment.rb:427:in `block in post_process_styles'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/attachment.rb:423:in `each'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/attachment.rb:423:in `post_process_styles'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/attachment.rb:417:in `block (2 levels) in post_process'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/activesupport-3.1.0/lib/active_support/callbacks.rb:390:in `_run_asset_post_process_callbacks'

... 9 levels... from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/activerecord-3.1.0/lib/active_record/base.rb:1745:in `block in assign_attributes'

trestles commented 12 years ago

same error here:

Errno::E2BIG: Argument list too long - convert '/var/folders/94/slp2488s6nvgg8qq0g0p5c0m0000gn/T/stream20120522-59995-851fnr.jpg[0]' -resize "200x200>" '/var/folders/94/slp2488s6nvgg8qq0g0p5c0m0000gn/T/stream20120522-59995-851fnr20120522-59995-1rqi6ns'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/activesupport-3.1.0/lib/active_support/core_ext/kernel/agnostics.rb:7:in ``'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/activesupport-3.1.0/lib/active_support/core_ext/kernel/agnostics.rb:7:in ``'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/cocaine-0.2.0/lib/cocaine/command_line.rb:30:in `block in run'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/cocaine-0.2.0/lib/cocaine/command_line.rb:51:in `with_modified_path'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/cocaine-0.2.0/lib/cocaine/command_line.rb:28:in `run'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip.rb:102:in `run'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/thumbnail.rb:77:in `make'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/processor.rb:33:in `make'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/attachment.rb:428:in `block (2 levels) in post_process_styles'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/attachment.rb:427:in `each'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/attachment.rb:427:in `inject'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/attachment.rb:427:in `block in post_process_styles'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/attachment.rb:423:in `each'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/attachment.rb:423:in `post_process_styles'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/attachment.rb:417:in `block (2 levels) in post_process'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/activesupport-3.1.0/lib/active_support/callbacks.rb:390:in `_run_asset_post_process_callbacks'

... 9 levels... from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/activerecord-3.1.0/lib/active_record/base.rb:1745:in `block in assign_attributes'

trestles commented 12 years ago

same error here:

Errno::E2BIG: Argument list too long - convert '/var/folders/94/slp2488s6nvgg8qq0g0p5c0m0000gn/T/stream20120522-59995-851fnr.jpg[0]' -resize "200x200>" '/var/folders/94/slp2488s6nvgg8qq0g0p5c0m0000gn/T/stream20120522-59995-851fnr20120522-59995-1rqi6ns'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/activesupport-3.1.0/lib/active_support/core_ext/kernel/agnostics.rb:7:in ``'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/activesupport-3.1.0/lib/active_support/core_ext/kernel/agnostics.rb:7:in ``'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/cocaine-0.2.0/lib/cocaine/command_line.rb:30:in `block in run'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/cocaine-0.2.0/lib/cocaine/command_line.rb:51:in `with_modified_path'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/cocaine-0.2.0/lib/cocaine/command_line.rb:28:in `run'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip.rb:102:in `run'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/thumbnail.rb:77:in `make'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/processor.rb:33:in `make'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/attachment.rb:428:in `block (2 levels) in post_process_styles'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/attachment.rb:427:in `each'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/attachment.rb:427:in `inject'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/attachment.rb:427:in `block in post_process_styles'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/attachment.rb:423:in `each'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/attachment.rb:423:in `post_process_styles'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/paperclip-2.4.5/lib/paperclip/attachment.rb:417:in `block (2 levels) in post_process'
from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/activesupport-3.1.0/lib/active_support/callbacks.rb:390:in `_run_asset_post_process_callbacks'

... 9 levels... from /Users/jt/.rvm/gems/ruby-1.9.2-p290/gems/activerecord-3.1.0/lib/active_record/base.rb:1745:in `block in assign_attributes'

geoffw8 commented 12 years ago

I seem to be having this error, is there an update? I see this was a few months ago now. We have sidekiq pumping 700 images out every few minutes...

geoffw8 commented 12 years ago

Hi - everyone on this thread - are you still having this issue? I appear to be, I'm talking to @jyurek over at Cocaine.

I just tried running 4k images (all of a similar nature, size etc etc). 2k worked fine, then inexplicably every image after that failed. Admittedly this was a different error to before (I was trying an experimental branch of Cocaine), but before I was repeatedly getting Argument list too long.

I have no idea what to do next!

Please can you guys let me know if you're still having this problem, or those of you that have perhaps moved to another solution, could you maybe let me know your general setup?

We've got a site we're literally ready to launch, but I have 50k products without images :) So desperate times!

Thanks all,

Geoff

jyurek commented 12 years ago

@trestles, can you try this again with the latest version of Cocaine? This error doesn't make sense, since we can obviously see the arguments are of a normal length. If you don't get the error with the latest version of Cocaine then we can start to pin it down a bit more.

dmitryv commented 12 years ago

Same issue. paperclip (2.7.0) cocaine (0.3.1)

dmitryv commented 12 years ago

Problem is in the PATH that gets sent to Process.spawn by Cocaine as param. I see that something keeps appending '/usr/local/bin' to it on every subsequent call. I assume that it comes from our environment configs: Paperclip.options[:command_path] = "/usr/local/bin"

Problem is here: paperclip.rb: Cocaine::CommandLine.path = ( Cocaine::CommandLine.path ? [Cocaine::CommandLine.path, command_path ].flatten : command_path )

my solution is to set Cocaine::CommandLine.path = nil after every paperclip object processed

jyurek commented 12 years ago

@dmitryv Wow, yeah, that's totally it. Honestly I didn't event expect this because I simply didn't think Process.spawn worked that way. Thanks very much for uncovering it. I'll get this one sorted ASAP.

jyurek commented 12 years ago

@dmitryv I may have spoken too soon. Both Paperclip and Cocaine handle this problem when setting the path. I confirmed that a too-large environment will trigger an E2BIG error, so there's at least that clue to work from, but I'm not convinced it's the PATHs at the moment.

FWIW, the latest version of Paperclip has this line instead of the one you quoted: Cocaine::CommandLine.path = ( Cocaine::CommandLine.path ? [Cocaine::CommandLine.path].flatten | [command_path] : command_path ) ... which does for a uniqueness check because of the | operation on the arrays.

dmitryv commented 12 years ago

Would be great to this fix it in 2.7.1, if you are planing on releasing it. Or are you saying that this is still an issue even in the latest version of paperclip?

jyurek commented 12 years ago

This doesn't seem to be an issue in the latest version, according to the test I was able to get failing in 2.7.0. I can cut a 2.7.1 to get this fixed, though. That should be done later today.

jyurek commented 12 years ago

Well, look at that, it's later today! http://rubygems.org/gems/paperclip/versions/2.7.1

@dmitryv, can you let me know if this solves your problem? It should, according to the test I got passing, so let's hope it's not something else.

jyurek commented 12 years ago

I'm reasonable sure that this problem is fixed in 2.7.1 and isn't in 3.2.0, I'm going to close this. If anyone still on this issue sees the error again, please reopen so we can get it taken care of.

Thanks all for their reports and research.

dmitryv commented 12 years ago

I've tried 2.7.1 w/o the Cocaine hack and it works just fine. Thank you for the release!

YavorIvanov commented 11 years ago

Just to confirm we have upped Paperclip every day possible and haven't ran into the problem from quite some time now.

jyurek commented 11 years ago

Thanks for the reports, guys. Sorry it took so long to fix.

geoffw8 commented 11 years ago

Hi Jon.

Sorry I haven't had the time to test this, we wrote the code in straight ruby and it works nicely. Pretty damn quick too.

Thanks for all your help on it though, I know it must have taken a while!

Thanks again

Geoff

On Tue, Oct 2, 2012 at 4:00 PM, Jon Yurek notifications@github.com wrote:

Thanks for the reports, guys. Sorry it took so long to fix.

— Reply to this email directly or view it on GitHubhttps://github.com/thoughtbot/paperclip/issues/722#issuecomment-9073914.

BR,

Geoff Wright Founder & CEO

w: weartolook.com t: @weartolook http://www.twitter.com/weartolook

weartolook LTD St John's House, 54 St John's Square, London, EC1V4JL

sylvainkalache commented 10 years ago

Hello,

It seems I have the exact same issue. I'm sending 2 files via curl using the --data option which sends content-type application/x-www-form-url encoded. I'm passing the files in base64.

If my picture and/or video is too big then I get the (Errno::E2BIG) error. If I reduce the size then it works.

I'm using: WEBrick cocaine (0.5.3) rails (4.0.2)

datenimperator commented 10 years ago

I think I ran into this today. Has this been fixed in Paperclip 4.2.0?