nesquena / rabl

General ruby templating with json, bson, xml, plist and msgpack support
http://blog.codepath.com/2011/06/27/building-a-platform-api-on-rails/
MIT License
3.65k stars 335 forks source link

File name too long #412

Open fermuch opened 11 years ago

fermuch commented 11 years ago

When using caching, I get the following error:

ActionView::Template::Error File name too long

Maybe the filename should be in sha1 or md5.

nesquena commented 11 years ago

What do you think @databyte

GarPit commented 11 years ago

+1. It happens on Heroku

databyte commented 11 years ago

Do you have a sample app that shows this? What's the name of the file? What's the name, value and timestamp of the object?

GarPit commented 11 years ago

It's like this:

ActionView::Template::Error: File name too long - /app/tmp/cache/B16/0E9/rabl%2Fusers%2F3305-20130607225823%2Fusers%2F4802-20130610105153%2Fusers%2F2356-20130606123554%2Fusers%2F2350-20130606123554%2Fusers%2F2268-20130606123554%2Fusers%2F2301-20130606123554%2Fusers%2F16-20130606123554%2Fusers%2F30-2013/0606123554%2Fusers%2F75-20130606123554%2Fusers%2F101-20130606123554%2Fusers%2F233-20130606123555%2Fusers%2F261-20130606123555%2Fusers%2F300-20130606123555%2Fusers%2F333-20130606123555%2Fusers%2F370-20130606123555%2Fusers%2F401-201/30606123555%2Fusers%2F749-20130606123556%2Fusers%2F822-20130606123556%2Fusers%2F918-20130606123556%2Fusers%2F1049-20130606123557%2Fusers%2F1126-20130606123557%2Fusers%2F1258-20130606123558%2Fusers%2F1410-20130606123558%2Fusers%2F1/442-20130606123558%2Fusers%2F1602-20130606123559%2Fusers%2F1603-20130606123559%2Fusers%2F1667-20130606123559%2Fusers%2F1731-20130606123559%2Fusers%2F2304-20130606123554%2Fusers%2F1743-20130606123559%2Fusers%2F1830-20130606123559%2/Fusers%2F1926-20130606123600%2Fusers%2F1989-20130606123600%2Fusers%2F2054-20130606123600%2Fusers%2F2120-20130606123600%2Fusers%2F2152-20130606123600%2Fusers%2F2203-20130606123600%2Fusers%2F2296-20130606123601%2Fusers%2F2391-201306/06123601%2Fusers%2F2451-20130606123601%2Fusers%2F2462-20130606123601%2Fusers%2F2392-20130606123601%2Fusers%2F2426-20130606123601%2Fusers%2F2661-20130606123601%2Fusers%2F3217-20130606123601%2Fusers%2F2669-20130606123601%2Fusers%2F2/638-20130606123601%2Fusers%2F2646-20130606123601%2Fusers%2F3218-20130606123601%2Fusers%2F2653-20130606123601%2Fusers%2F2654-20130606123601%2Fusers%2F2640-20130606123601%2Fusers%2F3224-20130606123601%2Fusers%2F2645-20130606123601%2/Fusers%2F3670-20130606123601%2Fusers%2F2678-20130606123601%2Fusers%2F4183-20130606123601%2Fusers%2F2648-20130606123601%2Fusers%2F2664-20130606123601%2Fusers%2F2668-20130606123601%2Fusers%2F2651-20130606123601%2Fusers%2F2681-201306/06123601%2Fusers%2F3226-20130606123601%2Fusers%2F3227-20130606123601%2Fusers%2F3228-20130606123601%2Fusers%2F3671-20130606123601%2Fusers%2F3229-20130606123601%2Fusers%2F3678-20130606123601%2Fusers%2F2464-20130606123601%2Fusers%2F3/244-20130606123601%2Fusers%2F2685-20130606123601%2Fusers%2F3253-20130606123601%2Fusers%2F2710-20130606123601%2Fusers%2F2696-20130606123601%2Fusers%2F2698-20130606123601%2Fusers%2F2699-20130606123601%2Fusers%2F2700-20130606123601%2/Fusers%2F3675-20130606123601%2Fusers%2F3232-20130606123602%2Fusers%2F2688-20130606123602%2Fusers%2F3242-20130606123602%2Fusers%2F3267-20130606123602%2Fusers%2F2702-20130606123602%2Fusers%2F2703-20130606123602%2Fusers%2F3273-201306/06123602%2Fusers%2F3256-20130606123602%2Fusers%2F2684-20130606123602%2Fusers%2F2705-20130606123602%2Fusers%2F2709-20130606123602%2Fusers%2F2758-20130606123602%2Fusers%2F3682-20130606123602%2Fusers%2F2763-20130606123602%2Fusers%2F2/358-20130606123602%2Fusers%2F4185-20130606123602%2Fusers%2F3683-20130606123602%2Fusers%2F2790-20130606123602%2Fusers%2F2796-20130606123602%2Fusers%2F3684-20130606123602%2Fusers%2F2797-20130606123602%2Fusers%2F2798-20130606123602%2/Fusers%2F2793-20130606123602%2Fusers%2F3688-20130606123602%2Fusers%2F2765-20130606123602%2Fusers%2F2771-20130606123602%2Fusers%2F3690-20130606123602%2Fusers%2F2836-20130606123602%2Fusers%2F2772-20130606123602%2Fusers%2F2815-201306/06123602%2Fusers%2F2818-20130606123602%2Fusers%2F2773-20130606123602%2Fusers%2F2774-20130606123602%2Fusers%2F3689-20130606123602%2Fusers%2F2839-20130606123602%2Fusers%2F2823-20130606123602%2Fusers%2F2824-20130606123602%2Fusers%2F3/691-20130606123602%2Fusers%2F2828-20130606123602%2Fusers%2F2833-20130606123602%2Fusers%2F3702-20130606123602%2Fusers%2F2775-20130606123602%2Fusers%2F3693-20130606123602%2Fusers%2F2817-20130606123602%2Fusers%2F2776-20130606123602%2/Fusers%2F2777-20130606123602%2Fusers%2F2855-20130606123602%2Fusers%2F2851-20130606123602%2Fusers%2F2856-20130606123602%2Fusers%2F2853-20130606123602%2Fusers%2F2866-20130606123602%2Fusers%2F2854-20130606123602%2Fusers%2F2864-201306/06123602%2Fusers%2F2881-20130606123602%2Fusers%2F3706-20130606123602%2Fusers%2F2779-20130606123602%2Fusers%2F2918-20130606123
GarPit commented 11 years ago

Stack trace:

 /app/vendor/ruby-2.0.0/lib/ruby/2.0.0/fileutils.rb: 245:in `mkdir'
 /app/vendor/ruby-2.0.0/lib/ruby/2.0.0/fileutils.rb: 245:in `fu_mkdir'
 /app/vendor/ruby-2.0.0/lib/ruby/2.0.0/fileutils.rb: 219:in `block (2 levels) in mkdir_p'
 /app/vendor/ruby-2.0.0/lib/ruby/2.0.0/fileutils.rb: 217:in `reverse_each'
 /app/vendor/ruby-2.0.0/lib/ruby/2.0.0/fileutils.rb: 217:in `block in mkdir_p'
 /app/vendor/ruby-2.0.0/lib/ruby/2.0.0/fileutils.rb: 203:in `each'
 /app/vendor/ruby-2.0.0/lib/ruby/2.0.0/fileutils.rb: 203:in `mkdir_p'
…esupport-3.2.13/lib/active_support/cache/file_store.rb: 161:in `ensure_cache_path'
…esupport-3.2.13/lib/active_support/cache/file_store.rb:  90:in `write_entry'
….2.13/lib/active_support/cache/strategy/local_cache.rb: 140:in `write_entry'
…/gems/activesupport-3.2.13/lib/active_support/cache.rb: 364:in `block in write'
…/gems/activesupport-3.2.13/lib/active_support/cache.rb: 520:in `instrument'
…/gems/activesupport-3.2.13/lib/active_support/cache.rb: 362:in `write'
…/gems/activesupport-3.2.13/lib/active_support/cache.rb: 299:in `fetch'
…le/ruby/2.0.0/gems/rabl-0.7.2/lib/rabl/cache_engine.rb:  18:in `fetch'
andrewhubbs commented 11 years ago

I've had this happen to. It is basically from setting a cache key that includes a collection of objects.

cache ["foo", @some_big_collection]

The solution I have usually done is to just hash it.

cache ["foo", Digest::MD5.hexdigest(@some_big_collection.map(&:cache_key))]
GarPit commented 11 years ago

Will try it, thanks!

GarPit commented 11 years ago

Should I invalidate such cache manually?

andrewhubbs commented 11 years ago

I don't think you should need to because you are generating the key from the other keys. If those keys are automatically invalidated, so too should the hash key.

On Wed, Aug 14, 2013 at 4:34 AM, PitOn notifications@github.com wrote:

Should I invalidate such cache manually?

— Reply to this email directly or view it on GitHubhttps://github.com/nesquena/rabl/issues/412#issuecomment-22629513 .

Cheers

Andrew Hubbs andrew@hub.bs @andrewhubbs https://twitter.com/#!/andrewhubbs

cthielen commented 11 years ago

Seeing this too while trying to cache a rather large collection.

rpassis commented 10 years ago

I'm experiencing the same issue in my development machine (OSX). I have a collection of @horses (60 items) and my rabl template includes a cache @horses which triggers the error described below. I have also included a screenshot to help describe the problem.

It seems that there's something suss with the cache key and there are several subfolders being created within the tmp/cache folder. I have included a screenshot of the directory structure to try and illustrate the issue but from what I can see each of the subfolders is 228 characters long and a child folder name is the "continuation" of the parent's folder name.

As an example, the parent folder... rabl%2Fhorses%2F1-20140822104632326582000%2Fhorses%2F2-20140822104632334029000%2Fhorses%2F3-20140822104632342018000%2Fhorses%2F4-20140822104632348698000%2Fhorses%2F5-20140822104632356796000%2Fhorses%2F6-20140822104632363303000%2

... has a child folder... Fhorses%2F7-20140822104632370048000%2Fhorses%2F8-20140822104632376589000%2Fhorses%2F9-20140822104632384375000%2Fhorses%2F10-20140822104632390739000%2Fhorses%2F11-20140822104632398727000%2Fhorses%2F12-20140822104632405257000%2Fho

... which in turn has a child folder... rses%2F13-20140822104632413365000%2Fhorses%2F14-20140822104632419554000%2Fhorses%2F15-20140822104632427381000%2Fhorses%2F16-20140822104632434277000%2Fhorses%2F17-20140822104632441393000%2Fhorses%2F18-20140822104632448239000%2Fho

screen shot 2014-08-24 at 9 51 55 pm

This keeps going until the OS reaches the maximum allocate space for filenames with the error message File name too long @ dir_s_mkdir...

rpassis commented 10 years ago

I should also mention that the solution proposed by @andrewhubbs above works around the problem but needs a small modification to convert the @horses.map enumerator into a strong before passing it to create the MD5 hash.

cache ["foo", Digest::MD5.hexdigest(@large_collection.map(&:cache_key).to_s)]