Dynamic content + cached mode

[iangreenleaf]

It's possible to add dynamic content to the manifest, for example:

offline = Rails::Offline.configure do
  cache "/pages/1/"
end

However, :cache => true trips over this because it expects that path to exist as a file so that it can hash the contents. I'm not entirely sure what the right solution for this is...

• Discourage people from using cached mode with dynamic content, since their browser ought to check it for updates.
• Skip hashing of anything that doesn't exist as a file, let the developer worry about forcing those updates.
• Allow the developer to provide an alternative cache "key" somewhere - either hardcoded or dynamically calculated (like pulling an updated_at timestamp).

[moll]

Have you worked on this by any chance?

I too am thinking of adding something like custom digests to change the manifest without adding actual file cache entries.

[iangreenleaf]

Nope, sorry. I ended up not needing this feature right away, and without knowing what solution @wycats would accept, I haven't been motivated to work on it.

[moll]

Okay.

I gave it a moment's thought, and invalidating the master entry cache (the HTML file itself) thoroughly ends up being a bit of a puzzle: when the HTML is composed of helpers, partials and dynamic content, the manifest generator has no way of knowing something changed without having it generated. And that kind of beats the caching part.

A decent compromise, I feel, might simply be calculating/adding the digest from a set of files that the master entry uses. You could just specify those files in the appcache generator manually. And with limiting yourself to not having dynamic content in the master entry, it's enough of cache-invalidation-automation without having to resort to manually incrementing revisions.

A syntax that I had in mind might be simply digest "app/views/foo.html.erb".

PS. We have forks exactly for cases when upstream is inactive or doesn't care. :)

[wycats]

Hey guys,

Sorry for the inactivity. I honestly don't know how to handle dynamic content... can you explain a use-case where dynamic content + cache manifest makes sense?

[iangreenleaf]

For example, say I'm building an [optionally offline] reading app. When a user clicks "Save this for later", the server sucks down the text content into a DB, and makes it available at /articles/:id.

[ptyagi16]

Store this in local storage

[andremedeiros]

The other issue here is that it won't use the generated assets' hashes when appropriate.

I propose pulling asset helpers onto the middleware so that:

1) when the file exists, use its hashed version (or whatever it is that rails does by default with its asset helpers), or 2) when it doesn't, insert it into the cache manifest as is.

[moll]

@wycats:

can you explain a use-case where dynamic content + cache manifest makes sense?

The case with common one-page webapps: render all templates out to the index and add that to the manifest. Or any other content in index that might change, like it's source file, because it's cached automatically by Appcache.

[andremedeiros]

+1 for @moll

[devinrhode2]

Could probably hack html5 offline itself and add a jsonp script to the cache manifest... but then when the cache updates the damn browser will re-request all of those, therefore... the jsonp api needs to return a 304 not modified as much as possible.

Could also use the numerous storage methods used in EverCookie

[moll]

@devinrhode2 Huh, how does adding a script URL to the manifest help invalidating the cached manifest?

[devinrhode2]

separate ideas - client side DB's for the web are basically non-existent. We have some primitive api's like localStorage. Opening a new issue regarding a better architecture for this plugin.

[devinrhode2]

This is a very hard problem. Personally, I view html5 offline mostly as a massive speed boost for web apps. Getting everything highly functional while offline is a different endeavor. Therefore, at this time, I view it as an enhancement, not a bug, and low priority.

That said, to get dynamic images and content offline, read on:

When a user clicks "Save offline" on an article, probably the best thing to do is request and save the content in localStorage. But your actual page urls are /page/1, and it's hard arbitrarily cache all /page/:id pages... therefore the url scheme page?id=1 would be better. page is offlined, and it just matters if the content for page id: 1 is stored somewhere on the client. localStorage has it's limits, but Ever cookie exposes every known client side storage, and there's one additional storage idea I have here: https://github.com/samyk/evercookie/issues/30

The browser cache and appcache could also be abused in a similar way to store dynamic images, but probably the easiest is to base64 encode the images to a string and treat them like any other data.

The appcache should primarily just be used for offline the assets and other application components, mixing in dynamic images makes things really complicated.

@moll Adding dynamic content to the manifest is abusing the manifest as a client side storage option. Other storage methods should be preferred for this.

Back to your question for invalidating the cache, obviously bytes have to change in the manifest. But how could we get the fully qualified appcache.manifest listing all the resources? When a request comes in for manifest.appcache, look at the origin, then use something like phantomJS to make a request the page of the app itself, or adopt some architecture where you can analyze /page?id=1 and know the dynamic images that will be needed in that page.

You can use PhantomJS to find all resources on a given page. There's an old project, Confess.js that can help with generating the appcache by using PhantomJS.

[moll]

I must admit I don't particularly get what you're saying, @devinrhode2, but what I meant was just caching the index page that's generated by the server and cached implicitly as it contains the manifest link. Having a way to invalidate the cache when that index template changes is what we're after.

Something in the vein of digest Rails.root.join("app/views/root/index.html.haml") would probably be a start.

[devinrhode2]

Oh, that's as easy as adding a comment like:

# host page hash: hf618n4jsd7f6zkjeqwr...

---

Also, @moll, could you tell me whether the manifest generates NETWORK / still? If so, we should file an issue to change this to * if it's just / then I think some resources on other domains can fail, but * is what should really be there.

If I remember right, this what Jake Archibald recommends in his talk on html5 offline here: http://blip.tv/jsconf/jsconf2012-jake-archibald-appcache-douchebag-6143723

[moll]

@devinrhode2 I was more thinking that this digest be calculated automatically, not added manually. ;)

I don't know about default, but on my app NETWORK is set to *.

[devinrhode2]

yeah manually would be a mess, this needs to happen automatically

-Devin http://zerply.com/DevinRhode2 http://zerply.com/devinrhode2

On Thu, Jan 3, 2013 at 7:11 PM, Andri Möll notifications@github.com wrote:

I don't know about default, but on my app http://mondayapp.com NETWORKis set to *.