Add caching to Lucky

[srcrip]

Lucky is making amazing progress but one of the biggest problems yet to be tackled is caching. I don't know if this is a problem that just gets tackled late in a web frameworks lifespan, but none of the major Crystal web frameworks have good application level caching yet. If Lucky can be the first it'll be a big win.

I suggest roughly basing Lucky's approach somewhat on the current situation in Rails 6.

Info on that: https://guides.rubyonrails.org/caching_with_rails.html

Caching methods that should be supported

• memcached (has crystal bindings)
• redis (has crystal bindings)

I think file system caches and in-database caches are so rarely used the project doesn't need them. I'd also argue that redis is more popular then memcached, so initial implementation could focus just on redis, just like the Lucky project does with postgres.

Native in-memory stores (in this case a crystal hash) are also common implementations.

These are the things you should/can cache:

Fragment cache

This is in the view layer. They'd be on pages in Crystal.

They look like this in rails:

<% @products.each do |product| %>
  <% cache product do %>
    <%= render product %>
  <% end %>
<% end %>

So in Lucky I'd expect a syntax like:

def render
  products.each do |product|
    cache product do
      render_product
    end
  end
end

This would cache the result of the HTML block based on whether or not the updated_at timestamp of the model has changed.

Low level cache

This example from the Rails docs explains it mostly:

 cache_key = "products_v1"

 def competing_price
    Rails.cache.fetch("#{cache_key}/competing_price", expires_in: 12.hours) do
      // Do something with model
    end
  end

This syntax is just to give you granular control over how you expire caches.

SQL cache

This is built in to active record, is it built in to Avram?

[paulcsmith]

Great suggestions! Luckily with Lucky we would only need the "low-level" cache. Since Lucky pages are just regular Crystal you can use the same syntax in your pages.

For SQL, we have an issue open for that in Avram so it has a per-request query cache: https://github.com/luckyframework/avram/issues/63

I think redis, and in-memory would be great to start. But we can set it up so people can make their own adapters for whatever else they want (like postgres, memcached, etc.)

[jwoertink]

There are only two hard things in Computer Science: cache invalidation and naming things. -- Phil Karlton

So maybe just for an extra level of difficulty, we make this thing a separate shard.... What do we call it? 🤯

[paulcsmith]

Yeah separate shard makes sense! It doesn't really need anything for Lucky to use it.

My thought is: CacheMoney

[jwoertink]

At my work, anytime someone writes "cache" in slack, a slack bot will automatically return a gif with money 🤑

[srcrip]

I definitely agree, there should be an implementation of the caching methods themselves that someone can plug different backends into (redis, memcached, whatever).

I would love to try and put together some initial attempts, but I'm still learning a lot about Lucky's internals. The implementation is rather simple, you just need to get the HTML rendered from a block in Lucky's DSL, and do some checking about whether its in the cache or not, if not put it in, if so take it out, etc., but I don't quite see yet how to actually grab the result of the HTML helpers.

I see the tags push info into a 'view' var but how do you get the result of just one block of tags?

    def {{method_name.id}}(options = EMPTY_HTML_ATTRS, **other_options, &block) : Nil
      merged_options = merge_options(other_options, options)
      tag_attrs = build_tag_attrs(merged_options)
      view << "<{{tag.id}}" << tag_attrs << ">"
      check_tag_content!(yield)
      view << "</{{tag.id}}>"
    end

    def {{method_name.id}}(attrs : Array(Symbol), options = EMPTY_HTML_ATTRS, **other_options, &block) : Nil
      boolean_attrs = build_boolean_attrs(attrs)
      merged_options = merge_options(other_options, options)
      tag_attrs = build_tag_attrs(merged_options)
      view << "<{{tag.id}}" << tag_attrs << boolean_attrs << ">"
      check_tag_content!(yield)
      view << "</{{tag.id}}>"
    end

I feel like these methods could be edited to return their html instead, does that make sense? would that cause problems? What might that look like?

EDIT

An idea on that front I had is adding this to the base tag class:

  @html = IO::Memory.new
  getter html

    // ... macro block starts

    def {{method_name.id}}!(*args) : IO::Memory
      {{method_name.id}}(*args)
      @html
    end

    def {{method_name.id}}!(&block) : IO::Memory
      {{method_name.id}}(EMPTY_HTML_ATTRS) do
        yield
      end
      @html
    end

This way you can have a variant of every tag with an ! at the end of the method name that will return a buffer with the html. Then it can be used for caching. Does this make sense or is this a bad approach? There are so many nil return declarations in the macro blocks I thought it would involve changing a lot to change it to returning an IO:Memory... This does have the problem of then every cached tag would need to use this bang syntax. Does anyone have a better plan?

EDIT 2

Actually better yet, this strategy can be used but instead of the bang methods there could be a cache method that sets a boolean that tells the methods to inject their html the @html (or call it @cache) var. Or passes said flag via arguments or something.

EDIT 3

Actually this is way simpler:

  @cache = false
  @html = IO::Memory.new
  getter html
  getter cache

  def cache(&block) : IO::Memory
    @cache = true
    yield
    @cache = false
    @html
  end

  // Followed by some changes to the tag macros:

    def {{method_name.id}}(options = EMPTY_HTML_ATTRS, **other_options, &block) : Nil
      merged_options = merge_options(other_options, options)
      tag_attrs = build_tag_attrs(merged_options)

      view << "<{{tag.id}}" << tag_attrs << ">"
      @html << "<{{tag.id}}" << tag_attrs << ">" if @cache
      check_tag_content!(yield)
      view << "</{{tag.id}}>"
      @html << "</{{tag.id}}>" if @cache
    end

  def text(content : String | Lucky::AllowedInTags) : Nil
    plain_text = HTML.escape(content.to_s)
    @html << plain_text if @cache
    view << plain_text
  end

Then if you do:

    ex = cache do
      div do
        text "hi"
      end
    end

    pp ex.to_s

You should get: "<div>hi</div>"

Then this can be changed to instead of returning an IO:Memory, it could pass it along to the caching method internals in another file.

[srcrip]

I have a good bit of stuff working that I can open a PR for soon.

I also found this, which is awesome: https://github.com/crystal-community/kiwi/blob/master/README.md

It's an API to basically every major key value store out there. If we leveraged this there'd be no major work to support multiple kinds of backend stores.

[srcrip]

Just an update, I'm getting really good time savings even with the tiniest of cached html. I would post some numbers but I also get a lot of fluctuations on page load of around ~10ms on a simple page, so it's hard to say, but there's definitely some good time savings.

[jwoertink]

I wonder if there's a way we can do this without modifying any of the HTML methods 🤔

[srcrip]

I think the answer is no... The tag methods just append strings onto a view variable on every page, so there's no way to get more granular bits of html out of the page.

I wanted to change the tags from returning nil to returning their contents as a string, as that would mean the actual cache implementation then didn't even need to be in the tags file at all, but that would involve changing a lot more. If that's acceptable to the project I think that implementation would be cleaner though.

But otherwise yeah, I don't see any other way to get more granular bits of the HTML out of a page as it stands right now without making some changes, unless someone knows something I don't about how it works.

[paulcsmith]

Here is what I think could work that would not require changing the HTML methods

def cache(key)
  cached_html = get_cached_html(key)
  if cached_html
    raw(cached_html)
  else
    original_view = @view
    # Temporarily override the view
    @view = IO::Memory.new

    yield # This will write to our temporary @view object

    html_fragment = @view.to_s
    save_cached_html(key, html_fragment)
    # Set instance var back to original view
    @view = original_view
    # Write fragment to original view
    raw(html_fragment)
  end
end

# in a page
cache("user-row-#{@user.id}) do
  text @user.name
end

I think this should work and keep the HTML tags untouched

[srcrip]

Thanks @paulcsmith, I'll try that. Unfortunately my computer crashed and I hadn't uploaded any of my changes yet so it'll be a bit delayed :(

[paulcsmith]

Oh no! Good luck getting everything set up again @Sevensidedmarble

[matthewmcgarvey]

This library looks like it does all the integration and base interface for us https://github.com/mamantoha/cache

[matthewmcgarvey]

Being able to generate a cache key with Avram::Model's seems like it would be a normal want with this and I've been looking into how Rails' does it and it turns out to be fairly complicated. Here's a blog post about it https://bigbinary.com/blog/activerecord-relation-cache-key

[matthewmcgarvey]

I made this repo https://github.com/matthewmcgarvey/lucky_cache

It integrates the cache shard with our HTML code.

I feel like it really needs to be able to convert Avram::Models into cache keys but looking at how Rails does it seems fairly complicated.

[srcrip]

https://github.com/crystal-community/kiwi Is another one I would recommend trying out. It had support for googles very performant LevelDB. (https://github.com/google/leveldb)

[jwoertink]

This is done now with https://github.com/luckyframework/lucky_cache/ being added in. We still need to document how to use it https://github.com/luckyframework/website/issues/894 but we at least have something started. We can now start building it up, and fix issues along the way.

I'm going to close this, but if anyone feels there's more we can do related, we can always re-open.