Pass in context to lambda functions?

[dswitzer]

Right now the lambda function only retrieve the template code as an argument.

However, I started thinking that it would make some sense to pass in a reference to the "context" item as an argument as well, so that you could potentially cache the results of the function call by replacing the context on-the-fly.

This obviously doesn't always make sense, but I have some situations where I have some variables I want to do as a lambda because the lookup expense is costly and they're not always used. By passing in a reference to the context item, I could change the context to point to the rendered string (instead of the function.)

Does anyone else think it's worth adding?

Here's a little proof of concept showing the basic idea would work:

// function to show off lambda
function cacheTest(context, item){
    var results = createUUID();
    // here we cache the results in the original context object
    context[item] = results;
    return results;
}

// a sample context object
context = {test=cacheTest};

for( i=1; i <= 10; i=i+1 ){
    // this logic is handled by the get() method
    if( isCustomFunction(context.test) ){
        x = context.test(context, "test");
    } else {
        x = context.test;
    }
    // show the results
    writeOutput("<div>#i# = #x#</div>");
}

[pmcelhaney]

There's actually a spec that defines how lambdas are supposed to work.

Lambdas are a special-cased data type for use in interpolations and sections.

When used as the data value for an Interpolation tag, the lambda MUST be treatable as an arity 0 function, and invoked as such. The returned value MUST be rendered against the default delimiters, then interpolated in place of the lambda.

When used as the data value for a Section tag, the lambda MUST be treatable as an arity 1 function, and invoked as such (passing a String containing the unprocessed section contents). The returned value MUST be rendered against the current delimiters, then interpolated in place of the section.

[dswitzer]

Thanks. I missed that. However, we already technically break that spec since we always pass in the template argument.

Part of the problem is other languages w/closures have ways to work around problems, that we can't easily deal with in ColdFusion.

For example, the problem I'm trying to solve is to provide a way to only process a value if it's actually used in a template--since the initialization cost is high.

In JS, I could do something like:

function proxy(s){
  return function (){ return s.toUpperCase() };
}

var context = {
  lambda: proxy('world')
};

Now if I call {{lamba}} in my template, the value would be "WORLD". While this is a super simple example, this is really handy way to call an expensive function only when it's used.

NOTE: Passing in a value and returning a function is necessary in my case, because my function needs to run w/some arguments.

ColdFusion 8 lacks this kind of facility, so I'm wondering if it doesn't make sense to diverge a bit from the spec. If a lambda also received the following arguments:

• Current Context (branch in a nested object)
• Current Key (so that you could replace the lambda)
• Root Context

You'd be able to work around the lack of closures. Of course it's completely up to your lambda functions to use these functions.

Maybe this is a bad idea. I'm just trying to think of ways where I can delay execution of expensive operations and then only execute the logic once. I'm also trying to do this all with the context of a singleton. If I didn't need a singleton, things would be easier.

I'm certainly open to ideas on the best way to implement a delayed evaluation. Any ideas that work in CF8?

[dswitzer]

Also, my initial idea was to add an onGetNotExists() event method that would be called when the get() method fails to get a valud. This method would return an empty string by default. This would keep the spec intact.

However, you could extend Mustache.cfc and write your own onGetNotExists() function which could be used for delayed evaluation.

The method would be something like:

function onGetNotExists(context, item, contextPath){
  // context = current context of get() operation
  // item = current key in context
  // contextPath = the full dot notation path (i.e. root.childA.childB)

  if( contextPath == "my.path" ){
    // cache the results
    context[item] = myExpensiveOperation();
    // return the real value
    return context[item];
  } 

  // return an empty string
  return "";
}

This obviously breaks the spec, but then it's up to the developer to break things by extending the base Mustache object.

I just thought maybe this was more hacking than passing a few more arguments to the lamba functions.

[pmcelhaney]

The current context should already be available to the "lambda" function. The function is a member of that context, so the current context is this. That should be enough. What you're suggesting (two posts up) implies that the context object knows that it's being used in a Mustache template, which defeats the purpose of keeping models and views separate.

As for memoization, that's possible in ColdFusion. It's just not as pretty.

<cfcomponent name="chessGame">

   <cfset this.positions = "an array showing where all of the pieces are">

   <cffunction name="movesUntilCheckmate" access="public">
       <cfif not structKeyExists(variables, 'cachedMovesUntilCheckmate')>
          <cfset variables.cachedMovesUntilCheckmate = calculateMovesUntilCheckmate()>
       </cfif>
       <cfreturn variables.movesUntilCheckmate>
   </cffunction>

   <cffunction name="calculateMovesUntilCheckmate" access="private">
      <!---
        Expensive algorithm that evaluates this.positions.
      --->
   </cffunction>

</cfcomponent>

[dswitzer]

@pmcelhaney

I'm currently trying to plug this into some code that using structures, so I'm actually using structures and not objects for the context. The current code doesn't have an object layer for the stuff I'm trying to use in the templates, so I was trying to work with what's there.

Maybe I'll just need to re-work the code to use an object for the context instead of a struct.

[markmandel]

Yeah, I have the same issue, as I'm using a structure as well. No access to the context, or to the rendering function.

Technically, passing in the conext is against the mustache spec.

This was why I broke our renderLambda() was so I could extend Mustache.cfc, and have a version that passed in the context and mustache itself.

[pmcelhaney]

Interesting. But I'm confused. If you're passing a struct, where does the function come in? Does the struct contain a pointer to a function? (Can you do that in ColdFusion?)

[dswitzer]

@pmcelhaney

A struct can contain a pointer to a function, and that's what I've been doing. Glad to hear I'm not the only one using structs as context.

Part of the issue I have w/how extend an object works is that any of the public Mustache methods end up being available as a lambda--which can cause issues.

For example, if a Mustache template would end up including "{{init}}" in the source, then a CF error is generated. While this isn't an issue for templates built by developers, it is an issue if you're allowing users to generate templates (which is our plan.)

I'm going to open up an issue for this.

What I've been experimenting with is using a private "Context" variable instead of public members to store my context--that way nothing leaks into the context.

[pmcelhaney]

In that case, I'm not opposed to passing the context as an argument when the context is a struct. Would that work?

[pmcelhaney]

BTW, my gut tells me that if you don't want to expose certain methods of your object, it would be better to create a wrapper/facade component. That way the interface you expose to Mustache templates is explicitly defined and won't change every time you modify or refactor the underlying object.

[dswitzer]

@pmcelhaney

What I played around w/yesterday was putting the "context" in a private "context" variable (which could also be public if you wanted.) This means my context was protected from public methods, but does allow me to use a pointer in my context to point to a function in the component.

As to passing in the context to a lambda, IMO, we need these 3 pieces of information (as stated above):

• Current Context (branch in a nested object)
• Current Key (so that you could replace the lambda)
• Root Context (the root context object)

You need the current context and the key so that you can override the results w/a cached value. The root context is handy for referencing other variables in the context that might be of need (but that's really only needed if you're using dot notation and deeply nested structs--which I am because I like the namespacing.)

Since Mark's already overwriting the renderLamba function because he needs similar functionality, maybe the better option would be do change the renderLambda() function to something like:

<cffunction name="renderLambda" access="private" output="false"
    hint="render a lambda function (also provides a hook if you want to extend how lambdas works)">
    <cfargument name="tagName"/>
    <cfargument name="template" />
    <cfargument name="context" />
    <cfargument name="partials"/>

    <cfset var local = {} />
    <cfset local.args = getLambdaArguments(argumentCollection=arguments) />

    <!--- if running on a component --->
    <cfif isObject(arguments.context)>
        <!--- call the function and pass in the arguments --->
        <cfinvoke component="#arguments.context#" method="#arguments.tagName#" argumentCollection="#local.args#" returnvariable="local.results" />
    <!--- otherwise we have a struct w/a reference to a function or closure --->
    <cfelse>
        <cfset local.fn = arguments.context[arguments.tagName] />
        <cfset local.results = local.fn(argumentCollection=local.args) />
    </cfif>

    <cfreturn local.results />
</cffunction>

<cffunction name="getLambdaArguments" access="private" output="false"
    hint="generates the arguments to pass to the lambda function/method">

    <cfset var results = {} />

    <cfif structKeyExists(arguments, "template")>
        <cfset results[1] = arguments.template />
    </cfif>

    <cfreturn results />
</cffunction>

The problem is CF8 does not allow unnamed arguments in the argumentCollection, so this line fails:

    <cfset local.results = local.fn(argumentCollection=local.args) />

The benefit to this kind of approach is we could fix the issue where lambda's referenced as variables don't pass in the "template" argument--which would make it conform to the spec--and we could easily overwrite just the getLambdaArguments if we wanted to extend what's passed to the lambda methods.

However, since the unnamed arguments doesn't work in CF8.01, we'd need to figure out a work around for the line above which fails. We could do something like this:

    <cfset local.argCount = structCount(local.args) />
    <cfif local.argCount eq 0>
        <cfset local.results = local.fn() />
    <cfelseif local.argCount eq 1>
        <cfset local.results = local.fn(local.args[1]) />
    <cfelseif local.argCount eq 2>
        <cfset local.results = local.fn(local.args[1], local.args[2]) />
    </cfif>

We could either cap the supported arguments at 2 (where you'd just use the 2nd argument to pass in a complex value of all other values you needed) or repeat this logic up to like 4 arguments (which is all the arguments passed in.)

If the unnamed argument collection worked, I would say this is a very nice way to add an easy hook into extending Mustache, without having to override the entire renderLambda() function. I never like overriding core functions, because if bug fixes are applied, then you have to make sure to fix the bugs in your overridden functions. Any time we can extend the API to put in extension hooks I think is ideal.

[pmcelhaney]

I don't think that replacing the lambda with a cached value is a good idea. Again, it's directly contrary to the spec.

 name: Interpolation - Multiple Calls
    desc: Interpolated lambdas should not be cached.
    data:
      lambda: !code
        ruby:    'proc { $calls ||= 0; $calls += 1 }'
        perl:    'sub { no strict; $calls += 1 }'
        js:      'function() { return (g=(function(){return this})()).calls=(g.calls||0)+1 }'
        php:     'global $calls; return ++$calls;'
        python:  'lambda: globals().update(calls=globals().get("calls",0)+1) or calls'
        clojure: '(def g (atom 0)) (fn [] (swap! g inc))'
    template: '{{lambda}} == {{{lambda}}} == {{lambda}}'
    expected: '1 == 2 == 3'

If you want to cache the calculation, do it within the lambda.

<cffunction name="expensiveCalculation">
    <cfargument name="context"> <!-- this should be the struct that has expensiveCalcuation as a member --->
    <cfif not structKeyExists(context, 'expensiveCalculation_result')>
       ...
       <cfset context.expensiveCalculation_result = ... >
    </cfif>
    <cfreturn context.expensiveCalculation_result>
</cffunction>

[dswitzer]

@pmcelhaney

I understand you think it's a bad idea--you don't need to do it. :) That's why I'm not suggestion we add support to the native library, just make changes that allow a developer to do what they feel is right for their project.

The problem with your solution is based on using components for context--not structures. When using a structure, a lambda might be global so doing that kind of caching could have scoping issues.

Also, my interpretation of the spec is that the Mustache engine shouldn't cache the results--which isn't what I'm suggestion--which I'd agree with. If you look at the same code, the function is returning a different value on each iteration.

So, I think it's completely reasonable that the developer is allowed to make the lambda function do whatever they need. If we had closures in CF8, this would be a moot point. Other languages have ways to deal with this kind of dynamic runtime caching. I'm just trying to suggest a way it can be done with CF8, one that doesn't break the core implementation but gives a developer the ability to override if they so choose.

[pmcelhaney]

Sure, it's open source so we're free to disagree and experiment with different ideas. :) I'm actually not even using ColdFusion at this point -- just trying to give you some guidance based on my experience.

The solution I just posted assumes you're calling mustache.render(template, someStruct), where one of the keys of someStruct points to the expensiveCalculation function.

I agree that function can do whatever it wants. I'm only concerned about what Mustache does:

1. Mustache shouldn't pass anything to the function other than the section contents and the current context. (The latter technically violates the current spec, but it's within the spirit of the spec and we could make a case for having it added as an option.)
2. The function is called every time. That means we don't replace the function with a static value.

[dswitzer]

@pmcelhaney

As I stated, I'm not suggestion Mustache.cfc break from the spec. However, I do think developers should have the ability to easily work w/Mustache, even if what they do may end up being a break from the spec.

You and I interpret the lambda cache spec thing a little differently. I read it that the Mustache engine shouldn't cache the results, not that the developer can't have the lambda return a cached result (or in my case, overwrite the value.)

I totally agree that the engine shouldn't make assumptions about caching, etc. However, I think it's reasonable that a developer is able to make the context object work in way that's most efficient to them. As a matter of fact there's very legitimate reasons why a lambda call might need to return the same value for every call w/in the current context. There's just not a good way to manage that using a structure as input--unless you overwrite the value.