Add template functions: intersect and in

[swill]

Here are the functions I am interested in: http://play.golang.org/p/G0xhxh65hT

The main use case for this is to be able to do a block on the side of similar posts for ALL of the tags. Yes, you can do it for one tag, but with the current functions available it is not possible to show all the pages which have at least one tag similar to one of the tags in the current post.

With the 'intersect' function we could easily loop through the posts once and compare the tags and show the link if the intersect has a length greater than 0.

For example:

{{ $tags := .Params.tags }}
{{ range .Site.Recent }}
    {{ intersect $tags .Params.tags | len | lt 0 | if }}
        <li><a href="{{ $page.Permalink }}">{{ $page.Title }}</a></li>
    {{ end }}
{{ end }}

I tried to expand the functionality in the hugo/hugolib/template.go file to add this functionality, but I am getting template errors when I try to use the new functions. Unfortunately even with the '-v' flag when running the server, there is absolutely no help when you get template errors. It basically just gives you a 'ERROR: 2014/09/26 Rendering error: html/template: "blog/single.html" is an incomplete template' and gives you no ACTUAL error as to why it is not working. This is VERY frustrating and should probably be improved as well to give better error messages if there are issues with the template.

I am a novice at golang, so it is possible I am making a rookie mistake, but I can't tell because I can't figure out what the error I am getting is. I would give you a pull request for the change if I could figure out why it is failing...

[tatsushid]

I agree with you it is difficult to find out what a problem with such an insufficient error message but it is generated by Go html/template official package, not by Hugo so it is hard to improve.

In this case, I think that if placed at the last of the pipes causes an error because if is an action, not a function (Please see http://golang.org/pkg/text/template/). I think this should be

{{ $tags := .Params.tags }}
{{ range .Site.Recent }}
    {{ if intersect $tags .Params.tags | len | lt 0 }}
        <li><a href="{{ $page.Permalink }}">{{ $page.Title }}</a></li>
    {{ end }}
{{ end }}

I also modify your Go playground example a little: http://play.golang.org/p/6etSInJlTS. .Params.tags is implemented as map[string]interface{} and .Params.tags means map["tags"] so the arguments should be interface{} type and are needed to use Type Assertion to get []string.

The second return value which must be an error type is important. When something problem occurs in the function, it stops template processing and tells users the error detail. To reduce our frustration written above, I think a function should be return an error.

If you want to use the intersection function with other type arrays, for example, int, uint, float etc, it might be a good idea to use reflection in it. Other functions like first, where etc. may be examples how to use it with Hugo data.

[swill]

Great, thank you for the response and the tips on working with the .Params.tags. I have to admit, I was assuming that the .Params.tags would result in a []string, so this makes sense as to why my code was failing. I will try to reimplement this using your tips.

As for the structure of the 'if' statement, that is completely untested because I could not even print out the result of intersect. I just wrote it out to illustrate the use case, I am not sure if the code is correct. However, according to the 'Pipes' description on this page (http://gohugo.io/templates/go-templates/), the syntax should be correct as I wrote it. I have not tested, so I could be wrong, but that is the documentation I was working from. It is my understanding that the expression on the left of the pipe will basically be appended to the right of the expression on the right and thus applying to the 'if' statement. I will test out your code and verify.

I think it would be nice to have the 'intersect' function work with additional types. I have to admit that I find the function signatures a little confusing when working with different types. I know I need to use the interface{} type, but that means that every calling instance has to pass interfaces. I think I could do this by adapting the 'in' function to take an interface and use reflection to determine the correct type.

If I can get the basic string example working, I will try to extend the solution to work for all types (like the first and where functions). If I get it working I will submit a pull request to contribute it back.

Thanks again for taking the time to help me out with this.

UPDATE: I tried your code and I am still getting a template error when I try to use the intersect function. Not really sure how to troubleshoot what is wrong.

Here is an example of what I have in my single.html template to do a basic test:

{{ $tags := .Params.tags }}
{{ range .Site.Recent }}
    {{ lst, err := intersect $tags .Params.tags }}
    {{ lst }}
    {{ err }}
{{ end }}

I also tried this:

{{ $tags := .Params.tags }}
{{ range .Site.Recent }}
    {{ $lst, $err := intersect $tags .Params.tags }}
    {{ $lst }}
    {{ $err }}
{{ end }}

Neither of which will even let the server start without errors...

[swill]

Ok, I have simplified my tests even more and I think something else is going on. I must be missing something...

In the hugo/hugolib/template.go I add the following:

A function called test: (eg: http://play.golang.org/p/i8d4DEMcNb)

func Test() string {
    return "It works..."
}

I then modify the NewTemplate() function to add the this function into the funcMap.

funcMap := template.FuncMap{
    "urlize":      helpers.Urlize,
    "sanitizeurl": helpers.SanitizeUrl,
    "eq":          Eq,
    "ne":          Ne,
    "gt":          Gt,
    "ge":          Ge,
    "lt":          Lt,
    "le":          Le,
    "test":        Test,
    "isset":       IsSet,
    "echoParam":   ReturnWhenSet,
    "safeHtml":    SafeHtml,
    "first":       First,
    "where":       Where,
    "highlight":   Highlight,
    "add":         func(a, b int) int { return a + b },
    "sub":         func(a, b int) int { return a - b },
    "div":         func(a, b int) int { return a / b },
    "mod":         func(a, b int) int { return a % b },
    "mul":         func(a, b int) int { return a * b },
    "modBool":     func(a, b int) bool { return a%b == 0 },
    "lower":       func(a string) string { return strings.ToLower(a) },
    "upper":       func(a string) string { return strings.ToUpper(a) },
    "title":       func(a string) string { return strings.Title(a) },
    "partial":     Partial,
}

I then rebuild hugo and move the bin into /usr/local/bin.

I also test to make sure that I am using the correct hugo bin (because I am paranoid at this point).

$ which hugo
/usr/local/bin/hugo

I put the following in my single.html template and test.

{{ test }}

This results in an error:

ERROR: 2014/09/26 Rendering error: html/template: "blog/single.html" is an incomplete template

So now I am really confused. If I remove the {{ test }} line, everything works. If I add that line, everything comes tumbling down. I am really confused why this is not working... Ideas???

[tatsushid]

Hmm, it's strange. I'll try the test and reply

[tatsushid]

Did it and worked it on my env. I added Test function, built a binary, put it into my hugosite top directory and ran it ./hugo server -v. It worked. When I was testing intersect function, I suffered from the same error message. In this case, it was caused by running a wrong binary.

[swill]

I am almost positive that my problem is that I am running the wrong binary. That is what I was suspecting, but I could not figure out how that was possible.

I just tried building and moving it into the top directory in my hugo site and running ./hugo server -v from there and it is still not working. :(

I am super confused.

Are you building hugo with the following?

go build -o hugo main.go

[tatsushid]

I build it just running go build in hugo's top directory, $GOPATH/src/github.com/spf13/hugo on my machine

[tatsushid]

Ah, did you run go build -o hugo main.go? It seems to be rebuild only main package, not rebuild hugolib we has been modifying. go help build says

If the arguments are a list of .go files, build treats them as a list of source files specifying a single package

[natefinch]

That shouldn't matter, all dependencies will get built either way. go build in that directory is the same as go build -o hugo. They'll both build everything and produce an executable called hugo. On Sep 26, 2014 6:25 PM, "Tatsushi Demachi" notifications@github.com wrote:

Ah, did you run go build -o hugo main.go? It seems to be rebuild only main package, not rebuild hugolib we has been modifying. go help build says

If the arguments are a list of .go files, build treats them as a list of source files specifying a single package

— Reply to this email directly or view it on GitHub https://github.com/spf13/hugo/issues/525#issuecomment-57028416.

[natefinch]

Err sorry, same as go build main.go -o hugo On Sep 26, 2014 6:27 PM, "Nate Finch" nate.finch@gmail.com wrote:

That shouldn't matter, all dependencies will get built either way. go build in that directory is the same as go build -o hugo. They'll both build everything and produce an executable called hugo. On Sep 26, 2014 6:25 PM, "Tatsushi Demachi" notifications@github.com wrote:

Ah, did you run go build -o hugo main.go? It seems to be rebuild only main package, not rebuild hugolib we has been modifying. go help build says

If the arguments are a list of .go files, build treats them as a list of source files specifying a single package

— Reply to this email directly or view it on GitHub https://github.com/spf13/hugo/issues/525#issuecomment-57028416.

[swill]

@tatsushid THANK YOU SO MUCH... That was totally what was wrong...

If I build with:

go build -o hugo main.go

The resulting binary does not work.

If I build with:

go build

Then the resulting binary works perfectly.

I would have NEVER caught that. Thank you so much for taking the time to walk through this with me. I will now try to actually implement the 'intersect' functionality. If everything works, except a pull request later this weekend.

[swill]

@tatsushid Thank you so much for your help with this. You have no idea how much time I spent trying to get this 'similar post' functionality to work. This is working perfectly now. :)

I will spend some time this weekend to make the code more generic (not just for strings) and I will submit a pull request so others can enjoy this functionality.

Just as a final note, here is my 'similar posts' code which is working how I want (with the addition of the intersect template function described above).

<ul>
{{ $page_link := .Permalink }}
{{ $tags := .Params.tags }}
{{ range .Site.Recent }}
    {{ $page := . }}
    {{ $has_tags := intersect $tags .Params.tags | len | lt 0 }}
    {{ if and $has_tags (ne $page_link $page.Permalink) }}
        <li><a href="{{ $page.Permalink }}">{{ $page.Title }}</a></li>
    {{ end }}
{{ end }}
</ul>

[tatsushid]

@swill I hope you'll write the function covered other types. And I realized the in function used in intersection may be useful itself as a template function. What do you think?

@natefinch yes, you are right. Both commands rebuild the entire package. I wonder why the problem occured but it's off topic...

[swill]

Yes, I will work on covering the other types. I have to admit this aspect of go is the most challenging for me to understand. I understand it in theory, but in practice I seem to have two left feet. This will be a good exercise for me to get more comfortable with the different reflection concepts and how they should be implemented in go.

Yes, I also think the 'in' function is valuable on its own. I plan to have it work for arrays, slices and strings (hopefully). I exposed the string version initially and planned to expose it in the pull request. Being an avid python developer, I have lots of love for the 'in' method. :)

[bep]

+1 I want these template functions!

I just wanted to chime in and say I had similar build issues with hugo. I later tried to reproduce my steps and create an issue, but I couldn't pinpoint it.

But it seemed like it was building from the wrong source or something.

So I spent:

• Three hours building hugo
• One hour learning myself enough Go to make the changes I wanted

It might be worth wile for someone to start with a clean system and try to follow the build guide step-by-step.

[swill]

I apologize for the delay getting this developed. This is a project that gets about an hour every 3 days, so it has been a bit slow going for me. Oh, and I am a noob at golang, so that is also slowing me down. :P

Since the In function is the foundation of the Intersect function, I am working on it first. I have run into a situation where someone might expect a true result and they get a false result because they might be testing something like 1 int == 1 int8. Since int and int8 are not equivalent types, the In function (as I have it currently written) is evaluating the values as being different.

Should this be the expected behavior? I personally think that all int types (for example) should be able to be compared against each other in this case because there is no way that someone can change the type in the templating language. Do you all agree? If you agree, do you know how I can change the vv.Type() == lvv.Type() condition to reflect that?

Here is a sequence to test to show my current logic and how it currently behaves. http://play.golang.org/p/aV7T8fxdP1

Thanks...

[spf13]

1 should equal 1 regardless of type.

Check how the go template library implemented the eq function. That should give you a good place to start.

[swill]

Ok, here is a better implementation. It now works to compare ints across types. However, it still does not work for floats. Is this expected or is there a way we can compare floats across types?

http://play.golang.org/p/C1T7jtxIyf

[swill]

I just tried using the DeepEqual function to compare a float32 and float64 and that does not work either. I think that comparing float32 with float64 just might not be possible given the difference in precision.

http://play.golang.org/p/To0o6VcxKA

This might just be something we have to live with.

http://stackoverflow.com/questions/22337418/golang-floating-point-precision-float32-vs-float64?answertab=votes#tab-top

[spf13]

I think we're probably ok. I believe most libraries only use float64. It's definitely less of an issue than the ints.

[natefinch]

You can cast everything to float64 and then compare. That's the way most things do number comparisons. On Oct 1, 2014 6:50 PM, "Will Stevens" notifications@github.com wrote:

I just tried using the DeepEqual function to compare a float32 and float64 and that does not work either. I think that comparing float32 with float64 just might not be possible given the difference in precision.

http://play.golang.org/p/To0o6VcxKA

This might just be something we have to live with.

http://stackoverflow.com/questions/22337418/golang-floating-point-precision-float32-vs-float64?answertab=votes#tab-top

— Reply to this email directly or view it on GitHub https://github.com/spf13/hugo/issues/525#issuecomment-57554436.

[natefinch]

I wonder what went wrong with your build. Go is really trivially easy to build with.

Make sure the go tool is in your path, following one of the install guides at http://golang.org/doc/install Set an environment variable called GOPATH, this is where all your Go code will be put.

Run go get github.com/spf13/hugo

That will put the Hugo executable in $GOPATH/bin ....That's it. There's nothing else to it. On Sep 27, 2014 3:52 AM, "Bjørn Erik Pedersen" notifications@github.com wrote:

+1 I want these template functions!

I just wanted to chime in and say I had similar build issues with gohugo. I later tried to reproduce them and create an issue, but I couldn't pinpoint it.

But it seemed like it was building from the wron source or something.

So I spent:

• Three hours building hugo
• One hour learning myself enough Go to make the changes I wanted

It might be worth wile for someone to start with a clean system and try to follow the build guide step-by-step.

— Reply to this email directly or view it on GitHub https://github.com/spf13/hugo/issues/525#issuecomment-57045543.

[swill]

"You can cast everything to float64 and then compare. That's the way most things do number comparisons."

This is what I am doing in this example: http://play.golang.org/p/C1T7jtxIyf

Casting does not work for floats because when you cast a float32 to a float64 it does not have the same value because of the precision changes the value. It works for ints though...

I think I will just do it this way and if people are trying to compare float32 and float64 they will only be equal if the precision actually lines up perfectly.

[swill]

Alright. I have everything working as expected: http://play.golang.org/p/thF-OCWclj

I have submitted a pull request for adding this functionality: https://github.com/spf13/hugo/pull/537

I am now using this code for my 'similar posts' functionality on the new blog I am building...

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.