Add support for looping through pages in a folder

[JimBrouzoulis]

It would be useful if it is possible to access all pages below some given folder/the current folder, in the template. This allows us to create a list of all the subpages, and links to them, without having to type them in one at the time. Supporting something like the Content Sections in Hugo https://gohugo.io/content-management/sections/

Example of what I would like to write in a template:

{{for page in folder}}
<a href="{{fill page}}">...</a>
{{end}}

Different syntax alternatives

• {{ for page in folder }} where folder is a file-path string
• {{ for page in page_variable }} where page_variable is an iterable containing a list of pages
• {{ for page in folder/ }} to convey a different semantic meaning of the loop, in this case, a file-system folder
• {{ for page in children() }} where children is a function that returns an iterable. Supporting functions like this would be very handy in many other cases as well, but you more or less get what I'm asking for with minimal effort.

I think the last bullet is my favorite, but if one can also nest functions, something like {{for page in sort(children(), :time)} or {{for page in sort(children(), :weight)} it becomes really powerful. But I guess this last part is actually another issue support functions and filters in templates.

[sophicshift]

See #331 and https://github.com/tlienart/FranklinTemplates.jl/issues/30; For now, you can use Julia code evaluation to do something like that. I have adapted the script in the second issue above so that it works in the newest version of Franklin:

```julia:exlist
#hideall

using Franklin
path = Franklin.PATHS[:folder]

function find_title(pg)
    content = read(pg, String)
    m = match(r"@def\s+title\s+=\s+\"(.*)?\"", content)
    if m === nothing
        m = match(r"(?:^|\n)#\s+(.*?)(?:\n|$)", content)
        m === nothing && return "Unknown title"
    end
    return m.captures[1]
end

println("~~~")
println("<ol>")
for (root, _, files) in walkdir("blog/") # (replace by your choice of dir)
    for file in files
        md   = joinpath(root, file)
        html = replace(md, joinpath("src", "pages") => "pub")
        html = replace(html, r".md$" => "")

        t = find_title(md)
        l = Franklin.unixify(html)
        println("<li><a href=\"/$l\">$t</a></li>")
    end
end
println("</ol>")
println("~~~")

\textoutput{exlist}

But I agree a syntax for that would be nice!

[tlienart]

Thanks both! So this is fairly easy to add, I just want to think a bit about the right syntax so that it's not too ambiguous with the for loop that loops over a page variable.

I'm leaning towards:

{{ forpath p in folder/ }} 
  <a href="{{fill p}}">...</a>
{{end}}

we could also just have for and distinguish from the usual one by detecting the presence of the final / but I'm a bit worried it's confusing and ambiguous compared to the for which loops over a page variable since that forpath does a bit more than just loop over elements, it would

• retrieve each file system path in a subfolder e.g. /Users/tlienart/test/blog/foo.md
• transform that into the relevant URL [/prepath]/blog/foo/

This would be in a similar vein with the existing {{ ispage /blog/* }}.

Comment on functions

I also like your last bullet point but having full fledged eval in HTML is not yet possible and I'm not entirely convinced it would be the best way forward. There is yet another way to do what you guys are thinking and it's to define a (global) page variable which is effectively a list of pages. The dummy way would be to put in your config.md:

@def subpages1 = ["path/foo/", "path/blah/"]

of course you want to fill that list automatically but you can in fact pass valid Julia code there too:

@def subpages1 = ["path/$(e)/" for e in readdir("path")]

and then {{for page in subpages1}} will just work.

One limitation of this is that, currently, the config.md file is only processed when it's modified and triggers a pass on all pages as the config.md may modify things that other pages depend upon.

But following that line of thought, we could have a vars.md file which contains only global page var definitions like the one above and that is re-processed upon every event. So something similar to config.md in that it sets page variables globally but distinct in that it's always re-processed.

For the most recent posts you could then have something like

@def recent_posts = (
    l1 = [joinpath("blog", page) for page in readdir("blog")];
    l2 = filter!(f -> endswith(f, ".md"), l1);
    l3 = sort(l2, by=f->stat(f).mtime, rev=true)
    ["blog/$(splitdir(f)[2])/" for f in l3[1:5]] # last five)

further you could even call functions defined in config.md (see #330).

What do you folks think?

[JimBrouzoulis]

Sorry for the later reply...

I think this is a good way and the solutions you provided covers my use-cases. Maybe it would be too much with having eval in the HTML, especially since you can achieve the same results in other ways.

Having something similar to vars.md, as you described, sounds useful. But personally I wouldn't mind restarting the server once in a while since one usually spends much more time writing content than adding pages.

[sophicshift]

It would also be very useful if the page variable in {{for page in subpages1}} made it easy to access variables defined in the page, like tags, title, category, or even content. Not sure if {{fill find_title(page)}} is valid syntax, but something like that would be needed to implement a nice list of pages in a blog.

[sivapvarma]

Having a metadata section at the beginning of a page is probably cleaner than the current way of @def title. Specifically I feel that having @def is a distraction. Jekyll does something like that. That way users have a cleaner way of adding other metadata like author, date, modified time stamp - just a keyword and corresponding value.

I really like that Franklin comes with OOB support for Julia and KaTeX. There is no standard templating framework in Julia yet, may be borrowing the best of either Jekyll (Liquid templates) or Hugo ( Go templates) or Pelican (Jinja2) could be the way forward.

[tlienart]

Ok so the looping is now allowed in v0.7 you would do something like this:

in utils.jl

function get_recent_blog_pages()
  paths = String[]
  for (root, _, files) in walkdir("blog")
    for file in files
      fpath = joinpath(root, file)
      # some filtering here to check the most recent or whatever
      push!(paths, fpath)
    end
  end
  return paths
end
recent_blog_pages = get_recent_blog_pages()

On whatever page you need the list to appear, say index.md

~~~
<ul>
{{ for p in recent_blog_pages }}
  <li> {{fill title p}} ... </li>
{{end}}
</ul>
~~~

[sophicshift]

@tlienart I think that this is not finding the title of pages where I did not set @def title explicitly, even though it could be inferred from the h1 title (I'm testing in my blog)

[tlienart]

Can you confirm this and potentially open a specific issue with it? it definitely should as per these lines:

https://github.com/tlienart/Franklin.jl/blob/033a41dcd56f0fb63b2446db92dfbb6894015c5d/src/converter/markdown/md.jl#L146-L148

[sophicshift]

Also, for some reason all my pages in the deployed site have links like ../page/ and not ../page.md, so when I do <a href="/{{fill p}}">{{fill title p}}<\a> I get a 404 :slightly_frowning_face: maybe this is related?

[tlienart]

ok yes that's a different issue, the easy way around is to put this in your utils.jl:

function hfun_fillurl(params)
  path = splitext(params[1])[1]
  # do the relevant thing here probably
  path == "index" && return "/"
  endswith(path, "index") && return path * ".html"
  return joinpath(path, "index.html")
end

(this needs some testing of course but you get the gist)

{{fillurl p}} should then do the right thing

[tlienart]

PS: I need to go through this in full details and show an example but haven't yet had the time to do it, if you manage to do this in a nice way let me know :)

[tlienart]

Ok so I tried on my side, there are some glitches so I need to look into this in more details. Thanks for raising the issue and apologies for the problem :)

I'll open a fresh issue to discuss it.