preaction / Statocles

Static website CMS
http://preaction.me/statocles
Other
84 stars 33 forks source link

eval is run on blog posts? #541

Closed dnmfarrell closed 6 years ago

dnmfarrell commented 6 years ago
% statocles -v
Statocles version 0.085 (Perl v5.26.0)

blog/2017/09/09/first-post/index.html:

---
title: Plotting With Perl 6
author: Moritz Lenz
tags:
  - inline-python
  - matplotlib
  - git

---

```
my $proc = run :out, <git log --date=short --pretty=format:%ad!%an>;
my (%total, %by-author, %dates);
for $proc.out.lines -> $line {
    my ( $date, $author ) = $line.split: '!', 2;
    %total{$author}++;
    %by-author{$author}{$date}++;
    %dates{$date}++;
}
```

Running statocles build:

Error in template: Error in template: Global symbol "$author" requires explicit package name (did you forget to declare "my $author"?) at /blog/2017/09/09/first-post/index.html line 7.
syntax error at /blog/2017/09/09/first-post/index.html line 7, at EOF
  (Might be a runaway multi-line "" string starting on line 1)
Global symbol "$author" requires explicit package name (did you forget to declare "my $author"?) at /blog/2017/09/09/first-post/index.html line 8.
Global symbol "$date" requires explicit package name (did you forget to declare "my $date"?) at /blog/2017/09/09/first-post/index.html line 8.
Global symbol "$date" requires explicit package name (did you forget to declare "my $date"?) at /blog/2017/09/09/first-post/index.html line 9.
syntax error at /blog/2017/09/09/first-post/index.html line 9, at EOF
2: ```
3: my $proc = run :out, <git log --date=short --pretty=format:%ad!%an>;
4: my (%total, %by-author, %dates);
5: for $proc.out.lines -> $line {
6:     my ( $date, $author ) = $line.split: '!', 2;
7:     %total{$author}++;
8:     %by-author{$author}{$date}++;
9:     %dates{$date}++;
10: }
11: ```
176: 
177:         $content = $t->process( \%args );
178:     }
179: 
180:     if ( blessed $content && $content->isa( 'Mojo::Exception' ) ) {
181:         die "Error in template: " . $content;
182:     }
183:     return $content;
184: }
185: 
186: # Build the Perl string that will unpack the passed-in args
preaction commented 6 years ago

The content is run through the Mojo::Template template parser to allow plugins like highlight which does code syntax highlighting. The parser treats % as the first non-whitespace on a line as a single-line template directive. So line 7 (%total{$author}++;) gets parsed as Perl 5 code.

The most direct way to fix this is to double up the % to get Mojo::Template to ignore it (so %%total{$author}++;). Here's the very basic template syntax.

Using the highlight plugin would also avoid this issue, since it'd be treated as highlighted code and not template to parse:

%= highlight perl6 => begin
use v6.c;
%hank<name> = "Hazel Murphy";
say %hank<name>;
% end

That should work, but I've never tested the Perl 6 syntax highlighting before so I don't know how good it looks. This is handled by Syntax::Highlight::Engine::Kate, which is the only good backend highlighter for Perl I could find. I could extend the highlight plugin to use highlight.js or something if needed.

Unfortunately, since you're doing Perl 6, I think you're going to come across this a lot. I could also come up with a way to either disable document template parsing or change what the directive leader looks like. Changing the template directive marks would be more difficult, since a lot of the good ones are taken by Markdown (#, !, = off the top of my head) or could still be confused for source code.

dnmfarrell commented 6 years ago

Thanks for explaining the issue. Selfishly, I'd like to be able to type:

``` prettyprint
$foo
```

Or:

``` perl
$foo
```

Or nothing for straight up pre:

```
% cp /foo/bar .
```

And Statocles would produce the appropriate HTML for prettify.js or highlight.js. It does strike me as a negative that Statocles can't parse markdown in some cases because of the template/plugin system.

On the other hand, I'm sure there are benefits to using embedded Perl templates and plugins, so I can't say for sure if this "should be fixed" or not. It feels like a design trade off, that you could make a case for either way.

I don't know how much work it would be, but could the template/plugin system be made optional?

preaction commented 6 years ago

I can allow the content template parsing to be disabled, yes. Likely I will make it a setting that could be added at the site level for all documents, at the application level for a single blog's documents, or at the document level (in the frontmatter) for a single document.

The biggest benefit the content templates give Statocles is including files. Most of the additional template functions Statocles currently has are built around that:

%# Include a perl script and highlight it
%= highlight perl => include 'example.pl';

%# Include a markdown file and render it to HTML
%= markdown include 'recipe.markdown';

Template includes are processed as templates and have their own variables, so one could make an "author bio" template and pass in the author's information from the document. (Funny enough, there's a way to disable template processing of an included file: %= highlight perl => include -raw, 'example.pl'; so adding a way to disable it for other content makes sense).

There are the things that Markdown makes tedious. Jekyll has ways of turning data structures into content, and this is something I also needed. This gallery page is generated from a data structure.

The last thing that the template stuff is used for is things that Markdown makes impossible/terrible. Images and other media is a big one. Of course, one is simply supposed to use HTML for these things, but there are other tasks one must do when preparing images for the web, including resizing, resampling, creating thumbnails, preparing different sizes for different browsers, and etc... (this plugin is described in #498).

Right now, I'm pretty sure Text::Markdown does not work with the fenced code blocks (```), and that's the Markdown parser that Statocles uses by default. I'm starting to have the opinion that I should move to CommonMark (which does include fenced code blocks, and even an "info string" which would then let me add the highlighting automatically like you mentioned), but @djerius and I have also been talking of ways to make non-Markdown documents, and making document handling more flexible might open up the tools for dealing with things like templates. If I were to move to CommonMark, the only available Perl module for CommonMark is an XS module, so I'd likely end up having to write the pure-Perl version.

But, so you can keep going, I can make a way to disable the template processing step which happens before the document is given to the Markdown parser.

preaction commented 6 years ago

Okay, v0.086 is uploaded to PAUSE, and allows you to add disable_content_template: true to your site.yml configuration file (in the site object) to disable content template parsing for all documents, or you can add it to the frontmatter to disable it for only a single document (or you can add disable_content_template: false to re-enable it for a single document).

I'm not sure I'm fully pleased with the name or the solution, but if I come up with anything better, this will remain for backwards-compatibility (the burden of all software projects...).

preaction commented 6 years ago

I've now also added the ability to configure the template directives, so you can completely disable the line-based code snippets, and change the tags to look like Template Toolkit tags:

site:
    args:
        theme:
            path: '::default'
            tag_start: '[%'
            tag_end: '%]'
            line_start: "\x1F"

This is probably all I'm going to be able to do to help with this.