Heredoc rendering problem

[openstrike]

In 2023-12-12.pod at the end (in section 6) are two separate code snippets with heredocs, one of which is indented on purpose. However, in the rendered article this indentation has been stripped.

In both cases the lines are contained within a <span class="heredoc_content"> so it looks like the leading whitespace is being stripped on purpose as a code-linting operation. That used to be a good thing but it makes Brian's article read wrong.

We need a way to preserve the leading whitespace in heredocs in more recent articles while (I guess) maintaining the old system for previous articles.

[oalders]

Is this happening in WWW::AdventCalendar?

[openstrike]

Not in the code that I can see. It could well be in one of its dependencies though - there are plenty in there about POD manipulation.

Might take a while to track this down.

[oalders]

On a related note, we're on a fork of WWW::AdventCalendar that is not up to date with upstream. I haven't checked on what the latest commits have added, but might be worth pulling those in at some point.

[thibaultduponchelle]

This issue is reproduced on https://perladvent.org/1999/1999-12-05.html

[thibaultduponchelle]

The following 2 sources are currently broken.

=begin perl

if( ... ) {
    my $string = <<~'HERE';
        This string is outdented!
        Here's another line!
        This is nicer!
        HERE

    ...
}

=end perl

(begin/end code block with syntax highlighted)

And

    #!perl
    if( ... ) {
        my $string = <<~'HERE';
            This string is outdented!
            Here's another line!
            This is nice
            HERE

        ...
    }

(indented code block with syntax from shebang)

[oalders]

Are these also broken using the vim syntax highlighting?

[thibaultduponchelle]

Yes, but this time it's not about indentation but syntax highlighting...

Same for =begin vim perl/=end vim version.

(it's now included in #414 )

[oalders]

I wonder if this is down to the version of vim we are using? I had thought indented heredocs were fixed in later vim versions. Or maybe this needs to be merged: https://github.com/vim-perl/vim-perl/pull/327/files

[oalders]

We could possibly try to use neovim instead. treesitter should do the right thing in this case.

[thibaultduponchelle]

The change https://github.com/vim-perl/vim-perl/pull/327/files looks promising, I will check that

[thibaultduponchelle]

It seems well related to a version of vim(-perl) but not this patch in particular.

[thibaultduponchelle]

@openstrike @oalders

I would like to present an alternative approach to syntax highlighting using a JavaScript library (in this proof of concept it’s prism.js) instead various Pod transformers.

You can view it here: https://thibaultduponchelle.github.io/Perl-Advent/1999/1999-12-06.html

Please tell if it’s a path we should consider.

It’s a bit hacky for now (still use Pod transformer for detecting code blocks but bypass syntax highlighting and leave it to JavaScript lib) but can evolve to drop some Pod transformers later.

[oalders]

That looks great. My concern with using a JS library is that it may not be maintained at some point, whereas something like vim/neovim will probably have much more consistent support for syntax highlighting. I'm going to ping @haarg on this as he may have opinions based on how MetaCPAN handles syntax highlighting.

I should add that PerlNavigator has hacked in some support to fix issues with Perl syntax highlighting. Not sure if we can make use of that. As far as the JS goes, it would also need to keep up with new features which get added to Perl etc.

Having said all that, the calendar needs to handle more than just Perl, so there's that to consider as well.

[thibaultduponchelle]

415 has fixed heredocs in code blocks using vim but not the ones declared with =begin perl or #!perl

[oalders]

I wonder if we should rewrite older code blocks to use vim for highlighting rather than having to support all of the TIMTOWDI?

[thibaultduponchelle]

You mean edit all posts or hack the transformation?

[oalders]

From a maintainability perspective, if it all "just works" with one highlighting method, we could bring all past posts in line with that. So, yes, we could rewrite old posts to use the one thing that works.

[thibaultduponchelle]

Or yet another submodule/fork to fix the other transformer? 😵

See https://github.com/perladvent/PPI-HTML/pull/1