feat: Pass language variable to {% highlight %}

[UriShX]

Summary

Highlighting code in template where a file is linked in the markdown front matter can currently happen only if the language of the code is known when writing the template. I propose that variables should be passed to the {% highlight %} tag, like it is for the conditional tags, though it may be more clear if it was in a format that resembles the {% include %} tag.

Motivation

I am building a portfolio template based on Minima, where the projects in a portfolio are stored in markdown files, gathered as a collection. I want to pass a code url (I'm using the jekyll-remote-include plugin for that), and the language to highilght it in, as some are C++, some are JS, and some are other languages still. I am including the links in the file's front matter, and would like to pass the language as well, in the following syntax:

array:
  - code: https://raw.githubusercontent.com/username/repository/branch/cppflie.ino
    lang: cpp
  - code: https://raw.githubusercontent.com/username/repository/branch/pythonfile.py
    lang: python

And in my template I want to write:

{% for i in array %}
    {% capture codeurl %}
        {% remote_include {{ i.code }} %}
    {% endcapture %}
    {% highlight i.lang %}
        {{ codeurl | strip }}
    {% endhighlight %}
{% endfor %}

The current implementaion requires the template to define in advance a language, and I believe it would be beneficial to pass the language using a liquid variable for creating templates which can be agnostic to the language of the file type to be highlighted.

I believe it is in line with the use of conditionals, and may allow easier building of templates which include several code excerpts, in different languages, for example in a tutorial about API usage in different languages. \ I also think that it may be more clear to the user if the syntax can be similar to the {% include %} tag, but the regex expression the paramaters are tested against in the initialize function does not include curly braces, and I'm not sure if adding them may break something down the line or not.

Reference-level explanation

Not sure, but I'll try. New to Ruby, please excuse :)

As far as I can tell, the {% highlight %} tag is defined in the file highlight.rb and its documentation is in tags.md. I believe that this feature will only affect these two files.

In the file highlight.rb, the parameters passed are tested for validity in the initialize function, downcased, then saved to the global variable @lang. It is then used in the render_rouge function. \ I propose to test for the exsitance of a fitting lexer in context, in the render function:

begin
    ::Rouge::Lexer.find((context[@lang]).downcase) != nil
    @lang = (context[@lang]).downcase
rescue
    @lang = @lang.downcase
end

In the initalized, it is then only needed to change @lang = Regexp.last_match(1).downcase to @lang = Regexp.last_match(1). \ In order for that to work, the import of Rouge should be moved to the top (from the current location in render_rouge) and be global.

In the file tags.md the following can then be added for documentation (based on include.md):

Using variables names for the language

The name of the language you for the code to be highlighted can be specified as a variable instead of specifying the language directly in the template. For example, suppose you defined a variable in your page's front matter like this:

---
title: My page
my_code: footer_company_a.html
my_lang: liquid
---

You could then reference that variable in your >highlight:

{% raw %}

{% if page.my_variable %}
 {% capture my_code %}
   {% include {{ page.code }} %}
 {% endcapture %}
 {% highlight page.my_lang %}
   {{ my_code | strip }}
 {% endhighlight %}
{% endif %}

{% endraw %}

In this example, the capture will store the include file _includes/footer_company_a.html, then the highlight will would match the display to match the syntax of liquid.

Drawbacks

1. It's a corner case (though in a tag included in Jekyll core).
2. Rouge should be imported either as global or both in render and render_rouge.
3. In the method I suggested above each time highlight is called the lexer is searched at least twice.
4. The syntax would be better if it was similar to include.

Unresolved Questions

I don't know what I don't know...

[jekyllbot]

This issue has been automatically marked as stale because it has not been commented on for at least two months.

The resources of the Jekyll team are limited, and so we are asking for your help.

If this is a bug and you can still reproduce this error on the latest 3.x-stable or master branch, please reply with all of the information you have about it in order to keep the issue open.

If this is a feature request, please consider building it first as a plugin. Jekyll 3 introduced hooks which provide convenient access points throughout the Jekyll build pipeline whereby most needs can be fulfilled. If this is something that cannot be built as a plugin, then please provide more information about why in order to keep this issue open.

This issue will automatically be closed in two months if no further activity occurs. Thank you for all your contributions.

[UriShX]

Published as plugin at https://rubygems.org/gems/jekyll-highlight-param. Github repo: https://github.com/UriShX/jekyll-highlight-param

[UriShX]

It appears my initial strategy was flawed, and Jekyll / Rouge were just failing gracefully. I have now updated my plugin after some checking on my part. If anyone's willing to take a look at my code I will be very grateful. Essentially, I changed the Regexp to have named groups by which the render() method defines the @lang variable, and added some error checking.

Regexp:

PARAM_SYNTAX = %r!(\w+([.]\w+)*)!x.freeze
      LANG_SYNTAX = %r!([a-zA-Z0-9.+#_-]+)!x.freeze
      OPTIONS_SYNTAX = %r!(\s+\w+(=(\w+|"([0-9]+\s)*[0-9]+")?)*)!.freeze
      VARIABLE_SYNTAX = %r!
        ^(
          \{\{\s*
          (?<lang_var>#{PARAM_SYNTAX})
          \s*\}\}|
          (?<lang>#{LANG_SYNTAX})
        )
        \s*
        ((?<fault1>[}]+\s*|)
        (
          \{\{\s*
          (?<params_var>(#{PARAM_SYNTAX}))
          \s*\}\}|
          (?<params>(#{OPTIONS_SYNTAX}+))
        )
        (?<fault2>.*))?
      !mx.freeze

@lang assignment in render():

if @matched["lang_var"]
          @lang = context[@matched["lang_var"]].downcase
          @lang.match(LANG_SYNTAX)
          unless $& == @lang
            raise ArgumentError, <<~MSG
              Language characters can only include Alphanumeric and the following characters, without spaces: . + # _ -
              Your passed language variable: #{@lang}
              MSG
          end
        elsif @matched["lang"]
          @lang = @matched["lang"].downcase
        else
          raise SyntaxError, <<~MSG
            Unknown Syntax Error in tag 'highlight_param'.
            Please review tag documentation.
            MSG
        end

I have also replaced the require "rouge" to its original place.

[jekyllbot]

This issue has been automatically marked as stale because it has not been commented on for at least two months.

The resources of the Jekyll team are limited, and so we are asking for your help.

If this is a bug and you can still reproduce this error on the latest 3.x-stable or master branch, please reply with all of the information you have about it in order to keep the issue open.

If this is a feature request, please consider building it first as a plugin. Jekyll 3 introduced hooks which provide convenient access points throughout the Jekyll build pipeline whereby most needs can be fulfilled. If this is something that cannot be built as a plugin, then please provide more information about why in order to keep this issue open.

This issue will automatically be closed in two months if no further activity occurs. Thank you for all your contributions.

[jekyllbot]

This issue has been automatically marked as stale because it has not been commented on for at least two months.

The resources of the Jekyll team are limited, and so we are asking for your help.

If this is a bug and you can still reproduce this error on the latest 3.x-stable or master branch, please reply with all of the information you have about it in order to keep the issue open.

If this is a feature request, please consider building it first as a plugin. Jekyll 3 introduced hooks which provide convenient access points throughout the Jekyll build pipeline whereby most needs can be fulfilled. If this is something that cannot be built as a plugin, then please provide more information about why in order to keep this issue open.

This issue will automatically be closed in two months if no further activity occurs. Thank you for all your contributions.

[dramalho]

:+1: I would love to see this PR go ahead :)

[UriShX]

Thanks @dramalho,

You can check out my plugin in the meantime.

[mauricioscastro]

This would be very much appreciated.