How to use highlight.js outside the browser?

[zeke]

I'm considering pre-highlighting code on the server-side to avoid making users download the highlight.js library, but I'm not exactly sure how to do it.

Say I have a README file in Markdown, and I use marked to parse it into HTML:

require("marked").parse(require("fs").readFileSync("./README.md").toString())

Now I have this HTML:

<h1 id="jiggle">jiggle</h1>
<p>Jiggle arrays to slightly rearrange their values</p>
<h2 id="installation">Installation</h2>
<pre><code class="lang-sh">npm install jiggle --save
</code></pre>
<h2 id="usage">Usage</h2>
<pre><code class="lang-js">var jiggle = require(&quot;jiggle&quot;)

// The jiggle() function takes an array as input and returns an array
jiggle([0,1,2,3,4,5,6,7,8,9])
// [ 1, 2, 0, 4, 3, 5, 6, 7, 9, 8 ]

// It can also jiggle strings
jiggle(&quot;jabberwocky&quot;)
// jbbeawockyr

// It always returns a new object without altering the original
var orig = [&quot;a&quot;,&quot;b&quot;,&quot;c&quot;,&quot;d&quot;,&quot;e&quot;,&quot;f&quot;]
var dupe = jiggle(orig)
// orig: [&#39;a&#39;,&#39;b&#39;,&#39;c&#39;,&#39;d&#39;,&#39;e&#39;,&#39;f&#39;]
// dupe: [&#39;a&#39;,&#39;b&#39;,&#39;d&#39;,&#39;e&#39;,&#39;c&#39;,&#39;f&#39;]

// And if you like jiggling you can keep on doing it
jiggle(jiggle(jiggle(&quot;supercalifragilisticexpialidocious&quot;)))
// supcaerligralitiiscpefaldioxciosiu
</code></pre>
<h2 id="tests">Tests</h2>
<pre><code class="lang-sh">npm install
npm test
</code></pre>

How would I use highlight.js to apply syntax highlighting markup to this HTML string? Please forgive me if I missed the documentation somewhere that explains this. :)

[sourrust]

There is documentation in marked for this. The example only uses auto highlighting when it is possible to specify a language like the Pygments example.

var fs     = require('fs');
var hljs   = require('highlight.js');
var marked = require('marked');

var markdownString = fs.readFileSync('./README.md');

marked.setOptions({
  highlight: function(code, lang) {
    return hljs.highlight(lang, code).value;
  }
});

var output = marked(markdownString);

[zeke]

Nice. Thanks, @sourrust

[xcatliu]

Give a compatible version:

marked.setOptions({
  highlight: function(code, lang) {
    if (typeof lang === 'undefined') {
      return hljs.highlightAuto(code).value;
    } else if (lang === 'nohighlight') {
      return code;
    } else {
      return hljs.highlight(lang, code).value;
    }
  }
});

[stla]

Hello, I get an error:

$ node marked_test.js  > xx.txt
/home/stla/Github/temp/node_modules/marked/lib/marked.js:1227
    throw e;
    ^

TypeError: src.replace is not a function
Please report this to https://github.com/chjj/marked.
    at Lexer.lex (/home/stla/Github/temp/node_modules/marked/lib/marked.js:138:6)
    at Function.Lexer.lex (/home/stla/Github/temp/node_modules/marked/lib/marked.js:129:16)
    at marked (/home/stla/Github/temp/node_modules/marked/lib/marked.js:1219:31)
    at Object.<anonymous> (/home/stla/Github/temp/marked_test.js:14:14)
    at Module._compile (module.js:409:26)
    at Object.Module._extensions..js (module.js:416:10)
    at Module.load (module.js:343:32)
    at Function.Module._load (module.js:300:12)
    at Function.Module.runMain (module.js:441:10)
    at startup (node.js:139:18)

Node version:

$ node -v v4.5.0

Source file:

var fs     = require('fs');
var hljs   = require('highlight.js');
var marked = require('marked');

var markdownString = fs.readFileSync('./node_modules/marked/README.md');

marked.setOptions({
  highlight: function (code) {
    return hljs.highlightAuto(code).value;
  }
});

var output = marked(markdownString);

console.log(output);

[zeke]

@stla we had some trouble running highlight.js at scale, and created https://github.com/npm/marky-markdown which uses https://github.com/atom/highlights to do syntax highlighting.

[stla]

Thank you. The conversion works with marky-markdown, but there's no styles in the html (only basic div and span, without css).

[zeke]

The conversion works with marky-markdown

Are you converting in the browser, or in node?

[isagalaev]

First of all, do I understand it right that the bug in question has nothing to do with highlight.js? Looks like it's something in marked.

Also, @zeke, could you please elaborate on this:

we had some trouble running highlight.js at scale

What scale, what was the problem (slow? bugs?) Thanks!

[zeke]

what was the problem (slow? bugs?)

In late 2014, we (npm, Inc) tried to use highlight.js to process the READMEs of all npm packages (about 90,000 of them at the time). Many of these markdown files contained code snippets that would cause highlight.js to leak memory and eventual crash the node process. We spent a good amount of time trying to debug it, but eventually moved to pygments (python) and ultimately highlights (js with a c-based regex parser)

[isagalaev]

Huh… Were you able to get any insights about the possible sort of memory leaks? One should be really unlucky to get that in a managed language, I can't think of anything unusual we might do in our code to cause memory leaks.

[stla]

Are you converting in the browser, or in node?

I tried in node and using the executable. I get output like:

<div class="line"><span class="text"><span>data&#xA0;Point&#xA0;=&#xA0;Point&#xA0;Float&#xA0;Float&#xA0;deriving&#xA0;(Show)&#xA0;&#xA0;</span></span></div><div class="line"><span class="text">

There's no css styles except class="line" or class="text".

[zeke]

@stla what does your source input look like?

[stla]

This is a Haskell script. Using pandoc, the conversion is perfect.

Le Vendredi 19 août 2016 19h34, Zeke Sikelianos <notifications@github.com> a écrit :

@stla what does your source input look like?— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.