Can't download. Broken .zip file returned

[Pingger]

Trying to download highlight.js using the website doesn't work properly.

I can download highlight.js without any languages, but as soon as I requesta few languages I instead receive a broken zip file highlight.zip Opening it with a texteditor shows, that the server throws an error:

{"errorType":"Function.ResponseSizeTooLarge","errorMessage":"Response payload size exceeded maximum allowed payload size (6291556 bytes)."}

I'm sorry, but without any support for languages, highlight.js isn't really useful to me.

It appears to me, that the Server is misconfigured and doesn't allow responses large enough to include languages in the response.

[joshgoebel]

Weird, nothing should make the response 6.3 megabytes though...

I tried it just now with a few languages and it worked just fine... maybe you need to be more specific on which languages you chose.

[Pingger]

Initially, I had everything ticked. Because the project I wanted to use hljs for has no specific constraint which language will actually be used. (Being a knowledge base for issues and solutions, that I encountered in programming and system/server administration)

My workaround was cloning the repo and for each file in the src/languages dir and download the .min.js from the CDN.

Then I cat them together, because linking so many js files slows down website loading immensely.

I am btw. very disappointed, that there is no "download with all languages" and you have to select every single language by individually on the download page ... No "select all in category" or similar... Quite a downgrade to the old website imho. (If didn't know hljs before, that would have immediately turned away, from even trying it. Even knowing it, my first response was searching for alternatives, of which I found a viable one)

[joshgoebel]

Initially, I had everything ticked.

This is not recommended.

Then I cat them together,

That's what the website's own build process does...

No "select all in category" or similar

@allejo I agree this is a pretty big loss... I'd at least love a way to select "common" easily and just adding chekboxes per group should be pretty simple...

[allejo]

Honestly, I completely forgot about this feature on the old website. Just added that in 22d1e8a0a1b53a7191fc3b6db6efb694fed67f6c.

As for the large download size, when I run the website locally and download a ZIP (📄 highlight.zip) with all the languages, it comes out to ~9.6MB. Including full copies of everything in ESM and CJS (I think) adds up, that's more than 5MB of text already.

[allejo]

After a bit of research, looks like this is a limit with severless functions; they have a response body limit of 6MB which means the structure of the download size needs to change.

• https://github.com/vercel/vercel/issues/3825
• https://stackoverflow.com/a/74242927

My thinking right now is to allow users to opt-in to what they want because duplication is what leads to the giant file sizes. I'm thinking to let users pick the following:

• CJS format OR ESM format
• Minified OR non-minified

I think with these limitations in selections, the sizing should be a lot smaller. Thoughts?

[joshgoebel]

I'm not sure how you are getting files so large. Our whole Node.js package (unpacked) is only 5.29mb... and that includes themes and everything... cdn_assets build is only 4.21mb...

I'm building browser with the whole kitchen sink and this is what I get locally (before compression):

-rw-r--r--   1 jgoebel  staff  1560440 Dec 24 17:29 highlight.js
-rw-r--r--   1 jgoebel  staff  1002496 Dec 24 17:29 highlight.min.js

I thought the website's main purpose was to build a combined highlight.js.min for those unable or unwilling to just do so from the source... has it historically included all the themes, etc?

[allejo]

I'm not doing anything special to bloat the file sizes, it seems like it's because I'm including both CJS and ES in one bundle that's doubling the size.

$ du -sh highlight/* | sort -hr
3.5M    highlight/es
3.3M    highlight/languages
2.3M    highlight/styles
1.6M    highlight/highlight.js
1.0M    highlight/highlight.min.js
 76K    highlight/DIGESTS.md
4.0K    highlight/package.json
4.0K    highlight/README.md
4.0K    highlight/LICENSE

i.e. es/ is 3.5M because it holds all of the languages that are also in languages/; except it's in a different format: CJS. That's why I'm proposing that the download page is updated to force users to pick one format. Ideally, also include/exclude minified/expanded versions too to reduce the size of the final zip.

[joshgoebel]

I'd say if someone wants a CDN build they should get it from cdn-release... the website doesn't need to build a FULL CDN release... it should rather provide just a "merged" web build of the JS for those unwilling or unable to build it themselves from source.

If someone wants the "rest" of the CDN bundle, they can build + download a cdn-release zip.

It doesn't bother me if this build service is ultra "minimal" since it's honestly trivial to build from source directly.