layouts/partials/scripts.html fetches external scripts. Proposal to fetch those when doing npm install to allow offline builds

[Nino-Bock]

First of all, I am not a web development expert at all, just trying to understand what is going on. There is something I encountered which is pretty awkward to me.

I was experimenting with the non-module approach, so I cloned the docsy theme into the example site. I did the npm install and it all worked well.

Rebooting my machine, cutting off the internet gave me a bunch of errors. I tracked them down and ended up at the file layouts/partials/scripts.html.

Inside this file we are fetching a few files from an external source. Why don't we fetch them right away while doing the npm install, so we don't need to fetch them each time we build the site?

We're not even fetching up to date versions of the files. Popper for example fetches version 1.16.1 with version 2 being available already. So if we already fetch specific file versions, why not do this on installation?

If for example the theme shall be used for an internal offline documentation or people want to work on the site while being offline, they can not build the site. I am thinking of situations where I am outside my office with a laptop and want to write a blog post or work on my website. If I can not build the site while being offline, that would be a drawback for me. The whole advantage of using a git submodule or local installation above go modules would be the ability to work on my site while being offline once I downloaded the npm modules.

So my proposal is to fetch the scripts which are being referenced in layouts/partials/scripts.html on installation and store them locally. The ones I checked are MIT license, so it should not be a problem to store/host them upon installation.

[deining]

Thanks for your request. Please note that this topic was discussed already: support of offline environments was raised in #873, and #605 deals with support of non-CDN resources. There is even a pull request (#1006) which makes asset urls configurable. Maybe this is of help for you. In any case, we have to keep in mind that we have both npm install and hugo module installation (possibly without npm), and the solution to be implemented should cover both ways of installation.

[Nino-Bock]

Thanks for your reply. I did not see those other requests.

Like others mentioned any tracking or external pulls are a problem as I live in Germany and we have strict laws here now.

I invested quite a bit of time customizing the Docsy theme to my needs just to find out that I probably can not use it because of privacy concerns. It's offers pretty much everything I was looking for, except built in privacy.

Now here is a follow up question: If I do the following, will this ensure user privacy:

• Disable Google analytics
• Disable Google fonts
• Use the local search option instead of the Google search
• Only use English and German as language (no need to pull other fonts)
• Adapt the changes mentioned in #1006 and download all the scripts into my project and reference them locally

uMatrix is showing me, that there jquery and unpkg. Blocking those scripts does not seem to make any difference for the look and feel of the demo site. Are they of any privacy concern? Can I disable those scripts somehow from within docsy/hugo?

Please help, as I really would love to stick with Docsy. It's such a wonderful Hugo theme. I just don't want to get in trouble for overlooking something by accident.

[LisaFC]

You would need local copies of both jquery and unpkg and update the references to them in Docsy files to point to your local versions - jquery is used for multiple features (including offline search) and unpkg for offline search.

[Nino-Bock]

I have tried that today and went through the changes described in #1006. It did not work well for me because local search does not work.

I downloaded the scripts to /assets/js/extern/SCRIPTNAME. After that I defined the variables like described in the pull request mentioned above and defined a variable in the config.toml of my website.

lunr = { src = "/js/extern/lunr.min.js", integrity = "sha384-203J0SNzyqHby3iU6hzvzltrWi/M41wOP5Gu+BiJMz5nwKykbkUx8Kp7iti0Lpli" }

It does not work out, as the script does not load.

This is probably some HUGO related issue I do not understand and all I need to do is to put the files into the correct folder and use the correct path. so what is the best path where I need to download the scripts to and how do I reference them properly?

[LisaFC]

So the absolute simplest thing to do is to put your .js in your /static/ directory, in which case it will be served at

<script src="{{ .Site.BaseURL }}path/to/script.js"></script>

The other more powerful way is to use Hugo Pipes, which will pull from the /assets/ directory and can do things like minify your js for faster loading as well as other asset processing - here's the docs: https://gohugo.io/hugo-pipes/introduction/

and here's an example of getting and minifying a JS resource (scroll to the bottom): https://digitaldrummerj.me/hugo-asset-pipeline/

Anyone who knows more about local Javascript in Hugo than me (that would be many of you!), please chime in!

[Nino-Bock]

Okay, thanks. I'll try the simple method first and work my way to the better methods afterwards.

So here are two more questions:

1. Why don't you include those scripts into the theme by default but rely on external sources. They are MIT license. Is there any legal issues?

2. You are using an old version of popper. Is there any reason why you don't upgrade to Version 2.x but stick with 1.6.x?

3. You pull popper from script, but when doing npm install popper also gets pulled. Why not use that already locally stored version instead? npm install givs me a warning that popper 1.6.x is deprecated which makes question 2 even more relevant to me.

EDIT: The simple method worked. Nice. Thanks a lot for the help. You saved my day.

EDIT 2: Using pipes also woks. Nice.

[LisaFC]

Hurrah, glad that helped!

1: No legal reason, but in general getting resources (esp jQuery) from a CDN rather than serving them yourself can result in better performance for site users. (Also there's the maintenance overhead of downloading/uploading/updating entire third-party files in our own repo rather than just updating version numbers) 2 and 3: There's work in progress in #1187 and elsewhere (if I remember correctly the version of Popper to use depends on the version of Bootstrap and the latest Bootstrap has breaking changes for Docsy)

[Nino-Bock]

Thanks a lot for the clarification.