pugjs / pug

Pug – robust, elegant, feature rich template engine for Node.js
https://pugjs.org
21.58k stars 1.96k forks source link

Can't get 'require' to work inside a template .... #2604

Open markcollinscope opened 7 years ago

markcollinscope commented 7 years ago

I'm not sure if this is a bug (and bear in mind I'm still using Jade, not Pug...

Two files: tmp.js contains: BEGIN module.exports = { value: 'value'} END

file.jade *** BEGIN

args.require is setup before rendering and is just the 'require' function we all know and love to include other modules.

When I render file.jade -- tmp.value is undefined...


FYI -- my objective here is to share field names between node.js routing code and .jade, .pug files. So I want a set of values that can be used in both js and jade, basically 'constants' of a sort -- so the code is not dependent on particular 'field-names' (for forms) being duplicated left right and centre ... but there is a single point of definition of whatever values are used...

<you may say this is a waste of time, I'd be interested in opinions...>

I also tried to include the .js file like so:

include tmp.js

But that didn't work.

Anyhow. Many thanks for any assistance....

TimothyGu commented 7 years ago

Contrary to common belief, the Node.js require function is not a global variable:

require isn't actually a global but rather local to each module.

https://nodejs.org/dist/latest-v7.x/docs/api/globals.html#globals_require

Since a Pug file isn't a Module in the Node.js sense, require is not available in the template.

markcollinscope commented 7 years ago

@TimothyGu Thanks for the feeback. There's a lot of stuff on this subject out there that seems to be completely wrong!

To date I've been passing in constants, etc. using res.locals ... however this seems to be a painful way when all I really want is to be able to use a 'const' value in a Pug file but also share that within JS files as well.

I could pre-process everything and solve it like that. but that seems inelegant. Is there a better way? Many thanks! Mark.

Dan503 commented 7 years ago

Pug needs a way to import and run javascript files from pug templates. I'd love to be able to use npm plugins in my pug files but at the moment the only way that I can see to use js functions in a pug file is to save a complete copy of the javascript file into a seperate pug file and place it inside a "-" javascript block

-
    function functionName(){
        //do stuff
    }

This is really bad for maintainability and makes using npm plugins that have a lot of dependencies extreamly difficult if not impossible to use in pug templates.

Even if npm plugin support isn't possible, I'd at least like an easy way to import my custom js files into my pug templates.

jeromew commented 7 years ago

usually this is done via locals.

var locals = {
  myVar: 1,
  myFunc: require('plugin-as-func')
}

this is just an idea of what can be passed as locals when rendering the template. Maybe that can solve your issue.

Dan503 commented 7 years ago

Ahhh ok. Yes it's possible to pass data into the template when rendering it and npm will have access to the require function when it is defining what that data should be.

That gives me an idea. When I pass the data into the template I'll also pass in functions that I wish to use in my pug templates. I think I'll set up something in my package.json file to define what packages to import.

Thanks! I can't wait to try and implement this work around :D

aMarCruz commented 7 years ago

@markcollinscope , @msantagata , try Brunch + pugjs-brunch.

With this tools, I'm using require inside .pug files in a big App. Ex:

- var dateFormat = require('./lib/dates').dateFormat

.form-group.input-daterange
  label.control-label Creation Date
  input.form-control(name="from_date"
    value=dateFormat(from_date)
  // ...etc

I'm happy with the results, source maps for free.

Dan503 commented 7 years ago

I got excited by that but then I read the documentation and it sounds like the whole project needs to be built around Brunch in order to use it. It's not just a simple little plugin.

I still think passing the required js through the site data is the best way to add the js to the site.

... oh I just realised... I wonder if you just pass the actual "require" function on its own into the site data if that would work...

aMarCruz commented 7 years ago

@Dan503 yes . Brunch is an app blunder with its own plugins.

Dan503 commented 7 years ago

YES!!! It worked!!! :D 💃

When defining the data that gets passed into the pug templates, if you set one of the locals to require:require then you will pretty much have full access to the require function at compile time.

For example this would be how to add it to a Yeogurt setup:

gulp/jade.js Code for adding require function

Then use it in your pug templates as you would normally use it in js files

- var functionality = require('npm-module');

Warning! If a js file has multiple exports, you need to do this to access the default function

- var functionality = require('npm-module').default;
megatolya commented 6 years ago

@Dan503 is it works with webpack on client or is this nodejs-only solution?

Dan503 commented 6 years ago

I'm not sure. Probably. The main thing is that the pug compiler needs to be able to pass variables into the templates.

I would avoid doing the technique outside of node.js though. I get the feeling webpack would try and package up and feed the browser your entire node_modules folder worth of js files if you tried to do this in a way that the browser can access.

aMarCruz commented 6 years ago

Support for require and other node.js features in browsers is resolved at build-time and is working for require in pugjs-brunch and ES6 import in rollup-plugin-pug.

I guess it should also work with WebPack and Browserify.

niksajanjic commented 5 years ago

Passing require to pug templates has one issue while requiring local files. If you have your .pug and .json file in the same folder, and you want to require .json inside your .pug, you would likely write something like this:

- const mappings = require('./cssModules.json')

Unfortunatelly, this will not work. The problem is that in this case require is relative to the folder where you passed it. In my case I'm passing it inside middleware function to ctx.state which later gets assigned as pug locals. That means I have to write relative rules from that place which can be confusing.

Dan503 commented 5 years ago

Ok I've found a pretty good work around for the relative path issue. Instead of passing the default require() function into the Pug locals, create a function that turns an absolute path into a relative path from the root folder.

The following function assumes this folder structure:

[root]
    gulp
        pug-task.js <-- function written here
    gulpfile.js
function pugRequire (providedPath) {
  // Test to see if the path starts with a dot
  if (/^\./.test(providedPath)) {
    throw new Error([
      'Relative paths in `require()` statements are not supported.',
      'Use an absolute path from the root folder instead.',
      'require("/from/root/to/file.js")',
      ''
    ].join('\n'));
  }
  // Test to see if the path starts with a slash
  var isLocal = /^\//.test(providedPath);
  // Make new path relative from root folder.
  var newPath = isLocal ? '..'+ providedPath : providedPath;
  return require(newPath);
}

Then pass it into your pug locals like so

var gulp = require('gulp');
var pug = require('gulp-pug');

function pugRequire (providedPath) {
  // pugRequire code
}

gulp.task('pug', function() {

  return gulp.src('src/**/*.pug')
  .pipe(pug({
    locals: {
      require: pugRequire,
    }
  }))
  .pipe(gulp.dest('output'))

});

You still wont be able to use relative paths but you will be able to reference files from the root folder instead.

// Load a module from the node_modules folder
var npm_module = require('thing-from-npm')

// Load a local js file using an absolute path from the root directory
var local_module = require('/absolute/path/to/file.js')

// Relative paths will throw an error
var error = require('./relative/path/to/file.js')

If the user tries to use a relative path, Pug will throw an error like this:

Message:
    C:\xxxxx\src\_layouts\base.pug:6
    4|   //- Otherwise default to relative pathing
    5|   - var baseUrl = config.baseUrl || './'
  > 6|   - var reqTest = require('./relative/path.js');
    7|   - console.log(reqTest)
    8|
    9| doctype html

Relative paths in `require()` statements are not supported.
Use an absolute path from the root folder instead.
require("/from/root/to/file.js")

Details:
    path: C:\xxxxx\src\_layouts\base.pug

Don't ask me how to make it work with Webpack, I'm not a Webpack fan.

niksajanjic commented 5 years ago

I've actually found a way to use require as I would do it in my JS files and I can use both absolute and relative paths (relative to the Pug file where they have been stated). There are Pug plugins option to achieve it (plugins are not yet official, but the code would still work):

const path = require('path')
const walk = require('pug-walk')
const runtime = require('pug-runtime')
const _ = require('lodash')

const pugPluginRequire = () => ({
  preCodeGen(ast) {
    return walk(ast, node => {
      if (node.type === 'Code') {
        const regExp = /require\(('[\d\w./\-_@!]+'|"[\d\w./\-_@!]+")\)/g
        const req = node.val.match(regExp)

        if (!req) return

        const newReq = req.map(rq => {
          const requirePath = rq.substring(9, rq.length - 2)

          try {
            return require.resolve(requirePath, { paths: [path.dirname(node.filename)] })
          } catch(e) {
            if (e.code !== 'MODULE_NOT_FOUND') {
              throw e
            }
            return runtime.rethrow(e, node.filename, node.line)
          }
        })

        let i = 0
        node.val = _.replace(node.val, regExp, (match, group) => {
          return _.replace(match, group, `'${newReq[i++]}'`)
        })
      }
    }, {
      includeDependencies: true
    })
  },
})

module.exports = pugPluginRequire

I have a plugin function that runs before code generation. In there I'm using pug-walk to traverse through each node (and it's dependencies) and look for the ones that are marked as Code. Then I'm matching a regular expression to check if that code contains any require statement. My regex requires require to contain a string with at least 1 character. Besides alphanumeric characters, I'm also allowing these characters ./-_@! which is enough for my use case.

If there are any matches, I'll strip the path from the require statement. Then I'm doing a require.resolve with that path as a first argument while passing options as the second argument. In those options, you can pass paths option which is an array of all the alternative paths that you want to use instead of regular ones. This allows us to pass path.dirname(node.filename) and force require to resolve pathname as if it was required from that file. This allows us to write relative paths that should be relative to that file where they are stated. require.resolve function would return absolute path (or error out if it isn't found).

Try/catch clause here is to check whether the file exists or we would show an error in the console. While using pug-runtime and it's function rethrow this allows us to format error as if it was thrown from pug and it will point the line of code at which this error was thrown. This allows us to debug code properly, same as it would be if it was thrown by pug itself.

Lastly, we are replacing every require statement inside node.val and passing in a new absolute path. This will allow require statement to resolve both absolute and relative paths.

Now, you can write require statements in your pug files:

-
  const _ = require('lodash')
  const someFunction = require('./relativeFolder/relativeFile')

NOTES/BUGS: 1) This code doesn't allow you to write relative dynamic require statements. For instance, if your statement is an expression require(`./path/to/${file}`) this will be ignored and not manipulated as it doesn't match regexp.

2) Also, if you are using some kind of module resolver, for instance babel-plugin-module-resolver that will not work out of the box. You can either pass manually all those aliases and roots to the paths option inside require.resolve or you can write your own logic that reads configuration from .babelrc.js and populate paths by yourself.

3) I didn't have to pass require statement to my pug locals. I also didn't pass in process.env but it is also available. If I do require.resolve.path('./') inside any of my pug files, it will show me that the path in which resolve happens is inside my routes. That is from where I render my pug templates so that is from where it is available. If I explicitly pass require as my local then the path would be the folder of that file where I passed local require.

4) There is a bug with require.resolve when passing paths as an option. You can see it here and watch if it gets fixed: https://github.com/nodejs/node/issues/18408

Until then, you can use something like this instead of `require.resolve` if you're scared of that bug happening:
```js
if (node.filename.length < 2 ||
  node.filename.charCodeAt(0) !== '.' ||
  (node.filename.charCodeAt(1) !== '.' && node.filename.charCodeAt(1) !== '/')) {
    return path.resolve(part.dirname(node.filename), requirePath)
} else {
  return requirePath // this is ok if you only have 1 node_modules folder, if you have multiple it can produce a bug
}
```

EDIT: Actually, I did have to pass require as my local and I did pass it accidentally in one of my middleware functions without noticing. So, make sure you pass it to.

Alexufo commented 5 years ago

Still

var locals = {
  myVar: 1,
  myFunc: require('plugin-as-func')
}

Current approach to include require in pug?

niksajanjic commented 5 years ago

@Alexufo You will still have an issue with relative require paths, because they are going be relative to the file where your locals are, not relative to the file where require was called:

middleware/locals.js

ctx.state = {
  myVar: 1,
  myFunc: require('plugin-as-func')
}

views/home.pug

-
  const meta = require('./meta.json')
  const generateHtml = require('./generateHtml')

doctype html
html(lang=meta('lang'))
  head
    title= meta('title')
  body= generateHtml('home')

Both require calls are relative to the middleware folder. My plugin up there fixes that issue, but it has other limitations, like dynamic require path.

lunelson commented 4 years ago

I do require in pug templates by adding the following require() function to the pug locals. Note the need to reference the pug source file in resolving the path to the module, and the line where you delete the require.cache entry for the module, as you want it to get a fresh one each time the template compiles:

const pug = require('pug');

const absolutePathToPugFile = '/absolute/path/to/file.pug';

pug.renderFile(absolutePathToPugFile, {
  require(relModule) {
    const absModule = require.resolve(relModule, { paths: absolutePathToPugFile });
    delete require.cache[absModule];
    return require(absModule);
  },
});
albedoa commented 4 years ago

Similar question to @Alexufo above, I would like to assign the result of an import to a variable in a template. So instead of importing a pug file like this and being stuck with the students variable:

-
  const students = [
    //- ...
  ];

I would like to use something like JS modules. I want to import an exported object and assign it to a custom variable. Am I correct that that is the same discussion as the one in this thread and that there is no native way to do it? Or am I describing a different topic? Thanks!

Alexufo commented 4 years ago

@albedoa I include json this way

    const json = JSON.parse(fs.readFileSync('./config.json', 'utf8'));

        .pipe(pug({
            pretty: false,
            locals: {json , fs: require('fs')}
        }))
lunelson commented 4 years ago

@albedoa yes there is no native way to do in pug itself, you have to 'plug it in' in one or other of the ways that have been demonstrated here