Module not found: Error: Can't resolve 'fs'

[chriscalo]

It looks like *.page.js files choke when trying to resolve core node modules.

Here's a simple page config file that grabs the contents of a SQL file and makes a simple database query:

import { file } from "ez-file";
import { query } from "~/util/db";
import page from "./index.vue";

export default {
  route: "/vars/",
  view: page,
  renderToHtml: true,
  async addInitialProps({ res: { req }, res, next }) {
    const sql = file("./vars.sql");
    const [ vars ] = await query(sql);

    return {
      vars: {
        ...vars,
        timezone: Intl.DateTimeFormat().resolvedOptions().timeZone,
      },
    };
  },
};

This seems to build successfully, but when visiting this page, I get the following error:

ERROR in ./node_modules/ez-file/index.js
Module not found: Error: Can't resolve 'fs' in '/Users/chriscalo/Projects/viaticus/node_modules/ez-file'
resolve 'fs' in '/Users/chriscalo/Projects/viaticus/node_modules/ez-file'

My guess is that webpack isn't configured to resolve core node modules, but I don't have experience using webapck to build for Node.js execution.

[brillout]

Yes exactly, webpack tries to build fs.

A trick to bypass webpack is to use eval:

import assert from '@brillout/assert';

export default {
  route: "/vars/",
  view: page,
  renderToHtml: true,
  async addInitialProps({ res: { req }, res, next }) {

    // Make sure that addInitialProps is always only called in Node.js
    assert.internal(isNodejs());

    // The `~/database/query` module will not be built by webpack.
    // Instead, Node.js's `require` will directly load the module.
    const query = eval("require('~/database/query')");

    const sql = "SELECT text from TODO;";
    const [ vars ] = await query(sql);

    return {
      vars,
    };
  },
};

function isNodejs() {
  return typeof window === "undefined";
}

Note that this works only because renderToHtml: true: when renderToHtml: true, Goldpage calls addInitialProps on the server. (You can see that the result of addInitialProps is serialized and added to the HTML of your page in order to make the data available in the browser when your page is rendered to the DOM.)

Otherwise, if renderToHtml: false, then Goldpage calls addInitialProps in the browser and any attempt to require('fs') will inherently fail. In that case you'd need to use Wildcard API. Let me know if you are not sure why — I'm happy to explain.

If you plan to always render your pages to HTML then you should be able to skip using Wildcard.

If it makes things easier for you, you can invite me to your private repo and then we can talk about the codebase directly.

Let me know when something's not clear.

[chriscalo]

Thanks for the explanation. I might give Wildcard API a try, that looks promising.

Here's what I'm trying to do. Do you see anywhere I'm going astray. Any recommendations on the approach?

Server-side render Vue components to HTML, passing in data from the file system / database. I'm hoping to avoid building an API just to merge data into a Vue component. And I don't want to have to limit my options just to universal npm modules that can run in Node.js and in a browser. I'd really love if the logic in Goldpage *.page.js config files ran only on the server, not in the browser, making it possible to use modules like fs and mysql-promise.

Rehydrate the Vue component HTML once it reaches the browser. I'm loving that Goldpage handles this for me. I don't want to give up rich client-side interaction to get the benefits of server-side rendering. Of course, I'll need to build an API for on-page interactivity that requires a server round trip, but that's not hard to do in Express.

Keep all routing logic on the server. I want to keep all of my routing logic the server, either in Express handlers or in Goldpage *.page.js config files. Every time a new URL is requested, there's a full page refresh. This means I don't have to write universal code that can run in Node.js or on the browser for each page refresh. Instead, I'd rather every page load be loaded entirely on the server. Down the road, I might consider something a little more sophisticated like what inertia.js is doing with their mix of HTML and XHR responses.

Thoughts? How do I write non-universal code for fetching data from a database in Node.js and passing it to a SSR'd Vue component using Goldpage?

Thanks again.

[brillout]

I'm hoping to avoid building an API just to merge data into a Vue component.

Yes that's a great idea, there is no need for GraphQL/REST when there is a 1:1 relationship between frontend and backend.

I'll need to build an API for on-page interactivity that requires a server round trip, but that's not hard to do in Express.

Yep exactly. You can then use Wildcard instead of creating Express routes yourself.

Any recommendations on the approach?

That's a wonderful approach that I would also choose.

How do I write non-universal code for fetching data from a database in Node.js and passing it to a SSR'd Vue component using Goldpage?

You can do that by using the eval trick I mentioned earlier. But you're absolutely right, what you are describing should have first-class support; you shouldn't have to use the eval trick to achieve that. I'll have some thoughts about this. I'll come back to you once I found a user friendly solution. In the mean time you can use the eval trick (let me know if the eval trick is not clear to you and I'll show it to you in a PR to your Goldpage starter).

I'm curious; what it is you are building?

[chriscalo]

A trick to bypass webpack is to use eval:

Is there not some way to specify Node.js as the target environment and not bundle specific dependencies for the server-side Goldpage build? Something like webpack-node-externals?

From their README:

---

In your webpack.config.js:

var nodeExternals = require('webpack-node-externals');
...
module.exports = {
    ...
    target: 'node', // in order to ignore built-in modules like path, fs, etc.
    externals: [nodeExternals()], // in order to ignore all modules in node_modules folder
    ...
};

And that's it. All node modules will no longer be bundled but will be left as require('module').

---

I'm curious; what it is you are building?

Nothing too exciting: it’s an old PHP app a friend wrote that I’m modernizing to Node.js + Vue. Because it’s something we won’t spend a lot of time on going forward, I’m trying to really optimize for development ergonomics so it's easy to make changes. But I’m also using this project as an opportunity to build out some patterns I like (a simple Vue SSR + Express setup) so I can quickly scaffold future projects.

[brillout]

Telling Webpack to ignore native node modules alleviates the problem only to a certain degree. Goldpage shouldn't add any server-side code to browser bundles.

Goldpage shouldn't add import { query } from "~/util/db"; to the bundles that are being shipped to the browser.

I'm thinking about a solution. I'll keep you updated.

[chriscalo]

Ah, got it. I keep forgetting that the page config modules are entry points for both server and client bundles. Thanks for giving this some thought 👍

[brillout]

I made good progress this WE on how to solve this.

I don't know (yet) when I will finalize and implement the solution.

In the meantime I recommend using Wildcard. (Instead of using the eval trick.)

Using Goldpage with Wildcard will provide what you want. If it's not clear let me know and I will elaborate.

I opened a new ticket for it: #13.

[chriscalo]

Sounds great, I will try using Goldpage with Wildcard. If you have a working example that I can reference, let me know. I didn't find a wildcard example in the Goldpage examples directory, and the example in the Wildcard API repo doesn't appear to be using Goldpage. If not, no worries, I'll see how far I can can get.

[chriscalo]

I was able to find reframe-full-stack repo. I'll start there.

[brillout]

:+1: Let me know if you have questions. Happy to do some more PRs to your repos.

[chriscalo]

I was able to get things working with Wildcard, and it's pretty compelling, so much cleaner than every other SSR approach I've seen. 👍

I'll try again integrating this into my app and report back.

One suggestion for the Wildcard API docs. I was getting the following error after setting things up naïvely:

****************************************
************* Wrong Usage **************
****************************************
Endpoint greet doesn't exist.

But then I import()ed / require()ed the file that attaches functions to the endpoints object, and things started working. I don't think this was mentioned in the Wildcard API docs, but I know this setup is a little different than what's expected.

import express from "express";
import { getApiResponse } from "wildcard-api"; // npm install wildcard-api

// Ensure API functions get attached to `endpoints` object
import("~/api");

const server = express();
export default server;

// Parse the HTTP request body
server.use(express.json());

server.all("/wildcard/*" , async (req, res) => {
  const { statusCode, contentType, body } = await getApiResponse({
    // `getApiResponse` requires the HTTP request `url`, `method`, and `body`.
    url: req.url,
    method: req.method,
    body: req.body,
    headers: req.headers,
  });
  res.status(statusCode);
  res.type(contentType);
  res.send(body);
});

[chriscalo]

Things are getting closer with Wildcard API, and I'm loving the elegant programming model. I ran into an error when trying to access this in a wildcard endpoint function and filed an issue:

https://github.com/reframejs/wildcard-api/issues/16

[brillout]

I don't think this was mentioned in the Wildcard API docs

It's purposely not mentioned in the docs.

The idea is to show proper error messages and normally you would have gotten an error like this:

    Endpoint `greet` doesn't exist.
    You didn't define any endpoint function.
    Did you load your endpoint definitions? E.g. `require('./path/to/your/endpoint-functions.js')`.

But I forgot to add this explanation as well for SSR. It's not fixed & published under 0.5.4.

I'm loving the elegant programming model

Glad to hear.

Btw thanks for pushing on goldpage dev & nodemon ./path/to/server.js. I like it a lot. It's lovely to have a user that pushes me to improve the design.

[chriscalo]

Btw thanks for pushing on goldpage dev & nodemon ./path/to/server.js. I like it a lot. It's lovely to have a user that pushes me to improve the design.

It's so nice to hear that, thanks 🙏. For me, that change was definitely the moment Goldpage went from something with a lot of potential to a very compelling tool. I've tried every other method I could find for SSRing Vue components, and Goldpage is the only one that both works and isn't a convoluted mess. I think you're really onto something powerful here.

[brillout]

isn't a convoluted mess

Yes, that was precisely my goal with Goldpage. To be do-one-thing-do-it-well whereas others are too framework-ish.

I've tried every other method I could find for SSRing Vue components, and Goldpage is the only one that both works and isn't a convoluted mess. I think you're really onto something powerful here.

Hehe thanks. Could I use this quote in a future landing page?

[chriscalo]

Could I use this quote in a future landing page?

Yes, feel free 👍