Url imports resources with Cache-Control: no-cache fails

[SBNTT]

Run next info (available from version 12.0.8 and up)

No response

What version of Next.js are you using?

12.0.8

What version of Node.js are you using?

16.11.1

What browser are you using?

Chrome

What operating system are you using?

macOS

How are you deploying your application?

next dev / next start

Describe the Bug

According to https://nextjs.org/docs/api-reference/next.config.js/url-imports , setting Cache-Control: no-cache on the response should force next to fetch the url each time if I understand correctly.

When I try to access a page in this conditions, the compilation stucks and sometimes this message appears:

error - Error: ENOENT: no such file or directory, open 'app/next.lock/.lock.3955.json'
[Error: ENOENT: no such file or directory, open 'app/next.lock/.lock.3955.json'] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: 'app/next.lock/.lock.3955.json'
}

Moreover, I cannot run "next build" because no next.lock exists.

Expected Behavior

In dev mode, the url imported module should be fetch each time to build the page.

When building for production, it should fetch the url imported module in order to statically render the page.

When using ISR, and during next start, it should re fetch the url imported module in order to rebuild a specific page. No .next.lock should be version controlled then as url imported modules should be fetched each time. Maybe next build should always fetch missing url imported module because as of now, it has no ways to know if that resource is served with no-cache directive or not

To Reproduce

Clone https://github.com/SBNTT/nextjs_url_imports_issue

cd modules/module_a
yarn install
yarn build
yarn start

http://localhost:3001/index.js should be accessible with Cache-Control: no-cache header in its response

Now, cd inside app directory and start it in development mode. Accessing http://localhost:3000 will fail with the internal server error mentioned above

[SBNTT]

@balazsorban44 is it possible to have en ETA or at least a feedback on that ?

[balazsorban44]

Hi, thanks for reporting. I think I could reproduce some weird behavior. The issue I found was that when using localhost as an URL and there was no lock file generated yet, the dev server just hung, and the build failed with your description.

Interestingly adding some other URLs to a different page and rendering that first in dev, the lock file is generated correctly. After that, visiting the page with the localhost import will work.

pages/confetti.js

import confetti from "https://cdn.skypack.dev/canvas-confetti"
import { useEffect } from "react"

export default function Page() {
  useEffect(() => {
    confetti()
  })
  return <p>Hello</p>
}

Adding and visiting this page first created the lock file for me and after that the build passed also. We will need to look more into this.

[SBNTT]

It works with any other url than one from localhost because there is no Cache-Control: no-cache header in the response.

While serving from localhost, and without this header, it works as expected

[logikaljay]

@SBNTT Did you ever find a solution for this?

I am running in to this issue at the moment.

[SBNTT]

@logikaljay Nope. We've postponed implementing this part for now :/

[logikaljay]

@SBNTT @balazsorban44

I have traced this back to be an issue with webpacks HttpUriPlugin.

Basically it is not checking that the lockFileLocation directory (next.lock) exists before trying to write to a file inside of it.

The easiest "band-aid" is to just mkdir next.lock as part of your package.json npm scripts. EG:

{
  "scripts": {
    "dev": "mkdir next.lock && next dev",
  }
}

A better fix would be to have the next.lock created if it does not exist in the next.js webpack.config.js file:

webpack5Config.experiments = {
  // ...
}
if (webpack5Config.experiments.buildHttp) {
  fs.mkdirSync(webpack5Config.experiments.buildHttp.lockfileLocation)
}

But IMO, the best fix would be for webpack to test if options.experiments.buildHttp.lockFileLocation exists and create it if not before trying to write to it.