[🐞] Non-deterministic builds

[Nefcanto]

Which component is affected?

Qwik Rollup / Vite plugin

Describe the bug

Very brief. We created a meta-framework on Qwik and Qwik-City. Then we reported https://github.com/QwikDev/qwik/issues/5281. And it did not get solved. We managed to work around it (abandoning some features). Yet now we're stuck again.

This is the problem. We have a signin page, and when we click on the button, we get this error:

Uncaught (in promise) TypeError: Failed to resolve module specifier "QuestionParts". Relative references must start with either "/", "./", or "../".

We can't create an MRE. We have tried tens of times with no success. But this time we did something different:

1. Ran npm run build and node server/entry.express.js in the development environment. The problem is not there.
2. We compared our code inside the dev docker container, with the code in our CI/CD. There were some differences, but we mitigated those differences (for example having 404.tsx in the development but not in the CI/CD as it causes SSG problems). Yet the CI/CD code still gave us that error.
3. Then we did one strange thing. Built our code in our CI/CD but without specifying ENV NODE_ENV production and copy/pasted CI/CD code just after build alongside the original source code into the development container. It did not work as expected. The node server/entry.express.js gave us that error.
4. Now we simply removed build directories rm -rf dist && rm -rf server && rm -rf tmp and ran npm run build on the same code inside our dev container. It worked.

This is an image using the Beyond Compare. Left side is the code that does not work. Right side is the same code that works:

As you can see, the only difference is the dist and server and tmp directories. And when we used grep -rl QuestionParts this is what we get:

As you can see, for the version that does not work, the QuestionParts is built into an additional file. But for the version that works it's not built into its own file.

Now we use our own dev image for building in our CI/CD pipeline. This means that both are built using the same NODE_ENV and using the same node_modules. Even you can see that both node_modules directories are symbolic links.

I can't find out why it is so. The question is, why do we not get deterministic build outputs for the:

1. Exactly the same source code (even inside the same directory)
2. Both on containers created from the same docker image
3. Both without NODE_ENV=production
4. Exactly the same node_modules

Reproduction

https://github.com/Nefcanto/QwikModuleSpecifier

Steps to reproduce

The reproduction is not simple. Because we have no idea where the problem lies. The specified reproduction is not a real MRE. It's code from our real project. Yet if you would like I can manage to work it out with you and give you a demonstration or send you a film.

System Info

System:
    OS: Linux 6.8 Debian GNU/Linux 12 (bookworm) 12 (bookworm)
    CPU: (8) x64 Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz
    Memory: 9.51 GB / 15.50 GB
    Container: Yes
    Shell: 5.2.15 - /bin/bash
  Binaries:
    Node: 20.12.2 - /usr/local/bin/node
    Yarn: 1.22.19 - /usr/local/bin/yarn
    npm: 10.7.0 - /usr/local/bin/npm
  Browsers:
    Chromium: 124.0.6367.78
  npmPackages:
    @builder.io/partytown: ^0.10.2 => 0.10.2 
    @builder.io/qwik: ^1.5.3 => 1.5.3 
    @builder.io/qwik-auth: ^0.1.3 => 0.1.3 
    @builder.io/qwik-city: ^1.5.3 => 1.5.3 
    typescript: ^5.4.5 => 5.4.5 
    undici: ^6.15.0 => 6.15.0 
    vite: ^5.2.10 => 5.2.10

Additional Information

No response

[wmertens]

Are you sure that the versions are pinned in both environments?

Bundling is done by rollup and we don't control it.

[Nefcanto]

@wmertens, yes, I'm sure that both use the same NPM packages (versions) for build.

Does rollup also build the source code? Is building and bundling mixed and done in one phase?

[Nefcanto]

Recently we have seen these types of errors:

It comes by itself usually when we start a project, and it goes away, automatically reloading the page.

Since we do not have a dynamic import in the Layout/Component I guess this might be related to the non-deterministic build.

[Nefcanto]

This is another example of the non-deterministic builds. It's the same app, side by side. Left side you see our development environment. The right side is the same app built and running in the production environment.

These are the differences:

1. When we click the image for each food, we show a modal. It works in the dev, but not in the prod (even no log in the console)
2. When we click on the category (top of the page) we scroll the page to the foods in that category. Works in both environments.
3. When we scroll the page, we choose the category automatically based on the position on the page. Works in the dev. Not working in the prod (and no log)

https://github.com/QwikDev/qwik/assets/12782347/14a2bf3a-963c-4cfa-b887-56413424c352

What do you think we should do? We're stuck at delivering sites to our customers. Because what works in the development environment, might or might not work in the prod.

[gioboa]

Can you share a minimal reproduction for this? A public URL can be good as well 👍

[wmertens]

@Nefcanto it seems as if you are mixing files from different builds somehow. When you run in dev or preview on your own system does it also fail in this way?