Nx 19 not properly loading environment variables in parallel jobs if daemon disabled

[mpsanchis]

Current Behavior

Given an Nx repo, if:

• Nx daemon is disabled (either because Nx is running in CI, or simply because we set NX_DAEMON to be false)
• Target t is present in enough projects
• Tasks are run in parallel, with a high enough p in npx nx run-many --target=t --parallel=p

Then some targets do not start with the environment variables from .env files loaded in their memory.

I am aware of how imprecise this description is. Please see example repo below and instructions on how to reproduce.

I attach a task tree that triggers this behaviour in my computer. The echo-vars targets do not find the environment variables.

Example logs when it fails:

Expected Behavior

Changing the number of tasks being run, their order, the --parallel flag, or setting on/off the daemon should not alter the behaviour of Nx when running targets.

Environment variables from .env files should always be provided to targets. Here is an example the logs after running the same command but less parallel jobs:

GitHub Repo

https://github.com/mpsanchis/nx-19-example

Steps to Reproduce

1. Clone the repo provided above, and make sure you have yarn and node
2. Run yarn install --frozen-lockfile
3. Create a bunch of .env variables in the fake/ dir by running npx nx run var-creator:create-vars
4. Disable the Nx daemon: export NX_DAEMON=false (equivalently for this exercise, export CI=true)
5. Run npx nx run-many --target=release-me --parallel=1 --skip-nx-cache --verbose
6. Run npx nx run-many --target=release-me --parallel=8 --skip-nx-cache --verbose

Note the difference when running the tasks with different level of parallelism. With parallel=8, the targets aren't provided with all the environment variables: the ones coming from the .env files are not in process.env.

There are also differences if you pass --projects=<comma,separated,projectNames>: one project will always work, but if you add many, it starts to fail.

Nx Report

Node           : 22.2.0
OS             : darwin-arm64
Native Target  : aarch64-macos
yarn           : 1.22.19

nx                 : 19.6.0-beta.6
@nx/js             : 19.6.0-beta.6
@nx/jest           : 19.6.0-beta.6
@nx/linter         : 19.6.0-beta.6
@nx/eslint         : 19.6.0-beta.6
@nx/workspace      : 19.6.0-beta.6
@nx/angular        : 19.6.0-beta.6
@nx/cypress        : 19.6.0-beta.6
@nx/devkit         : 19.6.0-beta.6
@nx/eslint-plugin  : 19.6.0-beta.6
@nx/plugin         : 19.6.0-beta.6
@nx/react          : 19.6.0-beta.6
@nrwl/tao          : 19.6.0-beta.6
@nx/web            : 19.6.0-beta.6
@nx/webpack        : 19.6.0-beta.6
typescript         : 5.5.3
---------------------------------------
Registered Plugins:
@nx/eslint/plugin
@nx/cypress/plugin
@nx/jest/plugin
---------------------------------------
Community plugins:
@ngrx/component-store : 18.0.2
@ngrx/effects         : 18.0.2
@ngrx/entity          : 18.0.2
@ngrx/operators       : 18.0.2
@ngrx/router-store    : 18.0.2
@ngrx/store           : 18.0.2
@ngrx/store-devtools  : 18.0.2
---------------------------------------
Local workspace plugins:
         my-plugin

Failure Logs

No response

Package Manager Version

No response

Operating System

• [X] macOS
• [ ] Linux
• [ ] Windows
• [ ] Other (Please specify)

Additional Information

Migrating to Nx 19 made some of our builds in CI break. It has an erratic behaviour since it depends of the number of tasks being dispatched to Nx. I had to match the task graph we had in the failing repo.

In the MacOS that ran this (M2 with 8+4 cores), up to --parallel=4 works OK. Then:

• With 5, one job fails
• With 6 and 7, two jobs fail
• With >=8, all three jobs fail

However it's not 100% the same in another repo that also contains the fake/ folder. The trend is the same, though: the more tasks Nx executes, and the more in parallel it attempts to do it, the less chances that the tasks get the env vars.

[mpsanchis]

I would be willing to contribute to the fix, maybe with a bit of guidance/explanations about the task executor and environment variable loading. My assumption is that reading the .env files is not completed before all tasks are dispatched.