[turborepo] Concurrency options are ignored when commands are chained

[osilkin98]

What version of Turborepo are you using?

1.8.8

What package manager are you using / does the bug impact?

Yarn v2/v3 (node_modules linker only)

What operating system are you using?

Linux

Describe the Bug

Suppose I have a package.json file in my root like the following:

{
  "scripts": {
    "dev": "npm run with-env turbo dev  --filter=@acme/app --filter=@acme/landing-page --concurrency=11",
    "dev:works": "turbo dev  --filter=@acme/app --filter=@acme/landing-page --concurrency=11",
    "with-env": "dotenv -e .env --"
  }
}

Running dev returns the following error:

[I] case0 ) yarn dev                                                                 19:09:25
yarn run v1.22.19
$ npm run with-env turbo dev  --filter=@acme/app --filter=@acme/landing-page --concurrency=11

> with-env
> dotenv -e .env -- "turbo" "dev"

 WARNING  failed to contact turbod. Continuing in standalone mode: connection to turbo daemon process failed. Please ensure the following:
        - the process identified by the pid in the file at /tmp/turbod/86d18445fc6dd7c0/turbod.pid is not running, and remove /tmp/turbod/86d18445fc6dd7c0/turbod.pid
        - check the logs at /home/oleg/.local/share/turborepo/logs/86d18445fc6dd7c0-case0.log
        - the unix domain socket at /tmp/turbod/86d18445fc6dd7c0/turbod.sock has been removed
        You can also run without the daemon process by passing --no-daemon
 ERROR  run failed: error preparing engine: Invalid persistent task configuration:
You have 10 persistent tasks but `turbo` is configured for concurrency of 10. Set --concurrency to at least 11
Turbo error: error preparing engine: Invalid persistent task configuration:
You have 10 persistent tasks but `turbo` is configured for concurrency of 10. Set --concurrency to at least 11
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

However, running yarn:dev works as expected:

[I] case0 ) yarn dev:works                                                           19:11:38
yarn run v1.22.19
$ turbo dev  --filter=@acme/app --filter=@acme/landing-page --concurrency=11
 WARNING  failed to contact turbod. Continuing in standalone mode: connection to turbo daemon process failed. Please ensure the following:
        - the process identified by the pid in the file at /tmp/turbod/86d18445fc6dd7c0/turbod.pid is not running, and remove /tmp/turbod/86d18445fc6dd7c0/turbod.pid
        - check the logs at /home/oleg/.local/share/turborepo/logs/86d18445fc6dd7c0-case0.log
        - the unix domain socket at /tmp/turbod/86d18445fc6dd7c0/turbod.sock has been removed
        You can also run without the daemon process by passing --no-daemon
• Packages in scope: @acme/app, @acme/landing-page
• Running dev in 2 packages
• Remote caching enabled
@acme/app:dev: cache miss, executing 1e112c868d139d41
@acme/landing-page:dev: cache miss, executing 2bde317a7485bfb7
@acme/app:dev: $ next dev
@acme/landing-page:dev: $ astro dev --port 3001
@acme/app:dev: ready - started server on 0.0.0.0:3000, url: http://localhost:3000
@acme/app:dev: info  - Loaded env from /home/oleg/case0/apps/nextjs/.env
@acme/app:dev: warn  - You have enabled experimental feature (swcPlugins) in next.config.mjs.
@acme/app:dev: warn  - Experimental features are not covered by semver, and may cause unexpected or broken application behavior. Us

Expected Behavior

I expect yarn dev to have the same output as yarn dev:works and be able to chain commands such as npm run with-env turbo dev.

To Reproduce

Simply create a monorepo with 10 packages or so (they can be simple hello world files and all duplicate) and configure your package.json as indicated above.

Reproduction Repo

No response

[jpgilchrist]

I am also facing the same issue. I was running things from the command line as such

dotenv -e .env.develop -c -- turbo run dev --filter=@acme/directory

which does not work as expected, but the following does:

turbo run dev --filter=@acme/directory

The first command will run for every app, but the latter only runs for @acme/directory as expected, but without environment variables!

EDIT: The good thing is that the environment variables are correct

UPDATE: I've opted for using env-cmd as it works (https://www.npmjs.com/package/env-cmd)

[ezeamin]

Hi. I'm facing the same issue as @jpgilchrist when trying to run the turno run dev command using dotenv and specifying the .env file. The command is:

dotenv -e .env.development turbo run dev --concurrency 18

where --concurrency is not detected by turbo, throwing:

Turbo error: error preparing engine: Invalid persistent task configuration:
You have 13 persistent tasks but `turbo` is configured for concurrency of 10. Set --concurrency to at least 14

[williamhaley]

I came across this today and realized my issue is specifically around dotenv-cli, which requires -- to pass flags to the underlying command

https://github.com/entropitor/dotenv-cli?tab=readme-ov-file#flags-to-the-underlying-command

Based on the other comments using the -e syntax and invoking dotenv as a CLI bin I think it may be the same issue. I don't think dotenv supports a CLI or -e out of the box, but dotenv-cli does.

Bad

Here the --concurrency and --filter flags are ignored

dotenv -e .env turbo run --concurrency=123 dev --filter=./apps/*

Good

Here the --concurrency and --filter flags are honored

dotenv -e .env -- turbo run --concurrency=123 dev --filter=./apps/*

[arlyon]

Looks like this is not an issue with turbo but with other tools. As long as the solution above works, I will close this issue. If that is not the issue please feel free to re-open or open a new one.