vercel / turborepo

Build system optimized for JavaScript and TypeScript, written in Rust
https://turbo.build/repo/docs
MIT License
26.39k stars 1.84k forks source link

Docs: Just-In-Time Workspace dependency change does not invalidate cache #8435

Closed psteinroe closed 5 months ago

psteinroe commented 5 months ago

What is the improvement or update you wish to see?

I have a pnpm workspace with just in time packages. If I only do changes to a package, the apps that depend on that package are not invalided. How can I make sure that they are? I expect turbo to check git changes also for workspace deps.

Is there any context that might help us understand?

I dont know if this an issue or a bug.

Does the docs page already exist? Please link to it.

No response

anthonyshew commented 5 months ago

Can you share your turbo.json?

psteinroe commented 5 months ago

sure! this is specifically about the deploy script.

{
  "$schema": "https://turborepo.org/schema.json",
  "pipeline": {
    "@hellomateo/supabase#start:lean": {
      "outputs": [],
      "cache": false
    },
    "build": {
      "dependsOn": ["^build"],
      "outputs": ["dist/**", ".next/**"]
    },
    "i18n:sync-missing": {
      "outputs": [],
      "cache": false
    },
    "i18n:sync": {
      "outputs": [],
      "cache": false
    },
    "prepare-build:node": {
      "outputs": ["package.json"],
      "cache": false
    },
    "build:node": {
      "dependsOn": ["^build:node"],
      "outputs": ["dist/**", ".next/**"]
    },
    "@hellomateo/zapier#generate": {
      "dependsOn": ["@hellomateo/supabase#start:lean"]
    },
    "@hellomateo/supabase#generate": {
      "dependsOn": [
        "@hellomateo/supabase#start:lean",
        "@hellomateo/supabase-to-zod#build"
      ],
      "outputs": ["coverage/**"]
    },
    "test": {
      "dependsOn": ["^build"],
      "outputs": ["coverage/**"]
    },
    "@hellomateo/google-notifications-service#test": {
      "dependsOn": ["@hellomateo/supabase#start:lean", "^build"],
      "outputs": ["coverage/**"]
    },
    "@hellomateo/api-service#test": {
      "dependsOn": ["@hellomateo/supabase#start:lean", "^build"],
      "outputs": ["coverage/**"]
    },
    "@hellomateo/integration-service#test": {
      "dependsOn": ["@hellomateo/supabase#start:lean", "^build"],
      "outputs": ["coverage/**"]
    },
    "@hellomateo/webhooks-service#test": {
      "dependsOn": ["@hellomateo/supabase#start:lean", "^build"],
      "outputs": ["coverage/**"]
    },
    "@hellomateo/fastify-graphile-worker#test": {
      "dependsOn": ["@hellomateo/supabase#start:lean", "^build"],
      "outputs": ["coverage/**"]
    },
    "@hellomateo/supabase#test": {
      "dependsOn": ["@hellomateo/supabase#start:lean", "^build"],
      "outputs": []
    },
    "clean": {
      "outputs": [],
      "cache": false
    },
    "start:lean": {
      "outputs": []
    },
    "lint": {
      "dependsOn": ["format:check"],
      "outputs": []
    },
    "@hellomateo/supabase#lint": {
      "dependsOn": ["@hellomateo/supabase#start:lean", "format:check"],
      "outputs": []
    },
    "lint:report": {
      "dependsOn": ["format:check"],
      "outputs": ["lint-results/**"]
    },
    "@hellomateo/supabase#lint:report": {
      "dependsOn": ["@hellomateo/supabase#start:lean", "format:check"],
      "outputs": []
    },
    "lint:fix": {
      "dependsOn": ["format:write"],
      "outputs": []
    },
    "@hellomateo/supabase#lint:fix": {
      "dependsOn": ["format:check"],
      "outputs": []
    },
    "format:write": {
      "outputs": []
    },
    "format:check": {
      "outputs": []
    },
    "typecheck": {
      "dependsOn": ["^build"],
      "outputs": []
    },
    "preview": {
      "dependsOn": ["^build"],
      "outputs": []
    },
    "cleanup-preview": {
      "outputs": []
    },
    "deploy": {
      "outputs": []
    },
    "deploy:native": {
      "outputs": []
    },
    "dev": {
      "cache": false
    }
  },
  "globalDependencies": [
    "package.json",
    ".npmrc",
    "pnpm-lock.yaml",
    "pnpm-workspace.yaml"
  ],
  "globalEnv": [
    "NODE_ENV",
    ...
  ]
}
anthonyshew commented 5 months ago

Which tasks are hitting cache that you're not expecting?

psteinroe commented 5 months ago

I have an app1 that has a dependency on package1 with "package1: workspace:*, and when I do changes only in package1, app1 is not deployed. the package is a just-in-time package.

my expectation is that turbo checks git changes not only in app1 but also in all internal workspace dependencies.

anthonyshew commented 5 months ago

Turborepo does indeed use Internal Package relationships to create its Package Graph and checks git changes. However, you'll still need to ensure that you have dependsOn relationships between tasks to instruct Turborepo about the dependencies in the Task Graph. Some notes on those graphs are here: https://turbo.build/repo/docs/core-concepts/package-and-task-graph

It's more likely that there's a missing dependsOn relationship that isn't creating the relationship you're looking for in your graph, which is why I asked which tasks were showing the unexpected behavior.

This page details how to create those relationships - but I'm realizing we need to add a debugging section. Here are some options for you while I work on that:

psteinroe commented 5 months ago

thanks for the elaborate response! I think I understand the problem now. Our workspace packages do not have a deploy script, just the apps. For turbo to include the packages into the graph for the app, I need to add a no-op script to the packages and make deploy depend on it?

EDIT: Can confirm that adding the no-op deploy to all packages does the trick.

anthonyshew commented 5 months ago

Awesome! Glad you got it straightened out.

psteinroe commented 5 months ago

for anyone coming here: I also added ^deploy into dependsOn for the deploy task