Nx Cloud Agents Cache Restore Error

cyrus-za commented 4 months ago

Current Behavior

I am using nx-cloud with a custom agent. It passes just fine, but the post-cache step fails

Expected Behavior

I expect cache-restore step to succeed

GitHub Repo

No response

Steps to Reproduce

I can't do a repro right now. But the logs point to tailwind issues.

Nx Report

N/A

Failure Logs

Starting logs...
/tmp/3456827675:5281
  const error = new ConnectError(message !== null && message !== void 0 ? message : "", code, metadata);
                ^

_ConnectError: [unknown] creating tarball: opening file node_modules/tailwind: open node_modules/tailwind: no such file or directory
    at errorFromJson (/tmp/3456827675:5281:17)
    at next (/tmp/3456827675:5643:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Object.unary (/tmp/3456827675:5612:14)
    at async Object.store (/tmp/3456827675:5070:22) {
  rawMessage: 'creating tarball: opening file node_modules/tailwind: open node_modules/tailwind: no such file or directory',
  code: 2,
  metadata: _Headers [Headers] {
    [Symbol(headers list)]: _HeadersList {
      cookies: null,
      [Symbol(headers map)]: Map(4) {
        'accept-encoding' => { name: 'accept-encoding', value: 'gzip' },
        'content-length' => { name: 'content-length', value: '138' },
        'content-type' => { name: 'content-type', value: 'application/json' },
        'date' => { name: 'date', value: 'Fri, 08 Mar 2024 17:48:59 GMT' }
      },
      [Symbol(headers map sorted)]: null
    },
    [Symbol(guard)]: 'none'
  },
  details: [],
  cause: undefined
}

Node.js v20.9.0

Package Manager Version

No response

Operating System

[ ] macOS
[X] Linux
[ ] Windows
[ ] Other (Please specify)

Additional Information

# agent.yaml
launch-templates:
  linux-medium:
    resource-class: "docker_linux_amd64/medium+"
    image: "ubuntu22.04-node20.9-v3"
    env:
      SELECTED_PM: "pnpm"
      NPM_CONFIG_PREFIX: "/home/workflows/.npm-global"
    init-steps:
      - name: Checkout
        uses: "nrwl/nx-cloud-workflows/v3.1/workflow-steps/checkout/main.yaml"
      - name: Cache restore
        uses: "nrwl/nx-cloud-workflows/v3.1/workflow-steps/cache/main.yaml"
        env:
          KEY: pnpm-lock.yaml
          PATHS: |
            node_modules
            ~/.pnpm-store
          BASE_BRANCH: "main"

      - name: Install pnpm
        script: |
          npm install -g @pnpm/exe@8.7.4

      - name: pnpm Install
        uses: "nrwl/nx-cloud-workflows/v3.1/workflow-steps/install-node-modules/main.yaml"

rarmatei commented 4 months ago

can you please try using v3.5 of the step? "nrwl/nx-cloud-workflows/v3.5/workflow-steps/install-node-modules/main.yaml"

cyrus-za commented 4 months ago

@rarmatei same error. (link removed after Rares read it)

rarmatei commented 4 months ago

Interesting - if you add a step to your agent to try and log what's in node_modules, something like:

      - name: Debug tailwind cache
        script: |
          ls -al node_modules | grep tailwind
          ls -al node_modules/tailwind || true

curious what's in there, and why would it complain about tailwind.

cyrus-za commented 4 months ago

https://cloud.nx.app/cipes/65f82f416bf1200961bdda74?step=347d6341-79bb-4abd-a020-28741d546f1f:post#step-list-pane

rarmatei commented 4 months ago

looks like it's a symlink, maybe that's what causing the issue - I'm trying to reproduce this on my side. Could you either share on here, or at cloud-support@nrwl.io your package.json contents (with any sensitive values, such as the access token, blanked out)? I want to make sure I am installing similar things to you.

cyrus-za commented 4 months ago

{ 
  "dependencies": {
    "@apidevtools/swagger-parser": "^10.1.0",
    "@auth/core": "^0.28.0",
    "@auth/prisma-adapter": "^1.5.0",
    "@aws-sdk/client-s3": "^3.367.0",
    "@aws-sdk/s3-request-presigner": "^3.367.0",
    "@casl/ability": "^6.5.0",
    "@casl/prisma": "^1.4.1",
    "@dnd-kit/core": "^6.0.8",
    "@dnd-kit/modifiers": "^7.0.0",
    "@dnd-kit/sortable": "^8.0.0",
    "@dnd-kit/utilities": "^3.2.2",
    "@emotion/react": "^11.11.1",
    "@floating-ui/react": "^0.25.2",
    "@headlessui/react": "^1.7.15",
    "@hookform/resolvers": "^3.3.4",
    "@phosphor-icons/react": "^2.0.15",
    "@prisma/client": "^5.7.0",
    "@prisma/extension-accelerate": "^0.6.2",
    "@radix-ui/react-accordion": "^1.1.2",
    "@radix-ui/react-checkbox": "^1.0.4",
    "@radix-ui/react-dialog": "^1.0.5",
    "@radix-ui/react-dropdown-menu": "^2.0.6",
    "@radix-ui/react-icons": "^1.3.0",
    "@radix-ui/react-label": "^2.0.2",
    "@radix-ui/react-slot": "^1.0.2",
    "@radix-ui/react-switch": "^1.0.3",
    "@radix-ui/react-toggle-group": "^1.0.4",
    "@segment/snippet": "^4.16.2",
    "@sentry/nextjs": "^7.106.0",
    "@sentry/node": "^7.106.0",
    "@stripe/react-stripe-js": "^2.5.0",
    "@stripe/stripe-js": "^3.0.5",
    "@swc/helpers": "~0.5.2",
    "@t3-oss/env-core": "^0.6.1",
    "@t3-oss/env-nextjs": "^0.6.0",
    "@tailwindcss/typography": "^0.5.9",
    "@tanstack/react-query": "^4.35.0",
    "@tanstack/react-table": "^8.10.1",
    "@trpc/client": "10.45.1",
    "@trpc/next": "10.45.1",
    "@trpc/react-query": "10.45.1",
    "@trpc/server": "10.45.1",
    "@types/papaparse": "^5.3.8",
    "@zodios/core": "^10.9.6",
    "bcryptjs": "^2.4.3",
    "celery-node": "^0.5.9",
    "class-variance-authority": "^0.7.0",
    "clsx": "^1.2.1",
    "command-score": "^0.1.2",
    "common-tags": "^1.8.2",
    "core-js": "^3.33.0",
    "date-fns": "^2.30.0",
    "date-fns-tz": "^2.0.0",
    "emailvalid": "^1.0.4",
    "highlight.js": "^11.8.0",
    "ioredis": "^5.3.2",
    "jotai": "^2.2.2",
    "jotai-devtools": "^0.6.0",
    "koa": "~2.14.2",
    "lodash": "^4.17.21",
    "marked": "^5.1.1",
    "marked-highlight": "^2.0.6",
    "next": "14.1.2",
    "next-auth": "5.0.0-beta.15",
    "node-gyp-build": "^4.6.1",
    "nodemailer": "^6.9.3",
    "openai": "^4.11.0",
    "papaparse": "^5.4.1",
    "react": "18.2.0",
    "react-dom": "18.2.0",
    "react-dropzone": "^14.2.3",
    "react-hook-form": "^7.50.1",
    "react-hotkeys-hook": "^4.4.1",
    "react-icons": "^4.10.1",
    "react-markdown": "^8.0.7",
    "react-pdf": "7.3.1",
    "react-resizable-panels": "^0.0.55",
    "react-syntax-highlighter": "^15.5.0",
    "react-textarea-autosize": "^8.5.3",
    "redis": "^4.6.8",
    "rehype-raw": "^7.0.0",
    "rehype-sanitize": "^6.0.0",
    "remark-gfm": "^3.0.1",
    "stripe": "^14.17.0",
    "superjson": "1.12.4",
    "tailwind": "link:@nx/react/tailwind",
    "tailwind-merge": "^1.14.0",
    "tailwindcss-animate": "^1.0.7",
    "trpc-api": "link:apps/copilot/src/utils/trpc",
    "tslib": "^2.6.0",
    "usehooks-ts": "^2.9.1",
    "uuid": "^9.0.0",
    "zod": "^3.21.4"
  },
  "engine": {
    "node": "^20"
  },
  "packageManager": "pnpm@8.6.6",
  "devDependencies": {
    "@nx-tools/container-metadata": "^5.2.0",
    "@nx-tools/nx-container": "^5.2.0",
    "@nx-tools/nx-prisma": "^5.1.0",
    "@nx/devkit": "18.0.8",
    "@nx/esbuild": "18.0.8",
    "@nx/eslint": "18.0.8",
    "@nx/eslint-plugin": "18.0.8",
    "@nx/js": "18.0.8",
    "@nx/next": "18.0.8",
    "@nx/node": "18.0.8",
    "@nx/playwright": "18.0.8",
    "@nx/react": "18.0.8",
    "@nx/rollup": "18.0.8",
    "@nx/storybook": "18.0.8",
    "@nx/vite": "18.0.8",
    "@nx/web": "18.0.8",
    "@nx/workspace": "18.0.8",
    "@playwright/test": "1.37.1",
    "@storybook/addon-actions": "^8.0.0",
    "@storybook/addon-essentials": "7.6.13",
    "@storybook/addon-interactions": "7.6.13",
    "@storybook/core-server": "7.6.13",
    "@storybook/jest": "0.2.3",
    "@storybook/nextjs": "^7.6.12",
    "@storybook/react": "7.6.13",
    "@storybook/test": "^8.0.0",
    "@storybook/test-runner": "0.13.0",
    "@storybook/testing-library": "0.2.2",
    "@swc-node/register": "1.8.0",
    "@swc/cli": "~0.1.62",
    "@swc/core": "~1.3.95",
    "@testing-library/jest-dom": "^6.4.2",
    "@testing-library/react": "14.0.0",
    "@types/bcryptjs": "^2.4.6",
    "@types/chance": "^1.1.4",
    "@types/common-tags": "^1.8.1",
    "@types/koa": "~2.13.10",
    "@types/lodash": "^4.14.195",
    "@types/marked": "^5.0.1",
    "@types/minimist": "^1.2.3",
    "@types/node": "18.19.14",
    "@types/prettier": "^2.7.3",
    "@types/react": "18.2.24",
    "@types/react-dom": "18.2.9",
    "@types/react-syntax-highlighter": "^15.5.7",
    "@types/segment-analytics": "^0.0.34",
    "@types/sinon": "^10.0.16",
    "@types/uuid": "^9.0.2",
    "@types/webpack-bundle-analyzer": "^4.6.0",
    "@typescript-eslint/eslint-plugin": "6.21.0",
    "@typescript-eslint/parser": "6.21.0",
    "@upstash/cli": "^0.2.0",
    "@vercel/remote-nx": "^1.1.1",
    "@vitejs/plugin-react": "4.2.1",
    "@vitejs/plugin-react-swc": "3.6.0",
    "@vitest/coverage-c8": "~0.32.4",
    "@vitest/coverage-v8": "1.3.0",
    "@vitest/ui": "1.3.0",
    "autoprefixer": "^10.4.14",
    "bun": "^1.0.2",
    "chance": "^1.1.11",
    "checkly": "4.6.1",
    "dotenv": "^16.3.1",
    "esbuild": "^0.19.5",
    "eslint": "8.48.0",
    "eslint-config-airbnb": "^19.0.4",
    "eslint-config-airbnb-base": "^15.0.0",
    "eslint-config-next": "^14.1.0",
    "eslint-config-prettier": "^9.1.0",
    "eslint-import-resolver-typescript": "^3.6.1",
    "eslint-plugin-import": "^2.29.0",
    "eslint-plugin-jsx-a11y": "^6.7.1",
    "eslint-plugin-n": "^16.1.0",
    "eslint-plugin-no-only-tests": "^3.1.0",
    "eslint-plugin-no-relative-import-paths": "^1.5.2",
    "eslint-plugin-playwright": "^0.15.3",
    "eslint-plugin-prettier": "^5.1.3",
    "eslint-plugin-react": "^7.33.2",
    "eslint-plugin-react-hooks": "^4.6.0",
    "eslint-plugin-tailwindcss": "^3.13.0",
    "factory.ts": "^1.4.1",
    "jsdom": "~22.1.0",
    "minimist": "^1.2.8",
    "nx": "18.0.8",
    "nx-cloud": "18.0.0",
    "openapi-zod-client": "^1.16.2",
    "openapi3-ts": "3.0.2",
    "pgvector": "^0.1.4",
    "postcss": "^8.4.24",
    "postcss-import": "^15.1.0",
    "prettier": "^3.2.5",
    "prisma": "^5.7.0",
    "prisma-accelerate-local": "^1.1.2",
    "sinon": "^16.0.0",
    "tailwindcss": "^3.3.2",
    "ts-node": "^10.9.1",
    "ts-node-dev": "^2.0.0",
    "typescript": "5.4.2",
    "vercel": "^31.0.1",
    "vite": "5.0.12",
    "vite-plugin-dts": "~2.3.0",
    "vitest": "1.3.0",
    "vitest-mock-extended": "^1.3.1",
    "webpack": "^5.88.2",
    "webpack-bundle-analyzer": "^4.9.1",
    "zx": "^7.2.3"
  }
}

rarmatei commented 4 months ago

I could repro. It's likely because of the symlink. We'll need to dig more to see if it's a general issue with any symlink inside node_modules or if it's specific to this line "tailwind": "link:@nx/react/tailwind",

In the meantime, I don't know if there's a way to not use the link: syntax, maybe by referencing @nx/react/tailwind directly? It's been a while since I worked on anything Nx core related so I'm not too familiar with it 🙈

cyrus-za commented 4 months ago

I remove the link: prefix and it did not make any difference

rarmatei commented 4 months ago

yea I think you might have to remove the whole tailwind reference from in there - use a combination of "tailwindcss": "^3.3.6", and imports from '@nx/react/tailwind', I'm really not sure though - basically we need to remove that symlink of the tailwind folder, through whatever means 😄 if you did this, and you removed tailwind from your package.json what is the new error? Is another package erroring now?

cyrus-za commented 4 months ago

Ok I removed it. Builds seem to work just fine. What does the @nx/react/tailwind plugin do? I see the nx monorepo uses tailwind but has no reference to it.

cyrus-za commented 4 months ago

Caching seems to now work. Thanks for the help. I'll post here if I run into any new tailwind issues

rarmatei commented 4 months ago

thanks for confirming! I am not sure what that is needed for, we use tailwindcss for our closed-source projects for example.

github-actions[bot] commented 3 months ago

This issue has been closed for more than 30 days. If this issue is still occuring, please open a new issue with more recent context.

nrwl / nx