Migrate from Yarn to pnpm

[Desttro]

The documentation is not yet rewritten, I would also like to configure turbo and scripts properly.

GitHub workflows are ready for backend.yml and next.yml Backend ends up with an error (but runs successfully), currently being addressed in https://github.com/pnpm/pnpm/issues/6234

I would be glad for ideas, comments and advice from more experienced developers than me.

[Desttro]

To be honest, this is my first contribution to an opensource repository. Can we still work on the PR now, or should I cancel it and create it when it's ready?

[Desttro]

I left some comments, I also have issues installing the project. I am using pnpm version: 7.13.1 What version are you on? Is there anything else I need to install?

Oh sorry, yes you must use the latest 8.6.6, it is defined in the main package.json:

  "engines": {
    "node": ">=18.16.0",
    "pnpm": ">=8.6.6"
  },
  "packageManager": "pnpm@8.6.6",

there is also a preinstall to ensure that only pnpm can be used

"preinstall": "npx only-allow pnpm",

[timothymiller]

Backend ends up with an error (but runs successfully), currently being addressed in https://github.com/pnpm/pnpm/issues/6234

The link is not accessible for me. Has it been resolved?

[Desttro]

Backend ends up with an error (but runs successfully), currently being addressed in https://github.com/pnpm/pnpm/issues/6234

The link is not accessible for me. Has it been resolved?

Sorry for the wrong URL, the correct one is here: https://github.com/pnpm/pnpm/issues/6234

The workaround mentioned in the thread didn't work for me.

[albbus-stack]

I've tested this and it works apart from the github action!! 👍

[timothymiller]

To be honest, this is my first contribution to an opensource repository. Can we still work on the PR now, or should I cancel it and create it when it's ready?

Let's keep it open and keep iterating on it until it works 😊

[albbus-stack]

Backend ends up with an error (but runs successfully), currently being addressed in https://github.com/pnpm/pnpm/issues/6234

The link is not accessible for me. Has it been resolved?

Sorry for the wrong URL, the correct one is here: pnpm/pnpm#6234

The workaround mentioned in the thread didn't work for me.

Check out https://github.com/actions/setup-node/blob/main/docs/advanced-usage.md#caching-packages-data

It includes only the cache: 'pnpm' option without having to setup anything else.

[Desttro]

I also found that I need to add dependencies that Yarn doesn't need. For example, I need to add @t4/api to app for everything to work seamlessly. For example, code completion for tRPC doesn't work without this in VS Code.

Or as another example, we needed to add the @t4/ui dependency for next-app.

But maybe it could be solved by setting the correct paths in tsconfig.json (just guessing)

here is an excerpt from swarmion.dev:

Pnpm is more strict than Yarn when dealing with dependencies. This is a good thing, because it makes package configurations self-sufficient.

However, migrating from yarn to pnpm, you may find that you miss some dependencies in your packages for this reason. In this case:

add the missing types packages for obvious ones dig into dependencies to find out source issues in less obvious ones Most often it should be an issue with peer dependencies that are not properly satisfied.

excerpt from dev.to:

Important! You need to keep in mind that pnpm doesn’t use dependency hoisting:

When installing dependencies with npm or Yarn Classic, all packages are hoisted to the root of the modules directory. As a result, source code has access to dependencies that are not added as dependencies to the project. By default, pnpm uses symlinks to add only the direct dependencies of the project into the root of the modules directory. pnpm

In practice it means that if you have a package A that imports a package B (import something from 'B') but doesn’t explicitly specify B in the dependencies or devDependencies, then the execution will fail.

[Desttro]

Backend ends up with an error (but runs successfully), currently being addressed in https://github.com/pnpm/pnpm/issues/6234

The link is not accessible for me. Has it been resolved?

Sorry for the wrong URL, the correct one is here: pnpm/pnpm#6234 The workaround mentioned in the thread didn't work for me.

Check out https://github.com/actions/setup-node/blob/main/docs/advanced-usage.md#caching-packages-data

It includes only the cache: 'pnpm' option without having to setup anything else.

Today I plan to solve workflows with pnpm. Action @setup/node has a built-in cache option, but it lacks an opportunity regarding cache persistence as mentioned here

[albbus-stack]

Backend ends up with an error (but runs successfully), currently being addressed in https://github.com/pnpm/pnpm/issues/6234

The link is not accessible for me. Has it been resolved?

Sorry for the wrong URL, the correct one is here: pnpm/pnpm#6234 The workaround mentioned in the thread didn't work for me.

Check out https://github.com/actions/setup-node/blob/main/docs/advanced-usage.md#caching-packages-data It includes only the cache: 'pnpm' option without having to setup anything else.

Today I plan to solve workflows with pnpm. Action @setup/node has a built-in cache option, but it lacks an opportunity regarding cache persistence as mentioned here

Now the backend one works already, at least for me. Install time goes from 1:40 to 40 seconds.

[Desttro]

Now the backend one works already, at least for me. Install time goes from 1:40 to 40 seconds.

Also, the Post Setup pnpm cache step ends without error? It ends with an error on my part.

[albbus-stack]

Now the backend one works already, at least for me. Install time goes from 1:40 to 40 seconds.

Also, the Post Setup pnpm cache step ends without error? It ends with an error on my part.

No errors on my part with the previous iteration. We have to resolve this by running a pnpm install locally and pushing the new pnpm-lockfile.yaml

Run pnpm install --frozen-lockfile --prefer-offline
Scope: all 9 workspace projects
 ERR_PNPM_OUTDATED_LOCKFILE  Cannot install with "frozen-lockfile" because pnpm-lock.yaml is not up to date with apps/vscode/package.json

Note that in CI environments this setting is true by default. If you still need to run install in such cases, use "pnpm install --no-frozen-lockfile"

    Failure reason:
    specifiers in the lockfile ({}) don't match specs in package.json ({"@types/glob":"^8.0.0","@types/mocha":"^10.0.1","@types/node":"16.x","@types/vscode":"^1.74.0","@typescript-eslint/eslint-plugin":"^5.45.0","@typescript-eslint/parser":"^5.45.0","@vscode/test-electron":"^2.2.0","eslint":"^8.28.0","glob":"^8.0.3","mocha":"^10.1.0","typescript":"^4.9.3"})
Error: Process completed with exit code 1.

Also remove the | tr -d '\n', it works fine without that and it shouldn't be there. (If that command actually worked all the ns in the path would be annihilated, fortunately that doesn't even work)

[albbus-stack]

Seems to work for me, do you still encounter the Template is not valid error? @Desttro

[Desttro]

Everything should be ready. I'm just not sure about the vscode extension and cli (I'd need someone to test it). I keep getting a Template is not valid error on backend deployment, but it runs correctly.

Maybe it's a problem with me, maybe we're missing a token in secrets if we need it. If it's working for you @albbus-stack, it looks like a local problem.

[albbus-stack]

Everything should be ready. I'm just not sure about the vscode extension and cli (I'd need someone to test it). I keep getting a Template is not valid error on backend deployment, but it runs correctly.

Maybe it's a problem with me, maybe we're missing a token in secrets if we need it. If it's working for you @albbus-stack, it looks like a local problem.

Update the pnpm-lockfile.yaml so we get rid of the error when using --frozen-lockfile then we should be good and I can test the whole thing back locally and on my fork.

[Desttro]

Everything should be ready. I'm just not sure about the vscode extension and cli (I'd need someone to test it). I keep getting a Template is not valid error on backend deployment, but it runs correctly. Maybe it's a problem with me, maybe we're missing a token in secrets if we need it. If it's working for you @albbus-stack, it looks like a local problem.

Update the pnpm-lockfile.yaml so we get rid of the error when using --frozen-lockfile then we should be good and I can test the whole thing back locally and on my fork.

Updated. Do you get a notification when I push a new commit? I better write this comment to be sure.

[timothymiller]

LGTM let's ship it! 🚀 Will test it deployments in the main branch