Support for yarn berry (v2)

[PaulRBerg]

Berry is yarn's v2 release, which is disruptive in many regards, one of which being the transition from "node_modules" to a new plug-n-play system that caches dependencies more efficiently.

Since there is no "node_modules" folder anymore, some of vscode-solidity's features will cease to behave correctly.

1. The local node compiler, which picks up the solc executable from node_modules
2. The "packageDefaultDependenciesDirectory" setting, which is essential for making this extension work with project that import contracts from third-party libraries like @openzepplin/contracts

While there are workarounds for both points (use a remote compiler and copy-and-paste the OZ contracts into your project), I wanted to open the issue anyways, in case more people stumble upon it and want to signal their support.

[juanfranblanco]

Thanks @PaulRBerg for this! surely the openzeppelin is going to hit more users.

[Lopol2010]

Hey guys, is there a way to have Go-To-Definition to work with yarn berry? Still to copy-paste? After I did a basic benchmark, I don't understand why many still use npm. I don't want to get back to npm NEVER. BTW my results if someone is curious:

Running install command:

yarn v2: | ~20s                    | 109 MB | lack of support among packages
pnpm   : | ~25s                    | 270 MB | don't find issues
yarn v1: | ~70s or ~ 45s cached    | 296 MB | don't find issues
npm  v8: | ~1m 20s or ~45s cached  | 390 MB | don't find issues

[juanfranblanco]

@Lopol2010 have you tried using "remappings.txt" https://github.com/juanfranblanco/vscode-solidity#remappings?

[Lopol2010]

@juanfranblanco thanks for the suggestion, though it seems to not work: This is remapping: :@uniswap=zip:/.yarn/cache/@uniswap-v2-core-npm-1.0.1-036e4e47ef-eaa118fe45.zip/node_modules/@uniswap And import: import "@uniswap/v2-core/contracts/interfaces/IUniswapV2Factory.sol"; Error I get:

To simplify things for you, I want to mention that packages live in .yarn/cache in zip archives. And there is Zip FS VSCode extension which is "maintained as part of the Yarn toolchain", it adds zip: URLs support.

[juanfranblanco]

Ohh thanks! That shows you my ignorance about the zipping. Yes, that is not going to work, in the current form. The extension Zip FS VSCode won't help as this needs to be extracted before hand to enable to navigate to a file, read it etc. So mainly everything uncompressed and in a folder (which might be temp). Also zip: is not something that will be supported by the compiler later on if they are given the remappings.

[juanfranblanco]

The only thing I can think of is to have something that simplifies the extracting process in the current project. But open to suggestions.

[Lopol2010]

Oh yeah, forgot that 'remappings' is the compiler's thing. Well, I'm a noob at extensions, my only suggestions is:

1. To have a separate mapping specifically for Go-to-Definition.
2. To use yarn itself somehow, from your extension. I think good starting point to consider is .pnp.cjs file. Here is some info about it from their docs:

Yarn generates a single .pnp.cjs file that needs to be installed for Node to know where to find the relevant packages.

And here is the docs describing API of this thing, as I understand you can get package names and paths on disk from it: https://yarnpkg.com/advanced/pnpapi

(edit) and this file is generated and managed by yarn automatically when you install packeges

[cameel]

The extension Zip FS VSCode won't help as this needs to be extracted before hand to enable to navigate to a file, read it etc. So mainly everything uncompressed and in a folder (which might be temp). Also zip: is not something that will be supported by the compiler later on if they are given the remappings.

Actually, the compiler does have a mechanism that would let you support zip:/-style imports - the import callback. The callback gets the import path and based on it you can detect that the file should come from a zip file and extract it on the fly. Or even just pass on the request to whatever API VS uses for this. The callback must simply return a piece of source code and the compiler does not care where it came from.

One downside of this would be that it would not be fully transparent - you'd get slightly different bytecode. You'd get a different metadata hash when compiling the same source coming from a .zip because the zip:/ part must be in the import part. Even if you "hide" it behind a remapping because the compiler adds all remappings to the metadata.

Another one would be that other frameworks do not support this and it's also only doable with solc-js, not the native compiler.

[juanfranblanco]

Thinking about the whole delivery of contracts in a zip file, it will be a generic nightmare, smart contracts are not just a random library you just trust, as you will want to inspect the contracts you are working on, navigate to them (go to definition), find issues with them (ie different compiler versions), auditing (regardless if it has been deployed, verified or not, they need to be matched to existing audits), creating a single file or multiple for verification (ie with etherscan)

[cameel]

Yeah, agreed on that. The compiler is not the only component that needs to access the files so having them just cached somewhere in an unpacked form would be much simpler and probably better overall.

Just wanted to say that as far as the compiler goes, it's not an impossible hurdle. Doing it with callback+remapping is actually pretty similar to what yarn does for JS files with patching require to be able to find the files in the new location without changing imports (Packages are stored inside Zip archives: How can I access their files?).

This will be a problem not only for vscode-solidity so it'll be interesting to see how frameworks like Hardhat (https://github.com/nomiclabs/hardhat/issues/586) and Truffle will handle this. If this way of storing packages gets popular enough, it would be a good argument to support this use case better in the compiler, just as we recently did with --include-path option to better support packages installed outside of project dir.

[Animalmix55]

Hardhat now appears to behave nicely when used inside of a yarn workspace in PnP mode (nodeLinker: pnp) . Has your team made any progress in the pursuit of leveraging ZipFS to support PnP functionality. As of now, it appears as though using PnP means that one cannot expect any help from the IDE when writing Solidity.

[juanfranblanco]

Hi @Animalmix55 , I have not looked at that, the packages need to be unzip so they can be part of the project as they need to be scanned, parsed, pass to the compiler etc. My guess the simplest thing will be to have an unpacked folder.

[juanfranblanco]

What do you think? If you want to look at a package source, navigate to its files etc...