Support for remappings

[BlinkyStitt]

I am using solc's remapping feature and brownie for my project. Doing either of these cause the vscode-solidity's plugin to fail loading my imports.

It would be really helpful if there were some sort way to include these remappings in the project's settings.

https://solidity.readthedocs.io/en/develop/layout-of-source-files.html?#use-in-actual-compilers

For example, if you clone github.com/ethereum/dapp-bin/ locally to /usr/local/dapp-bin, you can use the following in your source file: import "github.com/ethereum/dapp-bin/library/iterable_mapping.sol" as it_mapping; Then run the compiler: solc github.com/ethereum/dapp-bin/=/usr/local/dapp-bin/ source.sol

There's another plugin that has the feature, but it hasn't been updated in 2 years.

[juanfranblanco]

Still on the todo list, just reporting, github support is going to be a pain as we want to be able to get those files for autocomplete.

[cameel]

You don't really have to add any special support for github or even URLs in general to make URL in imports work. Remapping is a general mechanism that just swaps one prefix with another. These prefixes can be URLs, paths, @package_names or anything else. Does not matter because the only thing the compiler cares about is for the part after = to make sense as a part of the import after the replacement. The only thing you'd need for basic remapping support would be to make the extension aware of how remappings affect paths used in imports.

In the simplest scenario you can leave it up to the user to get files from github and to remap the URL to the local path where the files are available. Brownie solves that by providing a package manager that handles cloning the repo into the right location.

There's also another way to support imports from github - a custom import callback. Remix IDE uses this. It's because it's running in a browser where it's trivial for it to fetch files from arbitrary URLs. Remappings were actually added as an alternative way to handle such imports in the native compiler binary. Since vscode-solidity, unlike Brownie, is using the emscripten binary (via solc-js) just like Remix, both approaches are viable.

[juanfranblanco]

@cameel yeah my issue was with go to definition and auto complete, etc, For this the file from url will need to be downloaded and put in a temporary directory so it can be parsed and navigated to if needed, also if there is an issue with the file itself use it as part of the background compilation, so it can be navigated to as well. I guess here is where the import callback, could be helpful. Finally on the "final" compilation, just let solc do its thing.

[cameel]

@juanfranblanco What you describe would be one way to support URLs. Remapping would be another.

You could let users just define a list of remappings in project settings, which would allow them to get URLs to work "manually" (i.e. by explicitly remappng a given URL with a given local path).

A callback, on the other hand, would be a way to make URLs work automatically, the way you described above, without any configuration on user's part.

Since this issue is about remappings and not about supporting URLs, I'm just suggesting that you could implement one of these without implementing the other immediately. They're separate things and one does not depend on the other.

[merc-0]

any thoughts on moving this higher up the priority list? Without remappings, the extension doesn't play nicely with daptools directory layouts

[cameel]

@merc-0 Just wanted to add that you can work around the lack of remappings by creating symbolic links. To the compiler they basically look like a different name for the same file. That's not a reason not to add support for remappings but at least that's something you can use right away.

[cmeyke]

@merc-0 Just wanted to add that you can work around the lack of remappings by creating symbolic links. To the compiler they basically look like a different name for the same file. That's not a reason not to add support for remappings but at least that's something you can use right away.

How can I do that if I have a remapping like: import "@openzeppelin/contracts/token/ERC721/extensions/ERC721URIStorage.sol"; ?

Downloading the repository and then symlink to it?

[juanfranblanco]

@cmeyke I have not had much time to look into this, you can achieve that already in the settings by providing the path for your "library" / modules. This is the part that will need updated to allow for more "mappings" to be resolved, and linked to the compiler.

Edit, so for you by default this is already resolved (in your example) if your node_modules is at the root of the folder your have opened. If not just set a subfolder etc

[cameel]

@cmeyke Depends on where your copy of OZ is installed. For example if it's in lib/openzeppelin-contracts/, you'd normally have a remapping like @openzeppelin/=lib/openzeppelin-contracts/. You can get the same effect by creating an equivalent symlink:

ln -s lib/openzeppelin-contracts @openzeppelin

Just remember to put the symlink in the root dir of your project (the one that contains src/ and lib/) and not e.g. in src/.

With this I'm assuming that vscode-solidity follows symlinks without resolving them. I haven't checked but that's how the compiler behaves and how frameworks do it. For example if you have the symlink I have shown above, the it should see ERC721URIStorage.sol as @openzeppelin/contracts/token/ERC721/extensions/ERC721URIStorage.sol and not lib/openzeppelin-contracts/contracts/token/ERC721/extensions/ERC721URIStorage.sol.

@juanfranblanco Setting the dependency dir would be enough for OZ and other libraries which are typically installed with npm but there also libraries using dapp-tools' own convention that do need remappings. For example dapp-test contracts are typically installed in lib/dapp-test/src/ but imported with import "ds-test/test.sol". This does require a remapping (or the symlink trick I suggested above).

But this setting actually does make the symlink trick somewhat cleaner. If you set the dependency dir to lib/, you can create symlinks there and this way keep them out of the project root.

BTW, if you need to test this with an actual project, dapptools-template might come in handy. It has minimal amount of code and the typical dapp-tools structure.

[juanfranblanco]

@cameel Yes, thanks, that makes sense, actually the dapptools support was the first original supported by the extension, (dapple) hence the two different folders in the settings. So allowing multiple settings (ie path for node_modules), path for (lib + contracts) might be enough.

Don't you think that it will be a great time to standardise where the project folders, like for example the "remappings.txt" in the "dapptools-template"

[juanfranblanco]

"You could let users just define a list of remappings in project settings, which would allow them to get URLs to work "manually" (i.e. by explicitly remapping a given URL with a given local path)."

I guess combining that with support to "remappings.txt" to start with will be the simplest solution.

[cameel]

Don't you think that it will be a great time to standardise where the project folders, like for example the "remappings.txt" in the "dapptools-template"

I suspect that other tools might prefer to have it or already have it in their own config. For example Brownie has them in brownie-config.yaml. Truffle in truffle-config.js (though they're still buggy: https://github.com/trufflesuite/truffle/issues/2768). And Hardhat does not support remappings and, last time I heard from them, had no plans to add them in the near future (https://github.com/nomiclabs/hardhat/issues/1877). So this again boils down to standardizing the project format :) But I guess supporting remappings.txt directly in vscode-solidity would not hurt. Might actually encourage more people to use it.

The only thing you'd need for basic remapping support would be to make the extension aware of how remappings affect paths used in imports.

By the way, this part is now fully documented: Import remapping. The compiler is actually supposed to be able to do that translation for you automatically but unfortunately the resolved path is missing from the output (https://github.com/ethereum/solidity/issues/11756) - that will probably come in one of the upcoming releases.

[PatrickAlphaC]

This sounds amazing ^

[juanfranblanco]

@cameel @PatrickAlphaC @merc-0 @WyseNynja just checked in the initial support. Not perfect yet as it is driven by the global / workspace settings. I think going deeper will takes us back to the project file / module or the old dapp project file :). Which can contain each remapping, module / context name etc.

Notes:

• Remapping support included in the workspace / global settings, the remapping base path is relative to the workspace root folder.
• Issues at the moment, multiple folders in a workspace will use the same remapping (as it is a workspace setting and for that folder)
• No support for "context", so mainly this uses the workspace folder for all dependencies. Solution for this and above will be find the current "remappings.txt" (or project file) in each folder and use that as the Package remapping configuration. Context name (package name, could be driven from that).

Edit: "remappings.txt" is not supported yet (just in case is not clear)

[PatrickAlphaC]

Thanks so much @juanfranblanco! You're doing amazing work. I've tried a few times to get it to work... perhaps I'm doing something wrong... I'll try again later.

[merc-0]

using remappings.txt sounds like a solid solution to me. The dapptools template already has one in there for compilation

[juanfranblanco]

@merc-0 it is supported now, also @PatrickAlphaC confirmed in another thread it is working now with Brownie.