Consider splitting into multiple packages

[ethanfrey]

Hi

First off, I want to thank and congratulate you on this library. We were looking at web3 and found both the implementation lacking and started building our own typescript library on top of the json rpc interface. However, recently I found your library which is far more complete and of quite high quality. Nice job especially for a mostly solo effort.

I started Integrating this package into some of our code, which worked well, but I was wondering about the dependencies. We don't mirror the web3 api and actually work on a generic multi blockchain api, with some key differences, especially in key management. So, we only need a part of this code. But that brings a number of dependencies for parts we don't use.

In particular, eth-lib and contract/abi* are quite useful standalone.

I wondered if you considered a lerna monorepo approach to split into multiple package, with the same high level entry point.

If you are interested, I can make a series of pr doing a gradual transition. But I do not want to undertake such a task if you do not like the idea.

In any case, thank you for publishing this library.

[xf00f]

@ethanfrey Thanks for appreciating the library.

I'm not hugely keen to split it up into smaller packages. One of the things I didn't like about web3.js was the fact it ends up as many different packages, most of which didn't make sense in a standalone scenario. The addition of lerna just seemed to clutter and complicate things a bit.

I take your point however. Since introducing the EvmProvider there's a few dependencies introduced that are only needed if using that somewhat niche module. (rust.bn, levelup, leveldown, and bigint-buffer). I was considering taking this out into it's own repository as including is probably overkill for many users.

In your particular case I would point you to: https://github.com/MaiaVictor/eth-lib, parts of which I actually lifted into web3x as web3.js depended on it. It's my intention to remove this by refactoring and inlining the code into the surrounding web3x code. That particular module is not part of the public api you see.

r.e. The contract abi code, this is somewhat standalone but still requires utils and formatters. In your case it maybe easiest to just lift the code you need into your own application, and not to depend on web3x at all?

I'm not closed off to the idea of lerna. Maybe it solves the EvmProvider issue neatly. I don't know enough about lerna and it's tradeoffs, but it's certainly additional complexity and I don't want to commit to it immediately. I generally favour keeping things as simple as possible. If you're using web3x in a node.js background the additional dependencies are probably negligible. If using it to build a frontend app the unused code should all be tree-shaken away. Is there some other concern about unwanted dependencies I'm missing?

[ethanfrey]

Hi @xf00f and thanks for the quick reply.

As to code lifting, I started considering that, but the library is LGPL and our application is apache, which is fine (I believe) if we include web3x as a dependency and list it, but not okay if we lift code.

As to dependencies, besides bundle size, which I believe you rightly explain. We also try to limit dependencies in the framework of blockchain applications to limit possible attack vectors. There have been a few exploits by a corrupted dependency, and limiting those limits the attack surface.

However, you do have a pretty clean dependency graph and I do agree that the whole many repo approach of the original ethereum js code is a nightmare to navigate and understand.

I will use the code as is.

[xf00f]

@ethanfrey

I agree that limiting the attack vectors is important. It was one of my main aims when originally porting web3.js. I've subsequently let a few more dependencies creep back in so will certainly consider options to mitigate this.

r.e. the license, it is an issue. I'm not happy it's LGPL and not MIT. Large parts of the codebase are in fact completely new and could be MIT, including the contract-abi code. I will go through the codebase and make modules that are clearly not derived from web3.js MIT at some point in the near future.

[ethanfrey]

Thank you.

If you do ever look at splitting into multiple packages, I think the monorepo approach is far easier to develop and maintain than the several independent repo mess. We have not found it to be an issue with iov-core - look under packages. It also allows a high degree of independence (requiring dependencies, license, publishing or not, etc)

In such a case, I am more than happy to lend a hand doing some of the work with the scaffolding to make the transition easier for you.