Monorepository

[dilame]

Hello, dear @gcanti ! I highly value your work. I see there was a question about monorepo in the previous issue. To be honest, this also was my first thought when i first time looked at the new fp-ts project structure – this should have been organized as monorepository.

As far as i know you came to TS from another languages, so let me tell you about monorepository pattern and how to cook it nowadays. I organize my private code as monorepos for more than 3 years, so i have experience with it that i would like to share with fp-ts team.

There are some advantages to use monorepo for fp-ts instead of multirepo. First of all is issue tracking – It is easier to maintain the issues related to fp-ts in one place.

Another thing, right now everytime you release new @fp-ts/core version – you should update it's version in dependants @fp-ts/data, @fp-ts/schema, etc, so you end up with commits like this .

A lot of such commits indicates that these packages are part of one codebase rather than completely independent codebases. In short – it's monorepo. With monorepo style you will always work with the latest versions of all the packages. You also get the whole ecosystem source code in one IDE project.

You will be able to setup the publishing pipeline once for all the packages. One eslint settings, one tsconfig, one prettier, one everything.

For the end users it is much easier to learn the source code of all the ecosystem in one monorepo. You will not spray the attention of users. You will get a single, valuable, active spot. You will be able to unify the documentation and have it under one URL.

In short – you will get all the benefits of a single codebase for modules that by it's nature forms a single codebase, but publish it as separated NPM modules, because it's a separated NPM modules by it's nature:)

Now let me tell you a little about technical part of this solution. The NodeJS/TS ecosystem have significantly improved the monorepo tooling last years. You no more need a lerna or even yarn for building the projects – tsc supports it out-of-the-box with project references feature – you can build the whole project by running tsc -b from the root – TypeScript will compile all the separated packages in it's own directories. It's super native now. You will probably still need yarn or lerna for publishing the packages to NPM, but it's not a big deal.

Yarn berry also have a native workspaces feature designed specifically for monorepo. It helps to manage dependencies.

I already have TS project references configured in a right way in my project, so would be glad to help you with it and answer all the questions if you would like to apply monorepo for fp-ts.

Wish all the best to the fp-ts!

[arthurgubaidullin]

@dilame Thank you for the great proposal! I agree that it is important for all packages to be in a single, central location, regardless of the specific implementation of a monorepo. I fully support this approach.

[kalda341]

I too think a monorepo is the way to go. I'd also highly recommend pnpm as a package manager for managing a monorepo after having tried pretty much everything else.

[adrian-gierakowski]

I'd also highly recommend pnpm as a package manager for managing a monorepo

@kalda341 What’s the advantage over yarn berry?

[kalda341]

Workspaces are very easy to set up, a better solution for node_modules than hoisting (which had previously been a big issue for us), better CLI for working with monorepos, faster installs.

[adrian-gierakowski]

better solution for node_modules than hoisting

what’s the solution and why is it better?

[kalda341]

https://pnpm.io/symlinked-node-modules-structure

The biggest advantage I've encountered is disallowing importing things that aren't specified in the specific package's package.json. Prior to using pnpm I had issues where dependencies would work in the context of the monorepo, but not when published as some of the dependencies weren't in the specific package.json (but were in another).

[adrian-gierakowski]

@kalda341 you must be comparing with yarn 1.x. Yarn berry (3.x) with pnp linker does’t do hoisting and is also very strict about dependencies.

[aniravi24]

+1 on having a monorepo setup!

Monorepos are great for all the reasons mentioned by OP.

Some examples of monorepo setups I like: mantine and remix

I also found Nx to be incredibly useful for my projects!

[mikearnaldi]

Monorepos while being "cool" for app projects usually end up being messy for oss libraries, each repository has self containing issues and management segmented which is overall a good thing. Effect started as monorepo and migrated out due to CI taking way too long and repo management becoming messy.

In regards to the yarn vs pnpm discussion overall the different strategy is not only in linking but also in scoping, while yarn hoists monorepo dependencies at the root pnpm doesn't also in my experience it is significantly faster and less complicated (yes a properly configured yarn 3 project could be roughly equivalent to a default pnpm one).

[aniravi24]

@mikearnaldi In the interest of speed and focusing more time on development/documentation and less time on build tooling, I think you and @gcanti should do whatever you're comfortable with as far as the build tooling is concerned for now. Getting the libraries out there for people to use is more important than anything else.

With that said, while I agree with you that monorepos have certain benefits that are more obvious for applications, I think the pros significantly outweigh the cons if set up properly and most of the challenges you mentioned are basically solved problems with modern package managers + monorepo tools like nx or turborepo (project/issue management is probably best solved with the proper GitHub issue templates, tags, etc.)

Versioning, dependency management/analysis, linting/formatting, doc generation, caching, etc. look easier with a monorepo setup than the alternatives.

For whatever it's worth, I think the packages should continue maintaining the org prefix (i.e. @effect/ or @fp-ts/) cause if someone is eventually able to contribute to helping set up a monorepo, the package names don't need to change.

[mikearnaldi]

@mikearnaldi In the interest of speed and focusing more time on development/documentation and less time on build tooling, I think you and @gcanti should do whatever you're comfortable with as far as the build tooling is concerned for now. Getting the libraries out there for people to use is more important than anything else.

With that said, while I agree with you that monorepos have certain benefits that are more obvious for applications, I think the pros significantly outweigh the cons if set up properly and most of the challenges you mentioned are basically solved problems with modern package managers + monorepo tools like nx or turborepo (project/issue management is probably best solved with the proper GitHub issue templates, tags, etc.)

Versioning, dependency management/analysis, linting/formatting, doc generation, caching, etc. look easier with a monorepo setup than the alternatives.

For whatever it's worth, I think the packages should continue maintaining the org prefix (i.e. @effect/ or @fp-ts/) cause if someone is eventually able to contribute to helping set up a monorepo, the package names don't need to change.

We don't want to mix repositories that don't have to be mixed, co-locating schema with schema interpreters is fine but co-locating effect with query and node and other 100 projects really isn't, each repo should have its own scope to be better focused

[dilame]

I will talk for myself from my DX point of view. As authors mentioned before – the fp-effect ecosystem libraries tends to not have a documentation, because the type definitions is a documentation itself. And actually it's really hard to find any sort of documentation except the official typings-based docs. I thought I'd come in from the other side and use the sources instead. But now i should end up having an uncountable amount of IDE projects in order to learn everything i need to start developing. Would be really convenient to open all the sources in one IDE view and navigate through it.

[mikearnaldi]

I will talk for myself from my DX point of view. As authors mentioned before – the fp-effect ecosystem libraries tends to not have a documentation, because the type definitions is a documentation itself. And actually it's really hard to find any sort of documentation except the official typings-based docs. I thought I'd come in from the other side and use the sources instead. But now i should end up having an uncountable amount of IDE projects in order to learn everything i need to start developing. Would be really convenient to open all the sources in one IDE view and navigate through it.

The reason there is no doc is that the libraries are not ready, versions are out for contributor's usage. There will be doc targeting beginners. Putting everything in one repo is simply out of discussion.