nrwl / nx

Smart Monorepos ยท Fast CI
https://nx.dev
MIT License
23.28k stars 2.32k forks source link

Nx 11 Plans #3377

Closed vsavkin closed 3 years ago

vsavkin commented 4 years ago

Nx 11 Plans

Nx 9 => Nx 10 Highlights

  1. Computation caching. If your or your team member has already built or tested similar code, Nx will use their results to speed up the command instead of rebuilding or retesting everything from scratch. This is the most important addition to core capabilities of Nx since the tool's creation. It helps build much larger projects.

  2. Custom workspace layouts. The core of Nx always supported custom layouts, but plugins expected the apps and libs folders to be there. Now the layout can be configured in nx.json and is respected by all core plugins. We also created an "OSS" preset to make migration from Lerna easier. A lot of open source projects, including Nx itself, are now built with Nx.

  3. Optimizing performance. Nx collects metadata about a workspace to be able to use caching and code change analysis. This metadata is then stored on disk (in node_modules), so Nx doesn't have to recompute it from scratch every time. For large projects, however, even checking if this metadata is up to date could take hundreds of milliseconds, which is unacceptable. We reworked this to take advantage of git, which resulted in 10x improvements.

  4. Dep graph. We reworked the dep graph feature to handle much larger workspaces. Both, in terms of performance (you can actually render it), and in terms of ergonomics (you can make sense of it).

  5. React. We improved our support for the React ecosystem, including a much more solid support for Next.js. We also added initial support for Gatsby.

  6. Angular. We added support for Angular 10. We also added the npx make-angular-cli-faster command to add the Nx computation cache to any Angular CLI project.

  7. The Cypress, Storybook, Jest, ESLint plugins got updated to make sure they work with the latest versions of the tools.

  8. Community plugins. We made creating Nx plugins easier. npx create-nx-plugins scaffolds a Nx plugin project. We have guides and videos walking you through the process. And now you can add your plugins to the official docs site and to nx list.

  9. nx.dev got more high-quality guides. Plus, we integrated a much better full-text search.

  10. Nx Console--the VS Code plugin for Nx. We fixed bugs and improved performance.

  11. We love the Nx community. The Nx Community Slack is a great place to get your questions answered. You can subscribe to Nx Newsletter to get updates from the core team. "Nx Office Hours" by Brandon Roberts is something I'm personally very happy with. I hope other tech communities do similar stuff.

Nx is Different

The Nx core team spends more time working on large projects built with Nx than on Nx itself. We work with people who aren't familiar with Nx. We work with legacy projects ported to Nx exercising all possible edge cases. Different CI systems, different dev workflows. It's a lot of feedback.

I remember chatting with a contributor to another build tool. I mention that some folks find the tool hard to use. The person replied that it doesn't matter if it's hard to use. "They will use it regardless" And, of course, they didn't.

There is almost nothing in software itself that can keep you honest. It's too malleable. I hope because we work on different projects with so many different people, Nx will continue solving real problems and be pleasant to use.

Nx 11 Plans

Most of what we plan are incremental improvements: making things faster, fixing bugs, clarifying documentation. Every plugin will receive some attention. So, even though you don't see, for instance, Storybook mentioned below, we will improve our Storybook support, fix issues etc..

Node

Nx is already a great tool for building Node apps and libs. You can generate projects. You can test, lint, deploy them. You can reuse code. We are going to solidify the Node story. We are going to add the Node flavour of docs, videos, tutorials. We are planning to add affordances simplifying using Docker, affordances for creating e2e tests for APIs etc..

React

We are going to merge the Gatsby plugin into the main repo. We are planning to add initial support for React Native. We are also going to release more high-quality guides and free video courses made for React folks.

Angular

We are going to add support for Angular 11.

Java

The core of Nx is technology agnostic. At Nrwl, we develop a Java service in our private Nx monorepo. It's part of the same dependency graph. Caching works just as well. We also put a lot of effort in improving the run-commands builder, so you can easily integrate any technology into an Nx workspace. So what are we doing here?

First, we are going to document how to add Java/.Net/Golang/Rust to your Nx workspace. Second, we will add basic affordances for Gradle-based Java projects. So you can add a Spring Boot application to your Nx workspace without having to "figure it out".

Incremental Builds

The Nx distributed computation cache helps you build much larger applications. You don't have to rebuild the whole application on your machine. Instead, you reuse the computation done by your team-mates or CI.

You can already do it with Nx, as shown here. But because WebPack isn't optimized for this use case, it's awkward, and we can do a lot better. So we will focus on making setting up good dev experience for extremely-large applications easy.

yjaaidi commented 4 years ago

OH! Impressive! Great job to all of you!

Is there anything planned for Vue.JS support? This has already being discussed here https://github.com/nrwl/nx/issues/1541 but I'm wondering if the team's position is still the same. ๐Ÿ˜Š

Cammisuli commented 4 years ago

@yjaaidi we actually had some people from the community (thanks to @ZachJW34 and @BuckyMaler) submit a plugin that brought support for Vue. You can check it out here: https://github.com/ZachJW34/nx-plus/tree/master/libs/vue

yjaaidi commented 4 years ago

Thanks @Cammisuli but I was wondering if there were any plans to make it a core feature like for react.

BuckyMaler commented 4 years ago

@vsavkin and Nx team these are exciting plans! Thank you for sharing!

@Cammisuli thanks for sharing our plugin! @yjaaidi you should check it out! ๐Ÿ˜„

yharaskrik commented 4 years ago

Any plans (I know it has been discussed before) for using bazel for the computation caching to supercharge the incrementality of builds and tests?

nasreddineskandrani commented 4 years ago

Hi, @vsavkin i have a major blocker using NX.

The case: I want to benefit from all the tooling that NX provide like generation libs, schematics, etc.. i saw some nice articles from you about angular elements and more related to NX. I am working on a huge project of migration. Some modules are already migrated with angular in their own repo for now. Other migration are going to start in a couple months and other later 2021. All these modules are evolving at a different speed. So those that start now for example, are in angular 10. Those that were migrated 5 months ago will remain on angular 8/9 and we don't need to touch them since they already answer the client needs (no more dev for 4 years)... all this done by differents teams and deployed separately.

The problem: With NX there is only one package.json so this scenario i am mentioning is not covered by your tool right now which force me to either keep the project modules separated in multiple repos or turn to lerna. And with all the love i have for NX i can't move forward with it. Do you have any plan to move in the direction where we can have multiple package.json for the projects generated by NX? as addition to the current solution mode.


Also note that in your article about switching from Lerna to NX here someone posted something similar (not really but kinda) to the scenario i am mentionning on top (upvoted 112 times).

image

We didn't get an answer yet :). So i am trying here.


Thank y.

myspivey commented 4 years ago

I would love to see Module Federation be supported through Nx and enable a way to make it easy to adopt: https://indepth.dev/webpack-5-module-federation-a-game-changer-in-javascript-architecture/

danielquintero commented 4 years ago

Would be great to also document NX integration with python! and module federation should be a must ๐Ÿ˜ƒ

muratbeser commented 4 years ago

May be more build / ci stuff ? I know we can costumize our builds. But docs needs more touch on that configuration sections.

realityfilter commented 4 years ago

For us it would be definitely vue and nuxt support.

beeman commented 4 years ago

Would be great to also document NX integration with python! and module federation should be a must ๐Ÿ˜ƒ

Hey @danielquintero , a few days back @superdiana released nx-python:

https://github.com/codeonrocks/nx-python

danielquintero commented 4 years ago

Would be great to also document NX integration with python! and module federation should be a must ๐Ÿ˜ƒ

Hey @danielquintero , a few days back @superdiana released nx-python:

https://github.com/codeonrocks/nx-python

This is sweet! thanks a lot @beeman

xmlking commented 4 years ago

experimenting with nx style workspace for Java/Kotlin via Gradle https://github.com/xmlking/micro-apps

puku0x commented 4 years ago

~ESLint v7?~ It is available on v10.3.0! Thank you!

iangregsondev commented 4 years ago

@vsavkin Nice to see you are going to document Golang / .Net - these 2 things really appeal to me. Using golang and .net core would be great!

Please give some love to Dart / flutter :-) I should imagine having a golang/.net plugin then the dart/flutter would be very similar ?

Dart/Flutter is very popular right now and I have been using it for the past 4 weeks and I am completely blown away - its my go-to framework for mobile now (previously i played with react native) - also coming soon is flutter desktop to allow building desktop apps - I should imagine the popularity will continue to accelerate.

iangregsondev commented 4 years ago

@vsavkin In my honest opinion it would be really nice to have some sort of deployment help or integration with "conventional commits".

There is the excellent library called semantic-release but this is only for javascript - it would be really nice to be able to get some kind of support for

  1. conventional commits
  2. conventional changelog
  3. github releases

This is the thing that's giving me a lot of headaches, I still am not able to find a solution :-(

It would be nice to support some kind of abstraction so it can be used across all possible languages i.e. .net, golang, typescript, nodejs, etc

I have been looking into the docker deployment community plugin and its really nice but it published on tags i.e. latest or a specific version but because I can't get the above working - it isn't really a nice solution for me.

It would be nice to support the conventional commits / changelog / github releases so we could actually publish everything to github - work out the SEMVER using the conventional commits.

As I said before the semantic-release is a nice feature here BUT semantic-release doesn't work with monorepos and I don't see it working anytime soon - there has been a issue open for a very very long time - more specifically for working with LERNA - but the problems are going to be the same.

beeman commented 4 years ago

@vsavkin Nice to see you are going to document Golang / .Net - these 2 things really appeal to me. Using golang and .net core would be great!

@iangregsondev shameless plug(s), you can already us Go in Nx using nx-go.

With regards to releasing packages, check nxpm. It doesn't do changelogs yet, but I'm happy to talk about that/merge PR's that add it :-)

iangregsondev commented 4 years ago

Thanks @beeman , yep I found the nx-go after sending the above - thank you!

nxpm - I didn't know about! I am going to take a look - thanks again!

trumbitta commented 4 years ago

As I said before the semantic-release is a nice feature here BUT semantic-release doesn't work with monorepos and I don't see it working anytime soon

@iangregsondev I've been working with semantic-release in Nx monorepos for two years now. It's not optimal, but it's good. What kind of problems are you having?

iangregsondev commented 4 years ago

Hi @trumbitta,

Thanks for getting back. I will try and outline them here.

I thought of using only one package.json and using conventional commits and use the "scope" to identify which service it is for. I am assuming you are only using 1 semantic release?

Using the above method then there would be only ONE changelog.md but would contain all the scope changes ie..

feat (order-service) Able to order

fix (auth-service) Fix error on large passwords

Notice the scope in the brackets. As I say this would create 1 big changelog. ( I also deploy my services to docker-hub - and would love to use some kind of changelog for that - but right now - it's one massive changelog)

The other concern I have is how to release to github, I mean semantic release does this for you but now I am in a multi services (app) - so it would be way better to release the ZIPS per service and not as a whole and also ONLY list the changes on the github release for that particular app/service.

Also using only 1 package.json means I would be using one version to rule them all - meaning that when semantic release - releases, it's going to increase the version in the root package.json - so that means I need to release "auth-service" and "order-service" as a new version 1.5 - for example when order-service might not have any changes.

Not to mention, imagine have a "feat" in order-service and a "fix" in auth-service - semantic release would increase the MINOR because of the "feat" and ignore that actually the auth-service only had a "fix" - so it should release a "patch" increase and not minor.

So I am little confused about the workflow :-( I would love to hear what you are doing with it.

I really like semantic-release, it worked perfect for me BUT now with NX it does not - maybe I am missing something simple - like using multiple semantic-releases for each service and also multiple package.json.

Any help you can offer would be really appreciated.

As you can see, my ideas are all over the place :-( - It really comes from the confusion with multi apps in a single repo and versioning of a package.json and producing changelogs and github releases.

trumbitta commented 4 years ago

@iangregsondev for me it's just a matter of thinking in terms of a monorepo and embrace it as it is, instead of trying to make it work like I want it to.

I thought of using only one package.json and using conventional commits and use the "scope" to identify which service it is for.

Yup, that's what I'm doing as well. Sometimes I'll add a feature name to the package name in scopes, but it's up to the team to choose one convention over another for scopes.

Using the above method then there would be only ONE changelog.md but would contain all the scope changes ie..

Yep, I have one big ChangeLog. But it's ok because the repo is a monorepo so the ChangeLog is for the monorepo as a whole.

The other concern I have is how to release to github, I mean semantic release does this for you but now I am in a multi services (app) - so it would be way better to release the ZIPS per service and not as a whole and also ONLY list the changes on the github release for that particular app/service.

I've only used Nx with GitLab so far, but since I use affected:build it's rare that the release zip contains more than a couple artifacts. When one artifact per release is a strict requirement, I just don't let semantic-release upload them for me and I use a custom script to pick the artifacts from the dist directory of the workspace in the CI environment.

Also using only 1 package.json means I would be using one version to rule them all - meaning that when semantic release - releases, it's going to increase the version in the root package.json - so that means I need to release "auth-service" and "order-service" as a new version 1.5 - for example when order-service might not have any changes.

That's another thing I just accept as it is. But it's slightly different from what you are reporting if you factor in the affected family of commands: it's not rare to

auth-service effectively skipped 1.5, but it's not that much of a problem since versioning is automatic anyway and the ChangeLog is in sync.

OR, if you want, you could always build everything with --all (IIRC) and you'll end up with a auth-service 1.5 that's perfectly equal to auth-service 1.4, and then a order-service 1.6 that's perfectly equal to order-service 1.5.

But that's again fine, if not idiomatic of the monorepo paradigm.

So I am little confused about the workflow :-( I would love to hear what you are doing with it.

As I hope to have explained, I just embraced the paradigm and the environment and its outcomes.
Like I wrote in the first comment, it's not optimal but it's been good enough so far :)

iangregsondev commented 4 years ago

Thanks, @trumbitta , this has really helped a lot. Been busy setting everything up now.

I liked your way of thinking "just embrace it", after doing this, everything is easier to reason about.

Again, thanks for the help.

negativeentropy9 commented 4 years ago

Hi, @vsavkin i have a major blocker using NX.

The case: I want to benefit from all the tooling that NX provide like generation libs, schematics, etc.. i saw some nice articles from you about angular elements and more related to NX. I am working on a huge project of migration. Some modules are already migrated with angular in their own repo for now. Other migration are going to start in a couple months and other later 2021. All these modules are evolving at a different speed. So those that start now for example, are in angular 10. Those that were migrated 5 months ago will remain on angular 8/9 and we don't need to touch them since they already answer the client needs (no more dev for 4 years)... all this done by differents teams and deployed separately.

The problem: With NX there is only one package.json so this scenario i am mentioning is not covered by your tool right now which force me to either keep the project modules separated in multiple repos or turn to lerna. And with all the love i have for NX i can't move forward with it. Do you have any plan to move in the direction where we can have multiple package.json for the projects generated by NX? as addition to the current solution mode.

Also note that in your article about switching from Lerna to NX here someone posted something similar (not really but kinda) to the scenario i am mentionning on top (upvoted 112 times).

image

We didn't get an answer yet :). So i am trying here.

Thank y.

I have the same issue and don't know how to solve. Global package preference may lead some troubles, eg: I want to use package A in 1 version in app a, but I use package B in 2 version in app b, how to support separated package preference?

negativeentropy9 commented 4 years ago

focus on making setting up good dev experience

I use React. In dev env, any plan to support hmr instead of reload?

cheunjm commented 4 years ago

Is there a ETA for the React Native / Expo Plugin?

asbrum commented 4 years ago

@doudounannan

as mentioned in #3023 by @FrozenPandaz , different package.json should work. But you have to create it by your own. Give it a try if it works.

lazarljubenovic commented 4 years ago

I want to use package A in 1 version in app a, but I use package B in 2 version in app b, how to support separated package preference?

I thought one of the big reasons for using a monorepo is not to handle these issues and make sure everything is on the latest version?

nasreddineskandrani commented 4 years ago

@lazarljubenovic read here https://github.com/joelparkerhenderson/monorepo_vs_polyrepo#what-is-monorepo and you can guys contribute to these definitions to make them evolve :)

nasreddineskandrani commented 4 years ago

I thought one of the big reasons for using a monorepo is not to handle these issues and make sure everything is on the latest version?

I took 15min to think about what you wrote. Yes! i think you are right better to keep things separated if it's not to keep all the repo content in sync with one package.json. This case is probably when the repo is only front-end dedicated. The problem is as in the link i posted, monorepos are associated to Lerna which does exactly the opposit of your statement or maybe i am wrong.

When it's not possible to have one package.json i just go with polyrepos created by NX for the reasons i mentionned on top. Using NX for the generation of each repo. Angular based gives at least the benefit of auto setup eslint, jest, prettier etc... there is other things i can't benefit from like caching etc... :/

benatkin commented 3 years ago

I saw while trying to get a library building that @nrwl/web depends on Rollup 1.x, though Rollup 2 has been out for a few months. Would switching to Rollup 2 be on the roadmap for Nx 11?

I also wonder if it would be a good idea to provide an option of using a different bundler.

mckramer commented 3 years ago

With Angular 11 in pre-release, is there any consideration to start a pre-release branch of Nx to make use of cli 11 + webpack 5? As mentioned above, module federation is a very interesting feature that would be great to start to play around with to see how it might be used with Nx.

iammathew commented 3 years ago

@vsavkin Is there any chance for the extended Bazel support you announced a while ago in a blog post to land also for Nx 11?

airtonix commented 3 years ago

why does the @nrwl/react plugin create typescript applications/libs with @babel/preset-typescript?

It's probably due to my limited understanding, but I was under the impression that @babel/preset-typescript isn't real typescript and there are limitations on what it lets you do with typescript.

ankeshdave commented 3 years ago

@vsavkin Any chance of including module federation and related 'helpers and utilities' for quick and easy adoption fir the same.

abarghoud commented 3 years ago

Any news for angular 11 support, it has been released yesterday.

nickroberts commented 3 years ago

Excited for Angular 11, as well! I have updated my app to use it, and it looks like everything works just fine.

That being said, tslint is being deprecated in favor of eslint (yay!), so there will need to be some work done there. The guide to switch over from tslint to eslint can get you most of the way there, but it looks like it is going to be quite cumbersome to get all of the plugins / rules in place, especially if you have a lot of projects (as we do).

mokipedia commented 3 years ago

@nickroberts: did the same and upgraded to Angular 11. At first all seems fine. But I noticed strange issues: if I serve dev, all is fine and I get the new output format of angular-cli. But if I build for production, the old output format is used and things like font inlining is not happening. Deleting node_modules folder did not work. Adding optimization: {font: true} <-- object for finegranular optimization control throws an error (works fine in angular-cli projects) So probably take it with a grain of salt. Edit: found my culprit and it actually had nothing to do with nx, but the custom-webpack package

abarghoud commented 3 years ago

https://www.youtube.com/watch?v=W7b55ANw0fI&utm_content=145891678&utm_medium=social&utm_source=twitter&hss_channel=tw-1095103068373508101

llwt commented 3 years ago

@vsavkin / @FrozenPandaz, Is Next.js v10 planned for nrwl/nx v11?

lazarljubenovic commented 3 years ago

A short summary would be appreciated since I don't have the patience to watch an hour-long video :smile:

Eladigo commented 3 years ago

Thanks!! The Slack link is no longer active. Can I get a new working one? Thanks again.

// the current link: https://nrwlcommunity.slack.com/join/shared_invite/zt-ffstv0dy-wM2b~sYa_X7R8wXN02l2kA#/

yharaskrik commented 3 years ago

Join my Slack workspace! https://join.slack.com/t/nrwlcommunity/shared_invite/zt-98hoaiep-Rlpbyn4jR4lwOkzca64bqA

Eladigo commented 3 years ago

@yharaskrik Thanks! IDK why, but your link also open this page for me: image

yharaskrik commented 3 years ago

Here you go. Brand new link!

https://join.slack.com/t/nrwlcommunity/shared_invite/zt-jba969hz-e5zm1Ou_jiOP1J2h8UTu0w

LukasMachetanz commented 3 years ago

Hey. I am wondering what's the expected release date for Nx 11. Has anyone knowledge about this? Thanks in advance. :)

tonivj5 commented 3 years ago

Nx 11 is here :tada::clap:

jbadeau commented 3 years ago

Any ETA on that Java support?

maeri commented 3 years ago

@jbadeau
There's project https://www.npmjs.com/package/@nxrocks/nx-spring-boot officially listed in nx community approved plugins. Maybe that might help you

jbadeau commented 3 years ago

I saw that but I also saw that there was offical support & doc coming

First, we are going to document how to add Java/.Net/Golang/Rust to your Nx workspace. Second, we will add basic affordances for Gradle-based Java projects. So you can add a Spring Boot application to your Nx workspace without having to "figure it out".

I rather wait for official support if its coming?

vmptk commented 3 years ago

What about smooth and well documented integration with Vercel?