search

VulcanJS / Vulcan

🌋 A toolkit to quickly build apps with React, GraphQL & Meteor
http://vulcanjs.org
MIT License
7.98k stars 1.88k forks source link

Beyond Meteor #2017

Open SachaG opened 6 years ago

SachaG commented 6 years ago

This is a place to centralize questions about how we could one day move beyond the Meteor ecosystem.

Package-based Architecture

How does NPM handle package-based architectures? Should we use Lerna.js?

To clarify: in Vulcan's current architecture, 100% of the code is inside separate Meteor packages, there is no "app" code per se. This means the entirety of the codebase can be overridden on a package-by-package basis if a developer needs to do so.

Server/Client Split

In a Meteor package you can do:

  api.mainModule('lib/server/main.js', 'server');
  api.mainModule('lib/client/main.js', 'client');

What would be the NPM equivalent? Can import foo from 'foo' have different values on the client and server like with Meteor?

What about things like 

if (Meteor.isServer) {
  //…
}

Fibers

Where do we still rely on Fibers? Could we replace this by async/await code?

What about Meteor.wrapAsync? Meteor.bindEnvironment?

Meteor.startup(...)

What is the equivalent of Meteor.startup() in, say, Next.js?

Accounts

What do we use for Accounts? Auth0? Can we still preserve backwards compatibility with existing Meteor accounts?

Mongo

What do we do about Mongo.Collection? How do we run and connect to Mongo? What do we do about Collection.find()?

comus commented 6 years ago

Hi, do u hear spectrum. I got a lot ideas from their source code, and made a simple similar next.js app for my previous project. U may take a look on their github. Rethinkdb+graphql+backpack+passportjs U may get some thoughts from their structure

SachaG commented 6 years ago

That's a good idea, I'll check out their stack!

Apollinaire commented 6 years ago

For the lazy ones out there: Spectrum.chat stack

gaurav- commented 6 years ago

It's possible to achieve Meteor-like "isomorphism" by using Webpack as the build tool, thanks to its fine-grained configurability. While "zero config" is great for application developers, configurability is vital for framework developers not wanting to reinvent some wheels, imho :)

E.g. Meteor.isServer and Meteor.isClient can be easily emulated via Webpack's DefinePlugin – I do it in my current project (pretty large codebase) that uses Webpack to bundle Meteor client-side code (with some help from meteor-client-bundler)

Perhaps the above can also be done using other bundlers like Parcel and Fusebox (if anybody has experience with them, please share), but Webpack is clearly the most popular one today – almost the de-facto build tool for new web apps today.

The whole journey of migration away from Meteor could be a long one, so strategically it might be a good idea to do it gradually and iteratively. The fine-grain control that Webpack allows would come in handy for that.

Perhaps it would be best to start with only the client-side code first, because there's less of Meteor magic there, especially since this project seems to be already using Apollo-client instead of Minimongo etc. That way, you'd also not have to worry about removing Meteor.wrapAsync and Meteor.bindEnvironment initially, as they are thankfully only relevant on the server-side. Similarly, decisions on Meteor.startup and Mongo.Collection on server could be deferred. On client side, I believe Mongo.Collection may not be getting used much, and should anyways be migrated to Apollo-client if there's any stray usage.

During iterative migration, you may need to use both Webpack and Meteor for building and bundling client-side code; achievable with meteor-client-bundler. If it was feasible to build a Webpack loader or plugin for loading Meteor packages, that would have been a better approach – something I have wondered but never had the bandwidth to pursue. Alternatively, meteor-webpack could also be tried out.

I believe Meteor is still quite superior a framework and toolchain for server-side. It would make good architectural sense to have a separate "Backend Server" (Apollo-server on Meteor) that provides Web APIs for different datasources and a separate "Presentation Server" that provides isomorphic view rendering (via a framework specifically built for that purpose, like Next.js). This may also open up the possibility of the community implementing non-React presentation layers like Nuxt.js and Angular-Universal.

While for the server-side Meteor code, the "modernization" could involve migrating towards async/await and Promise. Wherever it gets tricky to use async keyword for any function declaration (because it would start returning a Promise<Value> instead of the original Value, and hence would affect the entire chain of calls), Promise.await provides a good stopgap (note it's Meteor/Fiber specific). But migrating server-side can be considered less urgent, imo. If you're interested in alternatives to Meteor for the backend too, I'd suggest considering Nest.js – a framework that I'd definitely consider for my next greenfield Node.js project

SachaG commented 6 years ago

Wow thanks for the detailed comments! That makes a lot of sense. I hadn't heard about Nest.js, I'll check it out as well.

SachaG commented 6 years ago

Here's an unofficial timeline for the next couple months:

gaurav- commented 6 years ago

I'd likely start incremental migration towards Next.js + Meteor for my own app. If I make any significant discoveries, will definitely share them here

j209 commented 6 years ago

Since Both of reactxp and Nest.js are made with typescript, What about making a fullstack framework based on Reactxp as frontend and Nest.js as backend?

gaurav- commented 6 years ago

@SachaG before starting the move to Next.js, check out Fusion.js that was open sourced by Uber's web platform team couple of days ago! The more I think about its architecture and features, the more it looks like what I'd imagine Meteor 2.0 to be 🤩

Perhaps ideally Vulcan's features should be reusable npm packages, so that they are as framework-agnostic as possible. And choose one of the frameworks like Next.js and Fusion.js as the showcase implementation. Do you think that's a direction worth considering?

w.r.t. https://github.com/VulcanJS/Vulcan/issues/2017#issuecomment-408011847, I'm now thinking of first evaluating Fusion.js for the presentation layer of my app.

paranoico commented 6 years ago

Hello, I recommend to avoid all Microsoft pseudo new technologies like ReactXP, I have been MS "expert" for 15 years with several certifications.

But since about 7 years I have seen many of the "new" MS politics, like with MS SQL Server, Windows Server, etc. going only on the paying model. And none of them are as good as .NET in the beginning or Windows 7, Windows NT, etc. So now we are leaving away from MS and using only true open source alternatives like Debian, Nodejs, Ubuntu and many others.

MS can change all their politics in the future and, like buying Github, can be against their users very easily.

Thanks a lot to all for this excellent framework.

SachaG commented 6 years ago

Just to keep everybody up to date, I've now had calls with core members of both the Next and Fusion teams.

I would say both projects look great, and they both have their pros and cons. My very rough impression so far is that migrating to Next might have a little bit more upshot (more mature product, larger community) but migrating to Fusion would potentially be a lot easier due to the fact that it uses a plugin architecture very similar to Vulcan.

So anyway, I'm keeping a close eye on both for now :)

meepeek commented 6 years ago

TLDR

I wish you would consider loopback.io as a backend to move to. I think we can place Vulcan's schema define everything concept on top of loopback authentication, datasource and ORM. Or at least, it is worth to learn their design if you decided to create your own.

Vulcan concept is interesting

I read your concept and it was interesting. I agree that meteor is currently the most user-friendly backend could be found in the market and we should be moving to lighter framework. I would like to share my opinion as follow

Backend suggestion

Have you considered loopback.io as a backend ? It has user management, authentication, ORM and REST api generation out of the box. With additional package available, I can get graphql online. I am waiting for their 4.x release on early 2019. It would be a big change and should already cross the half-line of their milestones by now.

This is what I currently have after playing around for a while. I integrated react-admin and fix some rest adaptor issues and vola!! admin interface ready. Please find the code here:

https://github.com/meepeek/loopback3-with-admin

My opinion about loopback

PROS

CONS

SachaG commented 6 years ago

Hmm, I don't know if it makes sense to migrate to something like Loopback. I feel like at this point we'd be more or less rewriting the entire framework from scratch. And also I don't think it would help with the main reasons why we'd be migrating in the first place (better bundling/SSR)?

Rockson commented 5 years ago

Imo Vulcan should provide components that depends only on apollo. So we can choose whatever ui library and platform we prefer(next, fusion, cra, gatsby ecc) And maybe implement 1 of them like bootstrap + next as default

x5engine commented 5 years ago

what if we use assembler instead? in the backend? it would be very very fast!

SachaG commented 5 years ago

Imo Vulcan should provide components that depends only on apollo. So we can choose whatever ui library and platform we prefer(next, fusion, cra, gatsby ecc)

That's an interesting idea but it would basically be a different project. Also running a bunch of React components on Fusion/Next/Gatsby/etc. is not the issue, the problem would be ensuring your Vulcan components can connect to your GraphQL back-end. Unless we end up in a world where there's a common standard for GraphQL schemas and APIs, you'd basically have to do 90% of what Vulcan does yourself which would defeat the point…

gaurav- commented 5 years ago

Unless we end up in a world where there's a common standard for GraphQL schemas and APIs, you'd basically have to do 90% of what Vulcan does yourself which would defeat the point…

@SachaG I think the current feature set and usability of Vulcan can be retained, and yet its reach be made broader by expanding on Rockson's suggestion. I had actually suggested something similar in my earlier comment:

Perhaps ideally Vulcan's features should be reusable npm packages, so that they are as framework-agnostic as possible. And choose one of the frameworks like Next.js and Fusion.js as the showcase implementation. Do you think that's a direction worth considering?

To summarize the whole idea properly, Vulcan should probably be broken into separate projects that play well together:

  1. A set of NPM packages that implement "end-to-end GraphQL data layer of various features" i.e server-side and/or client-side modules of those features.
    • They could be tightly coupled with Apollo, at least to start with, especially on the client-side
  2. Reference UI implementation of these features, using React.
    • That should hopefully not look much different than today's implementation.
    • Encourage community to implement them with other view technologies
  3. Reference SSR implementation using a framework like Next.js or Fusion.js.
    • Encourage community to implement them with other frameworks
  4. Reference GraphQL service implementation.
    • Can continue to be on Meteor.js, at least initially

Easier said than done. But the changes should mostly be about removing the dependence on (and assumption of) Meteor API and packages, at least for client-side stuff. I think this should be doable one package at a time, hopefully with the community chiming in to accelerate the process.

eric-burel commented 5 years ago

For the MongoDB dependency, you can take a look at Feathers adapter system. Basically they kept the MongoDB syntax as a common syntax (selectors, filtering) and let people use custom adapters to connect other kind of databases. That's similar to a micro-ORM.

SachaG commented 5 years ago

@gaurav- I think those are all great ideas, and if I was starting Vulcan from scratch today I might go that route. But at this point I just think the folks at Prisma or Apollo are closer to building this than I am.

Also given that making Vulcan work on a single stack is already an infinite quantity of work as far as I can tell, making it work on two or three stacks would mean two or three infinities of work. So I'll pass on that thank you very much.

gaurav- commented 5 years ago

@SachaG my suggestion is to maintain only one stack officially (aka "reference implementation"). But in doing so, I see an opportunity to also allow users to use their own stacks, if they choose to.

The path to that "one stack" (be it Next.js, Fusion.js or something else) could be gradual, like I proposed earlier in some detail: https://github.com/VulcanJS/Vulcan/issues/2017#issuecomment-406778893

eric-burel commented 5 years ago

A small part that would need a refactor is the newsletter system. It relies on a Meteor package while we should be able to do the same with Node cron (or any other npm package relevant to this) and bindEnvironment. See https://stackoverflow.com/questions/40687237/cron-jobs-in-meteor.

j209 commented 5 years ago

RNG (React, Next.js, GraphQL) can slay the MEAN stack

SachaG commented 5 years ago

That article is pretty lazy, since it brushes over half of the MEAN stack (Express, Mongo, and arguably Node) and just tells you to use a third-party service as your API server.

jay-jlm commented 5 years ago
  • December-January: depending on current state of the Meteor ecosystem, work on Next.js migration.

Any updates on this front? It's now March 2019.

SachaG commented 5 years ago

@jay-jlm Migrating away from Meteor is not currently on the roadmap. I think it'd just be too much work at this point and would kill any momentum Vulcan currently has. Let's see how far we can go with Meteor for now :)

ErikDakoda commented 4 years ago

I see there is a lot of activity in the Next.js version of Vulcan. Has there been a new decision made about whether Vulcan will be migrated away from Meteor?

eric-burel commented 4 years ago

Hi @ErikDakoda ,

We have indeed started to migrate toward Next, as a replacement to Meteor. Note that we will have to keep Meteor around for a while, we have legacy apps to maintain but even new ones being developed The server part of Vulcan will be abstracted away as NPM packages. Next week I'll improve the development environment of https://github.com/VulcanJS/vulcan-npm. It will be doable to maintain both Next and Meteor versions with a shared codebase

The process will probably be like this:

eric-burel commented 4 years ago

What triggered this migration is the addition of API routes in Next in v9, which makes it an hybrid framework, supporting both full-stack and pure frontend usage.

ErikDakoda commented 4 years ago

Okay, thank you for the update! I agree with the decision to move most of the code to NPM packages. Then the next biggest hurdle to moving away from Meteor will be the Accounts functionality. Very interesting - I'll be keeping an eye on this.

eric-burel commented 4 years ago

Regarding accounts we've introduced GraphQL endpoints in the latest release, so in the backend we don't depend on Meteor. We lack the UI at this point though, and support way less features than Meteor does, that's indeed a big subject.