Keith-Hon
commented
2 years ago I agree that Gatsby is slow for development. But how about using nextjs with typescript? IMO, it's better to use 1 framework across the admin and storefront apps.
Closed edardev closed 1 year ago
Keith-Hon
commented
2 years ago I agree that Gatsby is slow for development. But how about using nextjs with typescript? IMO, it's better to use 1 framework across the admin and storefront apps.
sangdth
commented
2 years ago Please no Gatsby. Just please. I tried it several times and never ever get it work even in development and right now I can't even make the admin work. Just please no Gatsby.
edardev
commented
2 years ago I love Gatsby for storefronts by the way 🚀🚀🚀
madalinignisca
commented
1 year ago Hi. I'd like to share my opinion also. The admin should be a pure React app. Also, it should be done in a monorepo, so it's common components could be reused to build react native or ionic admin app. I would prefer a package with all common react components needed for the admin and assemble something quick. Also, removing the monster gatsby that it's instance running can consume > 500mb of ram, I really can't see the point on needing extra hardware and another point of failure in a project, when a static SPA react app could ensure reliability of the running project.
sangdth
commented
1 year ago Update for my previous comment:
The reason it didn't work might be because I used pnpm with workspace feature, as I want my development repo with medusa in a monorepo. In my other projects, the pnpm works well with Nextjs, zero hiccups!
My experience with Gatsby was so terrible.
echocrow
commented
1 year ago To the maintainers in particular: Is there any interest in a migration of this admin package to NextJS or ViteJS?
I'd be willing to tackle this, maybe alongside an upgrade to React v17/v18, but would proceed only if such a PR is welcome and has a chance of being accepted. (i.e. I'm unsure if such a transition would conflict with any present visions for this repository.)
I know the topic of framework choices can be very controversial and opinionated. If this is not a hard-no, maybe there is room for discussion or discovery?
To share our motivation, some pain points that stuck out immediately as we were trialing the current Medusa Admin:
dev is painfully slow, despite the plethora of caches written to disc. (Medusa Admin under Gatsby was by far the slowest starting process by a long shot in our mono repo test.)gatsby develop. This eliminates the option of running the Medusa admin under a sub-path of a domain in dev mode.Some minor additional (more controversial) arguments against Gatsby for this project:
(Not saying to never use Gatsby, just arguing why it might not be the right tool for this app.)
A pure React + Vite approach seems like the simplest fit for this typo of dynamic, client-rendered app. However, looking at the current state of the repo, a migration to NextJS could be the lower-hanging fruit, and adopt a more widely-used React framework approach. That said, its SSR capabilities and wider feature set may be overkill.
Thoughts?
@sangdth to your point & update: We also set up Medusa Admin inside a monorepo with pnpm. We did get it to run, though had to jump through some hoops first—primarily installing missing peer dependencies. In my experience that is one downside of nesting existing repos inside a monorepo, as you may lose the exact set of dependencies installed (here locked in yarn.lock). With ViteJS or NextJS this particular issue may be less cumbersome to solve (due to potentially fewer missing peer dependencies), but likely still present unless all peer dependencies are meticulously documented.
sangdth
commented
1 year ago Thanks @echocrow for the initiative move. I think PR is welcome. If you need anything I would love to help.
srindom
commented
1 year ago Thanks for weighing in here everyone - would like to share some of our thoughts on where we would like this to go. First and foremost, I agree that the dashboard should be migrated from Gatsby to something more appropriate. Furthermore, we have two larger goals for admin in the near future:
In regards to shipping the Admin project as an NPM package, the thinking is that this would enable a local dev experience where one can visit http://localhost:9000/dashboard after running:
$ cd my-medusa-store
$ medusa developThis could simply happen by registering the Admin project in the Medusa server. It is, however, important that when moving to production, the admin project can be deployed independently from the backend server e.g. by running a medusa build command that bundles extensions with the rest of the admin code and places the build files in a directory that can be deployed on services like Vercel.
I think all the suggestions mentioned could work and would love to accept PRs for initiatives in this direction that could also align with the requirements mentioned here - I know that @kasperkristensen already researched potential approaches so maybe he can share some thoughts.
echocrow
commented
1 year ago @srindom thanks for sharing some of the plans and future goals. Love the idea of being able to extend the Admin with custom pages, without needing to be forking the whole Admin repo.
AFAIK Medusa uses Express for its core package (i.e. the API). Unless this aspect is subject to change, we'd need to find an approach that can be integrated nicely into Express in order to serve the Admin as part of the Medusa server (e.g. under /dashboard)
This limitation probably already rules out Next.js. There may be a way of serving a Next.js app under an Express route, but the two are more likely to fight each other than to synergize. (And, if SSR is unnecessary, Next.js may have been overkill anyway.)
If we can prebuild and/or stick to CSR-only, a common solution is to build, and then serve that output statically under an Express route.
Two quick examples that I recently came across that solved for this:
express.static to serve dist output of the admin UI under a fixed /admin route.express.static, same idea as DirectusIf that approach makes sense, how does this sound as a game plan?
medusa develop/medusa start.
(\ personally I don't see much need to allow deploying just the Admin, seeing as it also cannot run on its own, but heavily relies on the API server to be there. For reference, both Directus and Payload have flags to serve only their APIs, disabling the Admin, or serve APIs+Admin when both are desired. That said, it may not take much to offer Admin-only deployments nonetheless, assuming the API server is already running independently.)Personally I'm a fan of Vite over webpack, given its, IMO, saner approach to dev mode. However, either bundler should get the job done.
Thoughts?
kasperkristensen
commented
1 year ago Hi @echocrow,
I've had the same thoughts as you outline in your last comment, and it is the same approach I would take. Regarding Vite vs. webpack, I agree that Vite is the better experience and would most likely result in a faster development process as webpack is a bit convoluted. However, I think that webpack is the better option, due to our wish to make admin easily extendable. My current thinking for this would be through two means:
From doing a bit of research there does exist a library to handle micro frontends with Vite much like how webpack handles it, but it does not appear to have the same amount of support as webpack's Module Federation. Gatsby's shadowing API is also powered by webpack, so this would be a great place to find some form of inspiration on how to accomplish this.
This leads me to think that React + webpack is the ideal combination for what we want to achieve, but it would be amazing to get some feedback on this.
srindom
commented
1 year ago @echocrow - think your suggested game plan sounds great!
Regarding ViteJS vs. Webpack, I have no strong opinions and trust that you and @kasperkristensen can find the best solution. (Maybe @olivermrbl has opinions here?)
I think there are a few questions high-level questions to consider:
echocrow
commented
1 year ago @kasperkristensen you bring up interesting points, in particular about allowing components from other framework.
The plugin extension DX you described is sound: Write a Medusa plugin, write a component (/micro frontend, aka component with extra guts), then add a config that references that component and describes where it should be injected (e.g. order page, or as a net-new page). 👌 We'd have so many usecases for this.
RE Module Federation: From my understanding, MF does not provide a solution to how a plugin author needs to describe where this component should go, nor how the Medusa admin React app needs to be architected to allow for these slots or extensions. Solutions for those still need to be identified and implemented independently of MF, correct?
AFAIK what MF does try to solve is how these custom components can be build independently, and how that build output can then be used inside the consuming package. If that is a problem that needs to be solved, MF sounds like a solid option, and webpack will likely have the best support for this. Though, as you mentioned, examples and plugins do exist for all sorts of setups. (Off-topic: The most bizarre find while reseraching: Medusa, the solution to rule federated modules. 😵💫)
If Module Federation does need to be used, one thing I'd be cautious of is to not leak this implementation detail into how plugins are authored. That is, a plugin author ideally should not be required to have an intricate understanding of the Module Federation API, and fiddle with webpack config plugins. I'm unsure if this could be abstracted away?
Furthermore, if MF is an integral part of custom plugin extensions, how would this translate over to custom "in-repo" plugins? For example, Medusa already allows "non-packaged" extensions, such as in-repo services in src/services or endpoints in src/api. Would this approach also support in-repo admin extensions?
Apologies if this is be a naive question, but taking a few steps back, I do wonder: Is Module Federation actually necessary for these component plugins?
For example, I believe most React component libraries do not require MF, yet they can be imported, used, and bundled like any custom React component across packages just fine. (Doing some spot checks, many seem to just leverage Rollup for their builds.) - Am I missing a more complex problem these React component packages do not need to solve, but an Admin component plugin would? (other than preemptively declaring where the component should go in consuming app).
RE other frameworks: Support for custom components could be extended further to allow for web components. This too should allow plugin authors to use the framework (and bundler!) of their choice.
(Again, I haven't had much experience with Module Federation, so the above may be riddled with misunderstandings; please take with a generous spoon of salt.)
(Also none of the above offers ideas re shadowing; here I'm afraid I'm out of my depth without further research and experimenting.)
echocrow
commented
1 year ago (one day I'll to master the art of concise issue comments. today was not it.)
kasperkristensen
commented
1 year ago You are correct that webpack MF does not provide any native way for us/plugin authors to define where there component should be rendered/mounted. This is something we would have to come up with a solution for.
My initial thinking is that we could write this to a config somehow during the loading step, by using a loader placed in src/loaders of the plugin(s).
We would then need to check which MF’s we need to be able to import by reading said config, as we need to write them to remotes object in the webpack config of the host app (admin).
module.exports = {
entry: "./src/index",
mode: "development",
devServer: {
port: 7000,
},
// ...
plugins: [
new ModuleFederationPlugin({
name: "Host",
remotes: {
Remote: `Remote@${pluginBaseURL}/moduleEntry.js`,
// need to do this for all plugins that have admin components to load
},
// ...
}),
// ...
],
resolve: {
extensions: [".js", ".jsx", ".ts", ".tsx"],
},
target: "web",
};
Then we need to set up some infrastructure to find out where each component/page should be mounted and ensure that they are rendered correctly, which I am not certain of how to do yet.
In regards to your point of plugin authors not being required to know the details of Module Federation or web pack for that matter, I definitely agree, we should make the barrier as small as possible. I do believe we can get a long way by shipping a utility package with a nice API that would abstract this away, so the author would simply have to create a file webpack.config.js with something along these lines:
// in <plugin>/webpack.config.js
const AdminModule = require(“@medusajs/admin-module”)
module.exports = AdminModule({ <… optional custom config> })Or as a starting point, simply provide a good starter project for creating plugins with admin extensions.
I think if we are able to get something like this to work with plugins, then there is nothing stopping us from also making it work with "in-repo" extensions, as Medusa plugins in themselves are just slim Medusa servers.
I've also looked into Payload, and they also allow you to overwrite components and do so without the use MF, so the use of MF is definitely not a must. But from looking at their source code it seems as if passing these overwrites is a bit more involved, where I think the ideal case would be that we could make adding admin extensions as easy as doing yarn add medusa-plugin-something, adding it to your medusa-config, and voila you have installed an admin extension.
But as you can properly read from my comment I am still unsure about many aspects, and if it is even feasible to do an implementation like this. There is still a lot of research to do on the matter. So all feedback/ideas/sanity checks would be greatly appreciated.
But to get back to your initial outline for the approach to getting started with this. As you mention a great starting point would be to convert this current repo to a simple React project. Since work on this plugin approach is still a couple of months off, but the move from Gatsby to React could be done now to frontload some of the work, and would mean that we can start to reap some of the benefits that have been discussed in this thread.
echocrow
commented
1 year ago A minor blocker for migration is initial alignment on the tooling to move to, currently evaluating webpack vs Vite.
If we already know today we'll rely on Module Federation for plugins later, webpack should probably be the way to go. Otherwise, I'd still lean towards Vite. HMR will benefit a fair bit, and the overall config to get up and running is basically a one-liner, compared to configuring webpack for React. (Preemptively excluding CRA as an option here, especially if we need to modify the build process later.)
If we're not 100% sure on MF today, but are okay to potentially iterate later, we can pay the webpack toll if or when we do need to cross that bridge? Though double-migrations are rarely fun, moving from Vite to webpack should not so bad.
Would you feel confident with that approach? If so, I can start giving this a shot.
Re plugins:
Agreed that plugin installation is ideally be as simple as a package install, and maybe explicitly listing it in a config (implicit < explicit). And I think that can still be achieved without needing to resort to MF.
Payload may not have been a great example, and was mostly referenced to show that components can be naively passed around, without a MF middle layer. Other than that, its problem domain for extensions is just using custom components for the collections you're already defining.
Directus's extension system is a slightly closer match, with the support for both packages and in-source code to add new layouts, panels, etc. Its extension packages are incredibly minimal, too. However, they also engineered a whole custom build step around their extensions ecosystem that utilizes Rollup programmatically. Feels equal parts spicy and spooky.
Taking a step back, here are some sample features I can see useful for the plugin system:
This may be a naive idea, but could most if not all of these be solved by introducing some sort of "hook" (not the React kind) or middleware system? For example, key components could be wrapped in higher-order functions than can alter its props or output. Could that be leveraged to dynamically alter props or return a different component entirely?
Let's say the order detail page were refactored to something like
const OrderDetails = ({ id }: OrderDetailProps) => {
// ...
return withPlugin('OrderDetails', { order }, (
<PageLayout
body={[
<OrderInfo order={order} />,
<OrderSummary order={order} />,
<OrderPayment order={order} />,
// ...
]}
sidebar={[
<OrderTimeline order={order} />
]}
/>
))
}
// whereas
const withPlugin = (compName, context, component) => {
// get registered plugins.
// iterate over component overrides.
// return final component.
}As a plugin author, I might then hook into this, e.g.…
// components/MyOrderWidget.jsx
const MyOrderWidget = ({ order }) => {
return <p>Hello from sidebar for order #{order.display_id}</p>
}
// index.js
export default {
dashboard: {
components: {
OrderDetails: (Layout, props, { order }) => {
props.sidebar = [
<MyOrderWidget order={order} />,
...props.sidebar,
)
return <Layout {...props} />
}
}
}
}…or return a custom React or web component entirely.
There is undoubtedly a better syntax and API than this—just throwing half-cooked noodles at the wall here. But as a proof-of-concept this could allow extensions and overwrites wherever a hook is defined.
But I digress from this issue. May want to pause plugin talk to move it elsewhere?
srindom
commented
1 year ago @echocrow, @kasperkristensen - would it make sense to arrange a quick call between the three of us to discuss an approach?
We are all quite aligned on where to go in broader terms, so if we could just settle on whether or not to use MF in the first iteration, I think we can kick this off?
kasperkristensen
commented
1 year ago That sounds great to me, would be amazing to get the work for this off the ground 👍
echocrow
commented
1 year ago happy to hop on a quick call later this week or next! should we move the scheduling to Discord? (already in the community server as EchoCrow)
spoiler: I had some spare time on the weekend and already got this started (hence #736)
migration off of Gatsby might be done, pending some testing. React 18 along with package upgrades (e.g. reach router to react router) is WIP. I can open a draft PR later this week or sometime on the weekend.
I rolled with Vite for now for convenience. switching to webpack should be as "simple" as putting that webpack config together, and swapping the package.json dependencies & scripts from Vite to webpack; rest of prod code should be agnostic to the bundler.
edardev
commented
1 year ago @echocrow, @kasperkristensen - would it make sense to arrange a quick call between the three of us to discuss an approach?
We are all quite aligned on where to go in broader terms, so if we could just settle on whether or not to use MF in the first iteration, I think we can kick this off?
1- This project should take a similar approach to Strapi.io but with better architecture and development tools.
2-Since this is a "headless project" the team should consider allow 3rd party integrations from battle-tested auth providers such as Auth0, ORY, AWS Conigto, Custom providers, etc.
3-Another suggestion is to remove RedisDB and implement an event-driven architecture with a different approach.
Concept: As of today, the ultimate headless eCommerce solution will be an integration of headless and open source solutions from projects like Hasura, ORY Kratos, Strapi, Prisma.io, typesense, and many more.
echocrow
commented
1 year ago @edardev
1- This project should take a similar approach to Strapi.io but with better architecture and development tools.
Can you elaborate on the approach that Strapi took that would be a good fit here? Where did they fall short in their architecture & dev tools?
RE auth & event-driven: agreed that 3rd-party auth should be supported (unsure about the current state of this today). unsure about the concern regarding Redis and the current event driven architecture. However, those may be good topics to bring up in the main Medusa project? This repo here is just the backoffice admin UI for the (otherwise) headless core.
edardev
commented
1 year ago Strapi has improved a lot in the latest versions. They have an API server and Admin panel that run together or separately depending on our development or production use case. Most of my medusa-related concerns are on the server APIs, which I need to comment out in the core repository.
Some of the Strapi Admin Panel issues to note are:
An awesome suggestion for the admin panel is to make it mobile-first so it can be integrated into a Capacitor/Ionic app or something similar for an all-in-one implementation of the web/mobile app.
srindom
commented
1 year ago @echocrow - what is you discord handle?
Use Gatsby for the storefront apps only