Docs and demo app being on a separate branch is weird

Martinsos commented 4 months ago

The fact that docs for the template, and a demo app (which is template + some small modifications) are on a separate branch (deployed-version) from the actual template (main) is weird.

When updating the template, one would normally also want to update the docs, and maybe also the demo app. We should be able to update all of these in one PR.

What we are actually doing here right now is using multiple branches to have multiple projects in the same repo, but that is not how this should be done, we should use directories for that.

One reason for doing this before was that we allowed people to create a new repo by using GitHub's "template repo" functionality. We gave up on that, so that is not a valid reason any more.

One could also argue that having a demo app and docs on a separate branch allows for easy updating of them with new changes from the template (you merge in new stuff from the template when it updates). That does however also mean that those changes can't be done in the same PR in which they are done for the template, which is not great. But this is a better argument, and the question is, how are we going to be updating demo app and docs?

So, let's try to figure this out!

Current approach

Ok, so problems with the current approach:

You can't update docs or demo in the same PR where you are updating the template.
You have to do the changes twice -> you have to make one PR for updating the template, then another PR where you do those same changes to the demo app and docs.
It is just weird.

I wonder, how do we now make sure that demo app and docs are up to date? If somebody does a PR to deployed-version branch, does it get deployed automatically, both app and docs? I know we ask users to make changes for docs against deployed-version, but who makes sure that the demo app gets updated with those changes and re-deployed?

Does the fact that demo app is on separate branch help with updating it when template changes? Is it always one big fat commit on top of commits from template so you just keep rebasing that one, or merging into it?

All in dirs

What if we put stuff into directories?

So at the top level we would have template/ and demo-app/ (or open-saas-sh).

Then, one can make changes to everything in a single PR.

But, the bad thing is, they should make changes to everything in a single PR. If we did use some branch tricks to help us update the demo app based on changes in the template, we can't use that anymore, at least not as easily.

So how do we easily update demo-app and docs, when template changes? Maybe we could have them implemented in some smart way: so instead of having them be duplicates of template and its docs, we would represent them as patch files, and would have a script that applies patch files to the template to produce demo app and its docs? Is there some existing tooling or something that would help with this? Sounds a bit too much?

Separate repo

So instead of having multiple branches, we could have demo app in a separate repo. I am not sure we get much with this though. Kind of makes sense, because now it is an actual another app that is made from open-saas template, but other from it semantically being "better", I don't see any other benefits, it has all the same problems as the approach with multiple branches. It does feel a bit better though, less/not wrong.

What now

Ok, this is a problem statement + some potential solutions: let's discuss!

Martinsos commented 4 months ago

@infomiho @sodic also interested in your ideas here!

vincanger commented 4 months ago

@Martinsos I like your separate dirs suggestion.

I personally would go with template/ open-saas-sh/

This would make it easy for our wasp new -t saas command to only pull the template directory. In the case that someone clones the repo instead of using wasp new they can just delete the open-saas-sh dir.

The patch idea sounds cool, but also like a lot of extra work. At the moment, it's pretty easy to apply changes to both manually when necessary.

Martinsos commented 4 months ago

Cool, that is good then!

Ok, although I really hope nobody would directly clone the repo, I think the only way for us to keep open-saas maintainable is to give up on people directly cloning, because that limits us. So I would avoid considering that as a use case.

It is easy to apply changes -> that surprises me, I thought it would be quite hard to keep applying changes to both. For example, all these small PRs I did, how will we apply those to the demo app? We will also be having bigger PRs soon, how will we apply those changes? Sounds easiest to me to always reapply things that are unique to the demo app on top of whatever is the latest state of the actual template, which is why I was mentioning patch files, and why I thought that current approach with branches actually has those benefits.

Martinsos commented 4 months ago

We should take a look at https://github.com/wasp-lang/open-saas/pull/131#issuecomment-2125612924 while doing this and if we go with two dirs, also add a top level .gitignore for ignoring template/app/migrations/.

infomiho commented 4 months ago

I like the thinking generally in this thread 👍 the template and the demo app should be separate (dir level or repo level) and the updating of the demo app isn't that much work so it's okay if it remains a manual operation.

Fork idea

Another idea I had while I was thinking about this issue today: why don't we make this repo the "open-saas template repo" and then fork this repo where we then make changes to the template to match the current "open-saas demo app". If it's a fork, we can sync it to the main repo (Github has support for this in the UI) and that way we get the benefits from the changes in the main repo, while keep the commits that make up the demo app.

sodic commented 4 months ago

From what I see, we all agree we don't want main#app and deployed-version#app to have different meanings where one means "OpenSaas template" and the other means "Open Saas demo app."

That sounds good to me as well.

As for the other stuff...

At first, I was against giving up on users cloning the repo. I must admit, until today, it didn't even occur to me there was some other way to use Open Saas besides cloning the repo. I'd always clone it (although I technically knew I could use OpenSaas through wasp new, it just never occurred to me).

Anyway, I wanted to keep cloning (or something similar to cloning) because:

Cloning is (I think) the standard procedure for "using this as a starting point of my app." I believe most templates/boilerplates work this way.
Open Saas is a separate product from Wasp that people discover independently (it's arguably even more popular). We shouldn't cripple it by turning it into a "Wasp plugin" but should instead treat it as an "App boilerplate that uses and relies on Wasp." I believe that's better for both marketing and discoverability.

However, the docs and demo app thing is annoying, and it does seem more practical to separate this stuff into folders inside the same repo.

So, here's the compromise I propose:

The repo contains everything separated into multiple folders (template, docs, demo-app) as already suggested.
- If things get more complicated, we introduce two branches (main and release) and use them like we do in wasp-lang/wasp
No fancy stuff with patch files. I think it's too complex to justify itself.
In OpenSaas's README, we list two options for getting started with OpenSaas:
- Clone just a single folder from the repo using something like https://github.com/unjs/giget.
- If you already have Wasp, update it and run wasp new -t saas.
I'd be OK with @infomiho's Fork idea if it weren't for the docs. If we're keeping the docs on the main branch next to the template, cloning is already out the window, so we might as well keep the demo app there too (Miho, correct me if I missed something).

Martinsos commented 4 months ago

Ok thanks for the ideas @infomiho and @sodic !

Github template vs `wasp new -t saas`

@sodic there have been other friction points in the last months, all coming from the same fact that we on one hand need to be able to easily develop and test the template vs provide optimal starting experience for the users when starting with the template. Those two have different requirements, and conflicts keep popping up and complicating stuff when we try to keep it all one thing. Plus, having two ways to get the template means possible discrepancies (although this is less of an issue). So with my perspective/experience with this from the last months, I developed a strong opinion that we should have only one way of using the template and that is wasp new -t saas.

Updates to demo app are easy to made

@vincanger and @infomiho you both mentioned it is easy updating the demo app. I haven't tried updating yet, probably will soon, but I am wondering why do you find it easy, it sounds the opposite to me. For example I did a bunch of small PRs in the last couple of days, did none of those changes on the demo app -> how will that be easy to transfer? Soon we will probably refactor the whole code to be vertically organized -> that sounds tricky to transfer also? Am I missing something? What is the diff really between the demo app and the template -> is it trivial? I can dig this into further, but if you can give me a quick overview that would give me a good start.

Likely approach

Ok, I would say likely we go with multiple dirs (template/ and open-saas-sh) in one repo then! Let's finish the discussion, but sounds like the obvious direction. I know patch files idea sounds a bit wild, but I wouldn't dismiss it completely yet, maybe I would give it a quick try just to see, it might surprise us. I want to first hear more on the point above, why do you find merging changes easy.

sodic commented 4 months ago

@Martinsos I agree with the likely approach.

I developed a strong opinion that we should have only one way of using the template and that is wasp new -t saas.

I'm still not convinced here. How do you feel about the arguments on why we should also support something like giget (marketing, discoverability, its own product, etc)?

Conflicts keep popping up and complicating stuff when we try to keep it all one thing.

Could you list some examples of this (and also would the same problems persist in the giget solution)? I haven't worked on the thing, so I don't know.

sodic commented 4 months ago

Martin blackmailed me on a call, and we're doing it his way.

(just kidding, he explained everything and it makes sense)

vincanger commented 4 months ago

@Martinsos when updating the demo-app, i've often just been cherry-picking commits over to deployed-version immediately after making the changes. And I don't always migrate every change made to the template to deployed-version (e.g., minor, unnoticeable changes), although this could probably make things messy in the long run.

Btw, I've removed the Use this Template option/button from the repo now...

Martinsos commented 4 months ago

@Martinsos when updating the demo-app, i've often just been cherry-picking commits over to deployed-version immediately after making the changes. And I don't always migrate every change made to the template to deployed-version (e.g., minor, unnoticeable changes), although this could probably make things messy in the long run.

Btw, I've removed the Use this Template option/button from the repo now...

Ok that is what I suspected. I think that will become a problem as time goes, so we should consider how to make this process of updating the demo-app as straightforward as possible.

@vincanger could you just give me a very high level overview here, how does the demo app differ from the template? I expect an answer like "well it is 98% the same, but we added this thing here, and removed this thing here, and changed this title, and that is really it".

Martinsos commented 4 months ago

I started some initial work on this, so I assigned myself to the issue now.

Two main tasks:

Figure out the best workflow for the future (experiment with patching).
Figure out which changes demo app really needs to bring on top of template.

Vince I did some intial digging into the differences on my own, but will still need your help to finalize it. I made a draft PR where I documented what I did so far, and plan to continue from there: https://github.com/wasp-lang/open-saas/pull/154 .

wasp-lang / open-saas