Move Tracker and all other dependencies into separate repositories and NPM

mitar commented 8 years ago

For #11 to really work, we have to move all dependencies into separate repositories/NPM.

laosb commented 8 years ago

I'm against to having 1 package = 1 git repo = 1 npm module pattern. It makes everything here and there and hard to manage.

mitar commented 8 years ago

Note that you can always checkout the git branch to a local directory, and then an npm install of the subirectory. Which in fact would be easier for developer as you would not neet to commit and push before using the changes

I do commonly have to use forks of packages in my apps, like this: https://github.com/mitar/council-app/tree/master/packages (see all git submodules) So I do expect I will have to do something similar with NPM as well. And for me it is important that users of my app can just clone it and run it and it will work. They should not care that some dependencies are forks. And I should not have to publish them.

Also note that I am in favor of 1 package = 1 git repo = 1 npm module, just trying to put all the alternatives on the table.. ;)

I also think this would be the best, despite the overhead.

I'm against having 1 package = 1 git repo = 1 npm module pattern. It makes everything here and there and hard to manage.

And for which approach would you be?

murillo128 commented 8 years ago

I'm against to having 1 package = 1 git repo = 1 npm module pattern. It makes everything here and there and hard to manage.

If packages are not well defined, then yes, code is here an there ;)

laosb commented 8 years ago

@mitar @murillo128 Problem is, not all users can figure out where the bug is happened. Very soon we will be drown in the sea of issues in wrong repos...

martijnwalraven commented 8 years ago

If packages are not well defined, then yes, code is here an there ;)

Maybe that is part of the problem. Meteor core packages tend to be fairly fine grained and have many dependencies with related packages. I would really like to avoid separate repositories for individual packages in meteor/blaze for instance, but maybe part of the solution is to merge some of these packages.

mitar commented 8 years ago

I like how Keybase addresses that: https://github.com/keybase/keybase-issues#other-bugs-issues-comments-questions-and-suggestions

Keybase has a few repos (see below), and we can't expect you to know where to post issues. So post everything here. We'll tag and organize it all.

But their repositories still have issues per repository. So if you know, you go there, if you do not know, you open in the issues repository.

laosb commented 8 years ago

I suggest that we should separate some non-Meteor-specific packages into separated repos, like Tracker and Blaze, but not all.

mitar commented 8 years ago

Meteor core packages tend to be fairly fine grained and have many dependencies with related packages.

But this is in fact something which NPM likes. So if we are really going with "embrace the ecosystem" then we should really go with small NPM packages and one repository per NPM package. Everything is else is going against the ecosystem. See 29 pages of repositories @substack has.

mitar commented 8 years ago

Small packages is also what allows then easier dynamic loading of just needed code. When you have big packages then people start complaining about the code you are loading but not really needing for a particular use.

laosb commented 8 years ago

Look at rails. They didn't separate everything into repos, right?

murillo128 commented 8 years ago

Small packages is also what allows then easier dynamic loading of just needed code. When you have big packages then people start complaining about the code you are loading but not really needing for a particular use.

Also it allows better unit testing, and encapsulation. I like current meteor packaging fine grained style and as @mitar says, it is how node people use to work anyway.

martijnwalraven commented 8 years ago

Some large projects like React or Babel do have multiple packages in one repository. Babel wrote down their reasoning here.

mitar commented 8 years ago

Look at rails. They didn't separate everything into repos, right?

Rails is not NPM/node.js/JavaScript community.

murillo128 commented 8 years ago

Some large projects like React or Babel do have multiple packages in one repository. Babel wrote down their reasoning here.

This is dumb! Nobody in open source does this!
React, Meteor, and Ember, among others, do this.

circular reference found! XD

martijnwalraven commented 8 years ago

This looks useful (linked to from the Babel document).

mitar commented 8 years ago

Lol: https://github.com/babel/babel/commit/b23ffca1cb7e5c26393ef1d6e06dd4d07dec0ad1

mitar commented 8 years ago

Hm, this looks like exactly what you are currently doing with Meteor, when you have to bump all package versions when you do a new release: https://github.com/kittens/lerna#publishing

So are we changing this or not? Do we then even want to move away from meteor/meteor?

murillo128 commented 8 years ago

One question, wouldn't the monorepo, multiple npm packages be very similar to the original proposal of one umbrella repo with one submodule per package??

martijnwalraven commented 8 years ago

We definitely want to move away from Meteor releases pinning versions of packages. But it may make sense to consider Blaze something that should be versioned, tested and released as a whole.

mitar commented 8 years ago

But it may make sense to consider Blaze something that should be versioned, tested and released as a whole.

But that should not have anything with versions of dependencies, no?

murillo128 commented 8 years ago

git submodule foreach "npm version mayor && npm publish"

martijnwalraven commented 8 years ago

But that should not have anything with versions of dependencies, no?

No, but it might mean versioning and publishing all Blaze-related packages together.

murillo128 commented 8 years ago

No, but it might mean versioning and publishing all Blaze-related packages together.

one umbrella repo (blazejs) with one submodule per package

git submodule foreach "npm version mayor && npm publish"
npm version mayor && npm publish

blazejs could be just a proxy

module.export = require('meteor-blaze');

mitar commented 8 years ago

OK, my take on a summary of how I read this issue and why it is hard to decide:

we would like codebase (and permissions for git push access) around Blaze to be easy to manage, multiple repositories might make it hard to do so
there is an advantage of code being in multiple small packages, because you do code reuse, which minimizes the code served to the client, but also allows possibly lazy loading of only those parts of the code you need
we want each such package to be a NPM package
if we allow easy forking of a package, then this might make it easier to develop and test new features, by forking I mean that you can temporary (for development) or permanently (in an app) use an alternative version of a package without really having to publish it, just with a git reference
NPM does not support dependencies on a git repository subdirectory as a package

mitar commented 8 years ago

Labeling with "help needed" because we need some community help to resolve the design here. Does anyone know of any trick here to do this to allow all wanted properties? Currently it seems there are two options:

we go with one repository, one Meteor package, one NPM package approach and have some overhead maintaining all this
we go with monorepository, but then people will not be able to depend on a git fork of a NPM package (they can use npm link temporary for development, but cannot do depends in package.json)
an middle-ground way of using git submodules, to create a monorepository

I do not think using git submodules would really address the main issue of one repository per package (you would still have to maintain permissions for each subrepository).

Personally I am OK with monorepository, just that we find a way for using git forks with NPM. Alternatively, we could have repository per package, and then remove issues on all other than this repository, which would serve as a hub for the whole project.

laosb commented 8 years ago

@mitar

Alternatively, we could have repository per package, and then remove issues on all other than this repository, which would serve as a hub for the whole project.

Sounds not bad, but still hard to count contributions. Monoreposity should be right, and it's not a common case to use a temporary local package, I think. If that's something we should have but we don't have, merge it; else he may want to maintain his fork and publish it.

mitar commented 8 years ago

Sounds not bad, but still hard to count contributions.

For that we can have scripts.

Monoreposity should be right, and it's not a common case to use a temporary local package, I think.

The problem is that you cannot have an app which has a fork of one package. You would have to publish a package just to have a slightly customized package for your particular app.

mitar commented 8 years ago

I was looking into how vue has things and I like it:

common organization
many repositories
plugins/packages per repository, it seems
separate repository for documentation/landing page (allows more people to collaborate there, like for translations)

Maybe we should just go with Blaze organization. But some dependency packages are general. But maybe we can just decide on them which project is overseeing them (Apollo, Blaze, etc.) and then package is there.

aadamsx commented 8 years ago

NPM does not support dependencies on a git repository subdirectory as a package

Just curious, what's the rational behind this?

I was looking into how vue has things and I like it

I agree, I like the way vue has organized its project.

Maybe we should just go with Blaze organization. But some dependency packages are general. But maybe we can just decide on them which project is overseeing them (Apollo, Blaze, etc.) and then package is there.

What are the downsides to this approach?

mitar commented 8 years ago

Just curious, what's the rational behind this?

Read here: https://github.com/npm/npm/issues/2974

From my understanding, the only argument is "git urls would be too complicated". (But Python's pip does support this, so, it is possible.)

What are the downsides to this approach?

One has to manage user across repositories, but I think with new GitHub teams this might be easy.

Users might have issues knowing where to contribute something, where to open issues, and so on.

Personally, I do not mind this so much, but some people from MDG disliked it. Maybe if it would be inside meteor organization it would be complicated. But maybe we can create a new organization.

laosb commented 8 years ago

Maybe we should ping MDG members about this. @stubailo how do you think of having a separated organization.

aadamsx commented 8 years ago

Personally, I do not mind this so much, but some people from MDG disliked it.

Does MDG dislike it because of this:

One has to manage user across repositories [...] .

Users might have issues knowing where to contribute something, where to open issues, and so on.

If MDG is apposed to this approach, how will we bridge the gap?

But maybe we can create a new organization.

I thought starting an new org to manage Blaze and related tech was the plan all along...

mitar commented 8 years ago

If MDG is apposed to this approach, how will we bridge the gap?

Read comments from @stubailo and @martijnwalraven above. How do you read them?

I am reading comments now again and it seems @stubailo is more for the blaze org, while @martijnwalraven would prefer that we stat with one repo and maybe move later on.

But it seems it is pretty clear that we need more than one repo. So maybe we should just go with blaze org?

mitar commented 8 years ago

Do we know how has or is https://github.com/blazejs?

MiroHibler commented 8 years ago

No, but we can ask support for help ;)

mitar commented 8 years ago

I just did. :-)

aadamsx commented 8 years ago

So maybe we should just go with blaze org

I agree. What do we need to get the ball rolling on this aspect?

EDIT:

brain-storming: Do we set up a actual org or just push all our org democratic-parallel efforts around a domain (in this case blazejs.com) that some individual owns (in this case @voidale)?

mitar commented 8 years ago

I think we can work on all this in parallel. :-)

Let's make it a community effort. I prefer doocracy. Who ever wants to do something, should do it. We can always consolidate later on and move things under the org umbrella.

(Of course doocracy also means that some things might not be used later on. Or could be improved by others to make it much different.)

So, what I am saying is: if you have an idea how for a landing page (that one could be in Blaze) and docs (I might prefer static stuff here, so maybe GitHub pages + GitHub hook to compile on Travis CI), just go for it!

mitar commented 8 years ago

(But let's discuss docs and site in their own tickets.)

aadamsx commented 8 years ago

Are you going to start a new issue around a Blaze org effort (so I can post stuff to that)?

mitar commented 8 years ago

I think blaze org effort belongs here. But docs (#19) and landing page (#24) do not.

aadamsx commented 8 years ago

Ok, I edited my post before last to remove the website mention.

mitar commented 8 years ago

GitHub replied:

blazejs account and it's not dormant

So, we need some other organization name.

aadamsx commented 8 years ago

If we weren't tied to the blaze name, we'd have more options.

https://github.com/infernojs.com and infernojs.com

are available for example.

MiroHibler commented 8 years ago

So, have we reached a verdict on this one? I think we're losing too much time on such administrative effortless stuff. Let's JFDI.

brown2rl commented 8 years ago

I agree

Sent from my Windows Phone

From: Miroslav Hiblermailto:notifications@github.com Sent: ‎4/‎25/‎2016 3:47 AM To: meteor/blazemailto:blaze@noreply.github.com Subject: Re: [meteor/blaze] Move Tracker and all other dependencies into separate repositories and NPM (#13)

So, have we reached a verdict on this one? I think we're losing too much time on such administrative effortless stuff. Let's JFDI.

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHubhttps://github.com/meteor/blaze/issues/13#issuecomment-214188730

mitar commented 8 years ago

I think I would say that we go with organization and multiple repositories, per package. The issue is that we do not have a good organization name. blazejs is taken. What about meteor-packages? Or meteor-blazejs?

MiroHibler commented 8 years ago

I vote for meteor-packages.

mitar commented 8 years ago

Correction, meteor-package. So that it is then meteor-package/blaze and so on.

One idea above was also meteor-community.

Anyway, let's see wait a bit longer to get output from others.

laosb commented 8 years ago

meteor-community should be fine. I've taken it and I can transfer it to you anytime.

meteor / blaze

Move Tracker and all other dependencies into separate repositories and NPM #13