Atmosphere police

[dandv]

While evaluating various packages for a large project I'm undertaking, I've seen countless examples of abandoned packages and migration gone wrong. As an example,

https://atmospherejs.com/?q=d3

produces 26 packages, of which many are abandoned, no longer work with 0.9+, and have been superseded by forks that do. I've flagged each, and I've contacted the authors asking them to un-migrate them, after hunting for them via arcane Google searches (some have changed GitHub usernames etc.). If you look at my GitHub activity stream, you'll see a ton of issues and PRs like "Please remove package", "Update for Meteor 0.9", or "Please add Atmosphere URL".

I don't exactly enjoy being OCD about this, and I could use some help. The vast majority of users will not bother to flag mrt: packages that have been superseded, but they do clutter up the Atmosphere search results. What can we do about this? Here are two ideas.

1. Mercilessly flag broken packages.

If it doesn't show a README in Atmosphere, it means the author hasn't bothered to add a git URL, or it's an automatically migrated package that needs help. Flag it!

I've carefully flagged packages superseded by newer forks. If some other folks can please go through search results and re-flag them, the threshold for flagging should hopefully be met. Problem is, they can't see my flags.

2. Work directly with library maintainers to provide official integrations.

Here's how to submit a Meteor packaging PR to popular libraries.

I hope other mechanisms can arise. We're bragging about 2500+ packages on Atmosphere, but my guess after ~two weeks of combing Atmosphere is that half of them are broken in one way or another. This is not exactly a source of "trusted packages".

[dandv]

Here are actual stats from this d3 Atmosphere search. They're very similar to previous searches I've done for other packages (DataTables, forms, maps, you name it) - half the results are a serious waste of time.

UPDATE In the meantime some folks have flagged some of these packages, and they show as flagged globally - thank you! We do need more flags - the more popular a package is, the more flags it needs (10% of the number of downloads).

Old forks obsoleted by https://atmospherejs.com/d3js/d3

https://atmospherejs.com/garrilla/d3 https://atmospherejs.com/meteor/d3 https://atmospherejs.com/sergeyt/d3 https://atmospherejs.com/mrt/d3js-package

Old forks obsoleted by https://atmospherejs.com/pfafman/nvd3

https://atmospherejs.com/mrt/nvd3js https://atmospherejs.com/mrt/nvd3-revised https://atmospherejs.com/mrt/meteor-nvd3 https://atmospherejs.com/aramk/d3

Old forks obsoleted by https://atmospherejs.com/peernohell/c3

https://atmospherejs.com/kylesuss/c3

Now Futuregrapher

https://atmospherejs.com/mrt/d3graph https://atmospherejs.com/futurescaper/d3graph2

Now without dash in name

https://atmospherejs.com/kidovate/dagre-d3

Junk

https://atmospherejs.com/test/1j4psd3

---

26 search results for d3, 12 junk.

The star weighing doesn't always help. For example, the most up-to-date d3 wrapper is https://atmospherejs.com/garrilla/d3, has 3 stars, yet it's below "See More". This proves again that not enough users bother to flag broken packages and star working ones.

Is there anything better we can do than wait indefinitely until the community becomes large enough to weed out junk packages (no pun intended)?

How many users do we miss because of the confusing experience they encounter on Atmosphere?

[tmeasday]

Hey @dandv,

Thanks for taking the time to write this up. I think you make a pretty solid point.

First off, I think everyone agrees that there's not much value in these front end wrapper packages that don't really do anything. Exactly what the solution is is a topic for another forum (and a conversation that's progressing right now I think).

---

To address some of the specific issues you have:

1. Star weighting - actually you've uncovered an issue with the scoring here. The problem is that garrilla:3d didn't have proper scores, not that the starring isn't working. Will sort this out ASAP.
2. In terms of not enough users flagging packages -- Of the 24 packages I see for the search, 7 are visibly flagged. So it looks like the system is getting there, slowly. Perhaps you've made efforts to get others to flag these packages though?

---

We're wondering if it might make sense to bless certain active and trusted community members as "moderators" and to make it so their flagging has more weight (perhaps they insta-flag).

Would this make sense to you @dandv ? Or do you think a more democratic solution is better (perhaps simply lowering the required # of flags to trigger the global flagged state is all that's needed)?

[dandv]

Moderators as a solution have worked well for various software development and Q&A communities, and I have no reason to believe it won't work here. I've been performing this voluntarily for the past weeks and am happy to continue to help. As a safety measure, perhaps no moderator on their own should be able to determine the fate of a package, to avoid the situations like those encountered on StackOverflow.

The only problem with lowering the required # of flags that I see is competitive or revenge downvoting, but my sense is that that would be exceedingly rare. Again, requiring a minimum of X downvotes (which should not be published), would be a defense sufficient for the majority of cases.

Finally, making it easier for users who volunteer to retire their packages would be great. Something like a /retire route on the package, presenting the logged in user with a button to remove the package from Atmosphere (unmigrate for now) would enable me and other volunteers to directly paste that link in GitHub tickets asking them whether they're still maintaining a forgotten repo (example).

[raix]

Maybe there could be a consensus about using the version of a package as an indicator for how stable the package is? I normally use very low versions on packages that havent matured or isnt production ready. We have kind of a problem on the cfs team that its a pre-pre-release thing, but folks are using it and "demanding" bugfixes. I guess now we could mark it -alpha.1 etc. but the package system could perhaps use this info too?

The publish activity of a package is to me the most important indicator, its telling me that the maintainer is doing his job/ its not a dead package.

There are cases where a package could be really old and work perfectly - a reason why the maintainer isnt publishing new versions. (rare)

So a small "police corps" keeping maintainers on their toes are ok, but it could also be an automated thing where packages die if they are not published/updated every 6 month. Now this would result in bad numbers for Meteor marketing - on the other hand it would ensure that only maintained packages live.

The thing I personally would like very much is a way for package writers to be able to make a living on creating and maintaining packages. A package that is funded should weight more, since it would be maintained properly with the goal of being production quality. But theres no business model for package writers yet.

Just some thoughts, sorry about interrupting

Kind regards Morten

[hedcet]

i think - this is the first problem to solve in meteorjs packaging system

please keep different versions of a package under a single package name. In the case of bootstrap there were several packages like bootstrap, bootstrap-3 and so on.

@dandv http://ww2.meteor.com/spreadjs/linto i found atlest 81 package contains "bootstrap" in their name

[splendido]

wow, interesting link... The table seems not updated thought, how can this be used?

[dandv]

@splendido: which table? I've updated the list of d3 packages, striking out those that have been flagged. There are still quite a few left to flag. Come on, community! :)

[splendido]

sorry @dandv I was referring to the table you can see looking at the link posted by @HedCET

I think you're doing a great job here!!! ...but at the same time it cannot be only you! there should be a better way to improve the commitment of the community to keep he package database clean and up to date.

It doesn't make any sense you can find ~80 packages for bootstrap! I'd say the best way would be creating a Meteor organization called 'bootstrap' and then have packages like:

bootstrap:css-3.3.0 bootstrap:css-3.3.1 bootstrap:less-3.3.0 bootstrap:less-3.3.1 bootstrap:sass-3.3.0 bootstrap:sass-3.3.1 etc..

then the community should be somehow trained to prefer such a 'pattern' when looking for popular packages ported to meteor. At the same time the organization should accept a number of contributors among its member and keep a github repo where to receive issues and keep packages up to date.

The thing that should also be avoided is to publish 'private' packages which have only slight modifications w.r.t. 'original' ones simply to have them ready to add to your next app. This is a different problem and it is left to each developer to get more confident with local package development before thinking to publish a package for 'private' use.

Perhaps some more 'teaching' pages on Atmosphere about this kind of behaviors could help a bit more?

[udondan]

bootstrap already is taken as either a username or organization and AFAIK there's no way of finding out who's the owner. A while back I registered the org bootstrap3 (and with an eye on the future as well bootstrap4 and 5... ;-) ) for a canceled project. Unfortunately there currently is no way to delete an org but I'm of course willing to invite everyone who wants to use it for above suggestion.

[dandv]

@udondan: you can ping @estark37 to delete orgs or convert users to orgs. She's helpful :)

I've also reserved a large number of orgs, because someone who was not very nice, had reserved moment but apparently did nothing with it, and the official moment maintainers can't use it now.

[splendido]

this is surely another problem... 'bootstrap' was just an example to propose a pattern ;-)

[splendido]

hey @HedCET, in case 'fontawesome' is an organization, you can go to meteor.com login with your regular username look for the 'organization' 'tab' under your account settings and 'invite' new users based on their meteor developer usernames.

In case 'fontawesome' is only the package name, hence 'linto:fontawesome' you must use the meteor CLI to add another maintainer via meteor admin!

[dandv]

Thanks @splendido for answering that SO question - you should repost there and get the points :) PS @HedCET: might want to pay a bit more attention to the writing!

[splendido]

do you guys think we should try to put some pressure on organization/users (like @dandv is very well doing) also pinging MDG to eventually depose the current owner and give back the 'organization' to the community? How much would Meteor gain from such a 'rude' operation? Probably something to be considered?

[dandv]

I've already asked Emily to put me in touch with the moment guy. It would be the nice thing for him to add me and @ichernev to the organization.

On the other hand, sometimes this borders into trademark infringement. Someone has reserved "google", and I happen to be the only Meteor dev who works at Google. We plan to release some Meteor packages but the namespace is taken. Rather than have our legal/trademark people start talking to MDG/Percolate, I'd prefer to contact that user, who hopefully wanted to do something innocent (maybe wrap some Google API).

[dandv]

@HedCET: can you please add me to the fontawesome organization?

All: I've sent a pretty solid PR to Dave to add official Meteor integration to Font Awesome. Check out the PR files - note the font download tests and the visual checks.

[splendido]

@HedCET, IMHO fontawesome:fontawesome is not the better choice since 'fontawesome' is already clearly standing out from the first part. What about fontawesome:4.2.0, fontawesome:3.2.1, etc?

[dandv]

The actual organization behind fontawesome is fortawesome (see their GitHub). This is why I've published the official integration at https://atmospherejs.com/fortawesome/fontawesome. Please star!

[splendido]

@dandv, just saw it, was writing right now you took a better path...

so: fortawesome:fontawesome-4-2-1-less etc? Not sure whether the dot is allowed inside names...

[splendido]

ok, perhaps it's clearer to have only the name without the version, actual package version will reveal the version of the original package to which the meteor package is linked. Possible mistakes in publishing operations could be absorbed by publishing versions named M.m.p_w as explained in the official documentation about writing packages

[dandv]

Hey all, I've updated my guide on creating official Meteor integrations. Good luck, happy to answer questions and reach a large consensus!

[dandv]

@HedCET - nice table with lots of packages!

1. How did you build it? Where is the source of the data?
2. Can you add the description column? That explains why we see 81 results for "Bootstrap", when there are far fewer wrappers.
3. Can you enable sorting? We'll see the most starred/downloaded/etc. packages.
4. ww2.meteor.com looks official, like a secondary MDG server for www.meteor.com. You don't want to impersonate them, do you? :)

[dandv]

Of course I had click on column headers; nothing happened. Just now I tried to right click (a bit counter-intuitive) and managed to sort.

And I know ww2 is a subdomain just like "autocomplete".meteor.com, but it looks official (see http://stackoverflow.com/questions/1692273/why-ww2-sub-domains), as if it was maintained by MDG.

[hedcet]

@dandv why u don't touch mizzao:bootstrap-3, i think that's also a thirdparty package and most downloaded

[splendido]

is it me or there are a number of previous post disappearing above here ;-)

[dandv]

@splendido: @raix mentions in the README that he might edit the posts, and to complain if you "feel violated" :)

@HedCET: Hey @mizzao, your package is next! >;-> Seriously, a PR to have official Bootstrap integration straight from the Twitter guys would be great. I've laid out the steps here, and could use some help!

Problem is, the bootstrap org is taken. Anyone knows who has access to it?

[splendido]

@dandv I've read the README from @raix, but here we're on another repo (which as no README.md file at all...), not sure he could remove posts from this thread ;-)

Btw, no worries at all, my posts are still all here. the ones disappeared were from @HedCET...

[dandv]

@splendido: right, I was confused :) Too much packaging today #needsleep

Well, @HedCET might have edited his own messages. I'm not a contributor to this repo so I can't edit/delete.

BTW I find it disturbing that repo contributors (not just owners) can silently delete or edit anyone's comments without a trace, not even an "edited" label like on SO or FB. Hey @mdo, @schacon, @josh, @arfon, @tnm, @github, thoughts?

[splendido]

Ok, lets stop this here... I'm not that good at first fights ;-)

I had no intention to bring up such a discussion. I only though someone was trying to hide his traces and that made me a little upset. I didn't even know @raix has some power here..

Btw, thank you all for the clarification!

[raix]

Good point @splendido I can only delete my own comments here... Think @dandv thought we were in https://github.com/raix/Meteor-community-discussions and I only read his comment.

[dandv]

@HedCET,

@dandv why u don't touch mizzao:bootstrap-3, i think that's also a thirdparty package and most downloaded

I just did: https://github.com/mizzao/meteor-bootstrap-3/issues/14

[dandv]

@gadicc: what sort of role model are you? ;)

[gadicc]

Apparently a very bad one :) We were using this to debug the meteor publish fail on some packages from a while back. Anyways. Set unmigrated :) Thanks for the heads up.

[mizzao]

Seems like a long discussion, but I would be happy to hook releases into the build process of other packages and reduce the maintenance load, which is currently just very dumb work.

I was mentioning in https://github.com/mizzao/meteor-bootstrap-3/issues/14#issuecomment-64942535 that this process would be a lot easier with package renaming fully implemented, so we can take the most popular version of a package and just migrate that to an "official" one, so that continuity could be maintained.

Ping @ekatek and @n1mmy on if/when that would be possible?

[awatson1978]

The publish activity of a package is to me the most important indicator, its telling me that the maintainer is doing his job/ its not a dead package.

There are cases where a package could be really old and work perfectly - a reason why the maintainer isnt publishing new versions. (rare)

Usually Morten and I have similar views on industry practices and whatnot, but I'm going to respectfully disagree on this point. In most healthy package ecosystems, it's quite common for packages to reach maturity and stability and not need constant updates. Moreover, for someone seeking package stability, publish activity is usually a sign of breaking changes, or something fundamentally wrong with the package.

People are forgetting that there have been three major breaking changes in Meteor API in the past year that impacted how libraries get integrated...

• 0.6.5 which introduced Npm wrappers and various other features
• 0.8 which introduced Blaze and rewrote how to integrate libraries
• 0.9 which introduced Isopacks and namespaces

The breaking changes in core have distorted the ecosystem such that only the packages with regular maintenance activity are keeping up with the breaking changes in core. Now that 1.0 is out, and there is a commitment to backwards compatibility with older versions, this dynamic is going to change. People need to plan for that. Skate to where the puck is going to be, not to where it currently is.

Moving forward, I think it's going to be a bad idea to assume that any packages inactive for more than 6 months should be flagged or removed. That's skating to where the puck used to be.

[splendido]

What about developers that for a repo and publish it under a different name without changing one line of code!?

https://atmospherejs.com/voidale/accounts-ui-voidale

the github link links to the original repo!!!

Disclaimer: I'm the maintainer of the forked package...

[dandv]

I've seen the case @splendido mentions, many times. People fork a package and just republish it under their own namespace, without bothering to document how the fork is different, or even pointing to their own repo.

What's the excuse for that?

[awatson1978]

Most likely, they're simply learning how the package management system works. Or don't know best practices on documentation. Or were starting on a project, and MDG changed the API on them, causing them to give up. Or have other time commitments they have to take care of first in order to pay the rent.

Have compassion for other people's lives. And remember that any one of those people who managed to get a package forked and republished under their own namespace is a potential package maintainer in another 3 or 6 months.

[splendido]

the fork lives here and the last commit is mine ;-)

I'm not saying we're all born experts and I know we all have hard ways to learn. Only one year ago I was among those who knew nothing about web development and I'm not that much better now...

But using public spaces exclusively for your own interest without even considering it's a shared place is not among the things I can explain to myself...

I'm with you @awatson1978, I'll have compassion for @voidale hoping he's going to be among the team of useraccounts in a few months!

For sure I've always replied to all issues accepting almost all of them whether they were critics, suggestions or PRs. I'd like developers to contact packages' maintainers to see whether there is room to adapt them to their needs before starting to publish useless forks! ...private forks for private use are mandatory if you have something very specific you need to modify!

...and this is not because I'm jealous of my packages, their open source under MIT license in the end. But I always think that others might get confused and upset while looking around on Atmosphere if it is not clear what you're finding out.

In the end, Atmosphere is like a public garden for Meteor developers... ...I don't know how is it for you, bu I usually get very upset when I go out playing on a public garden with my children and found it full of dogs' poop ;-)

Atmosphere is also my garden in the end! :-)

[splendido]

If I'm not wrong, both the official Meteor documentation and Atmosphere under Learn more about Meteor package management are missing a section explaining how to clone/fork packages to work on them locally.

I've just added a section like this here ...copying a bit from here I admit!

Do you think having suck a guides more spread and officially supported might make things better for what we're discussing here?

[dandv]

The comparison of Atmosphere with a public garden is an excellent one, @splendido.

Indeed, there's no official easily accessible documentation (i.e. on docs.meteor.com) on how to fork package. There are some hackpads, but those are in a state of flux and look confusing even to me.

A first step would be for MDG to be a role model of packaging, and they're moving in that direction

GitHub linking bug

Hey @github, I'm editing this from the web UI, and want to make a link.

Pasting from GitHub support email:

GitHub linking bug

[splendido]

It seems we're going to have a dedicated section on Atmosphere about how to work locally with packages. See this issue :)

[dandv]

Kind of annoying to see yet another package that does exactly what an established pacakge does, @SachaG's Spin.js wrapper, without giving any credit or mentioning prior art. Please flag https://atmospherejs.com/hckrs/spin. And how's that moderator flagging feature coming @tmeasday ?

[tmeasday]

Yeah definitely @dandv ! We'll prioritize it for next sprint

[dburles]

I don't entirely agree in this case. Who's to say that Sacha's version is the be all and the end all. Flagging this new package and others in a similar case in the long term could end up stifling overall growth and quality of the package ecosystem.

This is one reason why I am not entirely convinced about a moderated approach to atmosphere.

While the package does use spin.js it's not simply a fork.

On 19 Dec 2014, at 6:09 pm, Dan Dascalescu notifications@github.com wrote:

Kind of annoying to see yet another package that does exactly what an established pacakge does, @SachaG's Spin.js wrapper, without giving any credit or mentioning prior art. Please flag https://atmospherejs.com/hckrs/spin. And how's that moderator flagging feature coming @tmeasday ?

— Reply to this email directly or view it on GitHub.

[raix]

I'm not sure about flagging either - I would contact the maintainer first to hear if they wanted to deprecate the package in favor of an official version. But we have to respect that some packages add extra features.

That said I dont believe anybody actually enjoys maintaining wrapper packages - Rather have two maintainers on one package than two packages with each one.

[dandv]

The package may very well be better than @SachaG's, but how? The author doesn't even mention the existence of the original package, let alone reasons why a user would choose one or the other.

It's a question of open source ethics. Give attribution, dude.

Note - I had already contacted the author of that package regarding a different wrapper they published, for summernote. I had approached the original summernote devs, submitted a PR, and they accepted it. Went back to @Jarnoleconte with an update - nothing. Yet he did have time to publish another wrapper.

So I haven't mentioned this entire backstory, but as a moderator, I'd watch for a healthy open source contribution attitude, a collaborative ethic, as well as package functionality.

[raix]

@dandv fair enough - We should give always give credit

[JarnoLeConte]

Hi @dandv, I got a few notifications about package management of you, so I will answer these. First of all I'm not really a package maintainer. We developed meteor-spin and meteor-summernote for a local project and I just published the packages because I have learned that it is good to share code. I wasn't even aware of the fact that there was already a spin package out there.

I have looked at the other spin package and see some differences. So maybe I can add a comparison to the readme for example. Also we will try to contact the authors of spinjs to do a PR request. That is what you call it right?

Also I will hide the summernote-package as you suggested. But what in the case we will add significant changes, because we have a plan to fully integrate summernote within the meteor style of programming. Must that always be done in the official package? Because that is not always what everyone likes, I think?

[dburles]

That's a good point @Jarnoleconte often packages will require some work to get them to play nicely with Meteor and there's definitely going to be more than one way to do that. Take google maps for example (I'm still iterating on this API) https://github.com/dburles/meteor-google-maps.

Personally I think the official packages should provide a proper Meteor integration. And if you fork the package and write one, I see no reason why you shouldn't then submit a PR. A lot of packages (such as summernote) require a bunch of work to integrate, it makes no sense for everyone to have to write that code themselves since it's supposed to be a Meteor package after all...