search

chaijs / chaijs.github.io

The chaijs.com website source code. Contributions welcome.
http://chaijs.github.io
49 stars 70 forks source link

Grunt-like plugin listing #34

Closed vesln closed 8 years ago

vesln commented 10 years ago

http://gruntjs.com/plugins

I think it's pretty neat and we'll stop forcing the people to submit a pull requests but fetch plugins automatically. I'll be doing the same for hydro.

@logicalparadox

keithamus commented 9 years ago

This idea has a lot of merit, jQuery recently moved to this approach, and customelements.io.

To do this we'd need to make it a requirement that every plugin has 'chai-plugin' (and/or ecosystem:chai) as a tag in their package.json. It would be worth spending time evangalising those that don't - but are on our current list - before we implement this on the site, to avoid missing any plugins.

To get a list of plugins on the website, we could either simply link to https://www.npmjs.com/browse/keyword/chai-plugin and allow users to browse npm for plugins.

If we want to keep the plugins page, I cannot find any docs about an available npm registry api for retrieving JSON of packages by tag name - which could be a blocker for this - this may mean we have to manually download a cache of the registry and parse through it regularly (just like npm search does).

keithamus commented 9 years ago

I managed to dig up an endpoint here which shows a list of npm packages that have the 'chai-plugin' keyword. The keys array is an index of [0] being the keyword, [1] being the package name and [2] being the description.

It doesn't fit our exact use case - if we are to maintain feature parity we'd ideally need to have the following:

For each plugin page (e.g. http://chaijs.com/plugins/chai-spies) we could probably get away with making a request to the individual package page (e.g. http://registry.npmjs.org/chai-spies) which has all of the necessary details.

The other problem is that the URL does not support CORS - meaning any XHRs we make to it will fail, without a proxy layer.

Maybe someone at @npm could help us with how to tackle this.

keithamus commented 9 years ago

Another option would be to have a script which fetches the npm search cache (https://registry.npmjs.org/-/all/since?stale=update_after&startkey=TIMESTAMP), and builds a plugins.json file from that - by having the search cache we'd have access to all metadata about a package.

The problem with this process would be the regularity of updates from which the plugins.json file is populated. We could use something like the travis ci cron trigger to run a nightly build which updates the plugins.json so at a minimum the list gets updated nightly.

logicalparadox commented 9 years ago

Alternatively we could separate the plugin listing from the main website and host it on a subdomain (http://plugins.chaijs.com). That way we can have the main site as a gh-pages thing and the plugins run separately. If we dockerize the plugins app (or alternatively the whole thing), I can migrate it to my more reliable infrastructure, and automate the release of said assets on github webhooks.

keithamus commented 9 years ago

@logicalparadox while I'd certainly enjoy putting together a dockerized app, the beauty of having a fully static site and using gh-pages is that it's pretty much fire-and-forget, auto-releasing with virtually zero configuration, and maintainable by anyone of our contributors, regardless of server side experience. It also has the added benefit of being free at every scale, while using Heroku/Amazon/<insert host here> comes at some cost, which also means billing & bus factor concerns. I guess its a case of "spend a lot of time once to get it to work nicely as a static site" vs "spend a little bit of time regularly to maintain upkeep of server, billing, upgrades etc".

keithamus commented 8 years ago

As a task for doing this, we need all chai plugins to update their package.json#keywords to include "chai-plugin". A list of the current plugins we know about is here: https://github.com/chaijs/chai-docs/blob/master/plugins.js. Someone needs to go through each one, and raise a PR that adds the keyword. As an example commit, see chai-bignumber#a742467

keithamus commented 8 years ago

Made some significant process with this. All plugins have their contents fetched, and the script we have pulls in plugins from npm with the keyword chai-plugin.

_legacy_plugins exists which is just an enumeration of all of the plugins that /don't/ have the chai-plugin keyword, but we still want to display on the website.

As per my previous comment - we need to go through each plugin listed in _legacy_plugins, and raise a PR to add the keyword to the package.json.

astorije commented 8 years ago

@keithamus, added the keyword to my plugin \o/ (https://github.com/astorije/chai-immutable/commit/39540ffc1f4f43f7aa3175eef7a90d59b46665b6). Congrats and thanks on this, it sounds cool!

Comparing to the current solution, I have a few questions though:

Of course, most of these questions assume you are keeping a 1:1 functionality with the current look of the plugins, so maybe it's irrelevant :-)

Thanks again and good luck on shipping this!!

keithamus commented 8 years ago

I'm guessing you are retrieving the description from package.json, right?

Yup this is correct. Seems like most (or even all) of the descriptions are fine for this purpose.

The name given in package.json and the actual name of the plugin will often be slightly different (for example, mine is "chai-immutable" vs "Chai Immutable". What's your recommendation/solution on this?

Personally, I think this is an acceptable compromise. chai-immutable is your canonical name in Github, npm and possibly bower. Using a friendlier name might look nicer from a design/typography perspective, but as a developer I really want to see the name I use when I run npm i .... If this is unacceptable to plugin developers, we could perhaps get plugin developers to add a friendlyName field to their package.json.

For the tags, will you now use the keywords listed in the package.json minus chai and chai-plugin? What happens to keywords like test, assertions, ... that are useful in the npm page but probably not on Chai's docs?

My plan is to filter out tags that are not relevant to us. So far this includes chai and chai-plugin, but I will certainly add test,assertions, probably also tests, testing, plugin, assert, unit, acceptance, and maybe spend some time talking with plugin developers about curating a nicer list of tags - for example some plugins have jquery while others have jQuery (capital Q).

How are you planning to display that plugin X does/does not have Node.js/browser support, and the file to use for browser?

Good question. I have been mulling this over and I'm not quite sure. Here are the options I've thought about:

P.S. support for browser and/or engines have been discussed as a codified way to handle browser packages within npm: https://github.com/npm/npm/issues/4321.

I can see how easy it can be to overload Chai plugins page by adding the chai-plugin to npm projects. How are you planning to blacklist/remove these?

npm-plugin-fetcher supports blacklists - so if we need to we can blacklist modules from appearing on the plugins page. Its a low-key solution and hopefully we wont have to use it. I'd rather just have more stringent checks on a package.json - for example exclude packages that don't have a repository key, or packages with a README of less than some number of words, etc.

Of course, most of these questions assume you are keeping a 1:1 functionality with the current look of the plugins, so maybe it's irrelevant :-)

While I'd like to keep a reasonable degree of parity, I think we'll definitely not end up being 1:1. I mean, we're already not - considering plugin authors are having to change their package.json to accommodate the new system, but this, to me, is an acceptable tradeoff towards getting a simple, automated plugin system.

If you have any feedback on the above, I'd love to hear it.

keithamus commented 8 years ago

Going to settle on a package having the keywords of browser if they support browsers, and browser-only if they do not support node, only browsers.

The following plugins need the following keywords added to their package.json files. Any one who reads this who wants to make PRs to these projects, I'd ask them to do so.

As and when these are done, please let me know and I will tick them off of this list. If they are not done by the time the new website goes live, I'll try to get round to making PRs for the remainder, but it is important to know that the site will go live even if not every plugin has the requisite tags!

Plugins

Note To Plugin Authors

You've been @mentioned because you have a chai plugin. Firstly thank you for being awesome! Secondly, sorry for the potentially unsolicited emails/notifications! Thirdly - in the above list - you'll find your plugin(s) and some keywords that I would like you to add to your package.json. You may also expect a PR adding these in from one of our contributors, or me if I get round to doing it before you. If you're wondering why I want you to add these keywords, here is the reasoning for each one:

If you edit your package.json, please drop me a note here so I can cross your plugin off of the list. Thank you for your time with this, it is much appreciated, and sorry if this caused you any inconvenience.

krampstudio commented 8 years ago

Here we go https://www.npmjs.com/package/chai-xml

matthijsgroen commented 8 years ago

:+1:

keithamus commented 8 years ago

@krampstudio thanks for updating chai-xml. Feel free to hit the unsubscribe button on this thread - as it'll probably get pretty messy in here. If you have any free time and fancy making PRs for the other repos it'd be huuugely appreciated :smile:

johngeorgewright commented 8 years ago

@keithamus keywords updates and published

keithamus commented 8 years ago

@johngeorgewright hi-hive :hand:! Feel free to hit the unsubscribe button on this thread - as it'll probably get pretty messy in here. If you have any free time and fancy making PRs for the other repos it'd be huuugely appreciated :smile:

matthijsgroen commented 8 years ago

@keithamus Chai-changes is updated, chai-backbone will be done later

keithamus commented 8 years ago

Thanks @matthijsgroen :smile:. Let me know when you've done chai-backbone please.

nathanboktae commented 8 years ago

Awesome work here. :+1: chai-dom has been updated and published.

keithamus commented 8 years ago

Sweet, thanks @nathanboktae. FYI you might want to publish with the new keywords - chai-docs will read from the npm registry, not github. Actually same goes for chai-xml (@krampstudio), and chai-changes (@matthijsgroen)

ryan-roemer commented 8 years ago

@keithamus -- Added keywords and published as chai-jq@0.0.9. Thanks!

keithamus commented 8 years ago

@ryan-roemer awesome thanks! I'll say the same thing I'm saying to others: feel free to unsubscribe from this thread, they'll be many more messages coming and you've done your bit in good time. If you want to be extra cool and PR some of the other repos it'd certainly owe you one!

asmarques commented 8 years ago

@keithamus updated keywords and published chai-bignumber@0.9.2 :+1:

keithamus commented 8 years ago

@asmarques sweet thank you very much! Same thing goes: feel free to unsubscribe from this thread, they'll be many more messages coming and you've done your bit in good time. If you want to be extra cool and PR some of the other repos it'd certainly owe you one!

matthijsgroen commented 8 years ago

@keithamus chai-backbone is now done as well

keithamus commented 8 years ago

@matthijsgroen Thanks very much! Also thank you very much for PRing against chai-js-factories! If you fancy making any more PRs then that'd be much appreciated, otherwise I'd suggest unsubscribing from this thread, as you've done your part :smile:.

keithamus commented 8 years ago

Had 5 minutes, so went through a bunch and added pull requests. Most are in the chaijs org - so I or one of the chaijs maintainers can tend to these, but for the others I'd appreciate if the maintainers merged them!

onechiporenko commented 8 years ago

@keithamus chai-string is updated and published chai-string@1.1.4

keithamus commented 8 years ago

@onechiporenko thank you very much for doing this! if you fancy making any more PRs then that'd be much appreciated, otherwise I'd suggest unsubscribing from this thread, as it's going to be busy in here :wink:.

hurrymaplelad commented 8 years ago

@keithamus chai-colors, chai-json-equal, and chaid tagged and published. Thanks for the easy instructions!

astorije commented 8 years ago

@keithamus, added chai-plugin and browser to https://github.com/astorije/chai-immutable, and released on npm (version 1.5.3).

To answer your original answer though, here it is (sorry for the latency, got very busy):

Personally, I think this is an acceptable compromise. chai-immutable is your canonical name in Github, npm and possibly bower. Using a friendlier name might look nicer from a design/typography perspective, but as a developer I really want to see the name I use when I run npm i .... If this is unacceptable to plugin developers, we could perhaps get plugin developers to add a friendlyName field to their package.json.

That's fair, and I'll update chai-immutable rather than poke on features on Chai's side. Better to have consistency across all plugins I think, and the rule here is reasonable.

Good question. I have been mulling this over and I'm not quite sure. Here are the options I've thought about:

  • Browser plugins need to add a chai-plugin-browser keyword in addition to chai-plugin. This will probably be the interim solution we use.
  • We could hook off of the browser key in the package.json. The package.json browser field links to the appropriate build file, e.g. { "browser": "chai-immutable.min.js" }. This helps bundlers like jspm bundle browser deps, and we could use it as a signal that the plugin supports browsers - so it's likely to already be needed.
  • We could abuse the package.json's engine field - to see if package.json#engine.browser exists, we could also use it to get more detailed metrics - e.g. which user agents/browsers are supported, what versions, etc. However this is not a standard inside of the package.json.

P.S. support for browser and/or engines have been discussed as a codified way to handle browser packages within npm: npm/npm#4321.

I was pretty much against the browser and browser-only, pretty obnoxious on Chai's side... but then I saw your PS :-) I like hooking off of existing "standards", although rather than subtractive keywords, I'd have preferred additive ones: browser, node, maybe all-engines, ... But it's up to you at the end, no big deal. Just, if further keywords happen to be necessary and outside of all conventions, make sure to prefix them with chai-plugin-....

Anyway, great work, thanks a lot for this, and long live Chai!

(BTW, thanks for the recommendation, but I am not planning to unsubscribe this thread :-) )

keithamus commented 8 years ago

@hurrymaplelad thank you very much for updating all of those, and kudos for the impressive collection of chai plugins :wink:. If you fancy making any more PRs then that'd be much appreciated, otherwise I'd suggest unsubscribing from this thread due to the volume of messages.

keithamus commented 8 years ago

@astorije thanks for updating chai-immutable :smile:, and thanks for your feedback, it's definitely appreciated.

although rather than subtractive keywords, I'd have preferred additive ones: browser, node, maybe all-engines, ...

I am inclined to agree, but at the same time I'd much rather us stick to popular convention than try to invent our own "standard", so it is what it is.

Just, if further keywords happen to be necessary and outside of all conventions, make sure to prefix them with chai-plugin-....

Yup - absolutely! I don't really foresee us having any other special keywords but if we do, we'll certainly make sure they remain under a chai namespace (unless they're super generic and almost a convention like browser).

and long live Chai!

I certainly hope so!

electricmonk commented 8 years ago

Hi there, I recently published a Chai plugin: https://github.com/electricmonk/chai-react-element I added the chai-plugin keyword. Is there anything else I need to do to get my plugin listed on chaijs.com? also, am I missing a doc page that describes the procedure to get a plugin listed?

Thanks

keithamus commented 8 years ago

Hey @electricmonk. The plugin won't be published just yet - our new plugins system is part of the redeveloped site, you can see the work in progress over here. When the site is complete, you should see it :smile:

Because the system is completely new we haven't yet documented it. This will come in due time when all of the site is done and we know exactly what we want from plugin developers.

electricmonk commented 8 years ago

@keithamus thanks. Is there any way of getting my plugin listed in the current site, until the new one is available?

keithamus commented 8 years ago

@electricmonk I want to get the new site will be live before the end of the year, 3 weeks at most. Putting effort into updating the old site right now is not really worth it - part of the reason why we're rewriting the thing is because the old site is too much of a manual process, so to answer your question: unlikely.

electricmonk commented 8 years ago

@keithamus fair enough :)

svallory commented 8 years ago

chai-param updated!

keithamus commented 8 years ago

Hey y'all who are still following this thread; we have a working plugin page on the github pages site: https://chaijs.github.io/chai-docs/plugins/. The keywords are clickable and filterable, and the tiles go to individual plugin pages - the README data is fetched from your README.mds, the keywords from your package.jsons.

NOTICE

There are still a few outstanding packages - packages which do not include the chai-plugin keyword. If you are an author of one of those plugins, please note that you will need to merge the PR I have sent you, and release a new version to continue to have a place on Chai's plugin listing. Gradually, we will be removing plugins which do not have these keywords, so please add them to your packages!

astorije commented 8 years ago

Great work @keithamus! :clap:

keithamus commented 8 years ago

I'm going to close this issue. The only outstanding plugins are the ones in chai core - which we will get around to as part of other refactoring and standardising work, and Chai Bookshelf (by @elliotf) which has an outstanding PR: elliotf/chai-bookshelf#1.

I'd say we can consider this pretty well done.