Open dandv opened 9 years ago
:smile: It's not you. It is Font-Awesome by Fort-Awesome. Check the URL: http://fo**r**tawesome.github.io/Fo**n**t-Awesome/
Fort awesome ahahah!
On Tue Dec 09 2014 at 9:17:19 AM Pierre-Eric Marchandet < notifications@github.com> wrote:
[image: :smile:] It's not you. It is Font-Awesome by Fort-Awesome. Check the URL: http://fo_r_tawesome.github.io/Fo_n_t-Awesome/
Reply to this email directly or view it on GitHub https://github.com/MeteorCommunity/discussions/issues/14#issuecomment-66319810 .
@dandv You mentioned in your lightning talk that you wanted to add automatic GitHub scraping. We at jsDelivr CDN use a home-grown libgrabber. We set it up by adding a config file in our repo. Then libgrabber compares each project's local version against their GitHub-releases, & downloads the few files listed in the config. Thought you might want to study it for ideas!
Thanks @tomByrer - definitely worth a look. The less library authors have to do, the better.
Hei @dandv,
I've just played with your publish.sh
script but this part has a little problem: it always thinks there was an error! ;-)
This is what I get on my log:
We got an error. Please post it at https://github.com/raix/Meteor-community-discussions/issues/14:
--------------------------------------------- 8< --------------------------------------------------------
Reading package...
Bundling source...
Creating package version...
Uploading source...
Publishing package version...
Bundling build...
Creating package build...
Uploading build...
Publishing package build...
Published useraccounts:unstyled, version 1.4.0
Done!
--------------------------------------------- >8 --------------------------------------------------------
but this is the normal successful log from meteor publish
@splendido: thanks for the nudge - I've updated the .sh scripts for FontAwesome. As you know from jsplumb/community-edition#99, We're trying to move away from them to Gruntfile updates (see the Hammer.js integration) because library authors tend to be unsympathetic to shell scripts.
Yes I know ;-) Tnx anyway for updating!
...Yesterday I started to have a look at how the Ionic team is using TravisCI to automatically create new releases: they have a number of bash script to be run directly on Travis servers... Within their guide they say
Travis will detect that this commit changed the version in package.json and push out all necessary for this new release (tags, release files, site config, ...)
Honestly I had no time to dig it to the end, but I feel this could be something worth understanding.
also, Ionic might be another candidate for an official porting. The only available package is now urigo:ionic but somehow goes through angular-meteor which might be skipped I guess...
Happy to announce that @FezVrasta has accepted my PR for Bootstrap-Material-Design (7,600+ GitHub stars).
@splendido, @tomByrer: bug me if I don't look soon enough into libgrabber and the Ionic team's use of Travis to auto-release :) Also CDNjs,
git ls-remote
to see if there are any updates, pull them if there are, and then check to see if there's a release change in thereSure, & I'll assume "soon" = after the holidays :)
OT, where can I open an issue for the Meteor website? (the theme has a little bug)
@dandv, We'll done! ;-) ...I'll also try to get to the end of it!
well done with bootstrap-material-design! nexts up: Semantic-UI and Ionic ...far more GitHub starts each one ;-)
@FezVrasta: at https://github.com/meteor/meteor/issues/new
Might also mention that the Sign In button doesn't render on IE11.
ok, I have a couple of ideas I'd like to test a bit...
Would anyone mind if I create/do the following:
packagingbot
user on meteor.com (in case it is still available...)packaging
organizationpackagingbot
to the packaging
organizationpackaging.meteor.com
test-package
repo on GitHub under MeteorPackagingMight be too much but hopefully not if you think the idea is worth some testing ;-)
the title could be Automatic Meteor publish through TravisCI
create a meteor app at packaging.meteor.com which requires GitHub login and gets an access token to dig into the list of your repos and then
packagingbot
to be used for automated publishing: see http://docs.travis-ci.com/api/#settings:-environment-variablespublish.sh
script on TravisCI (to be found into the repo under, e.g., meteor/publish.sh
) on every new published releasehttps://github.com/pwmckenna/node-travis-ci could be useful to speed up js communications with TravisCI. Anything better for JS?
The publish.sh
should:
curl https://install.meteor.com/ | sh
echo 'username\npassword' | meteor login
meteor publish
That's it!
Any feedback would be appreciated ;-9
@splendido: nice! I had already created the meteorpackaging
MD org, and I've added you to it. Also created the packaging
org and added you to it. I had created a meteorpublish
user on GitHub and will email you the password.
This looks very promising; good luck with the next steps!
Oh wow, ...the easy part has gone :(
Tnx ;-)
@splendido inspiration for the hard part: https://github.com/raix/Meteor-mrtbulkrelease/blob/master/meteor.js#L187-L241
Background: I've updated the mbr tool to publish binary packages on all platforms pr. auto - yesterday I failed to publish cfs:gridfs for archs, this caused alot of people (and one team) to get errors - so I didnt want to get in that situation again - why I updated the mbr tool to do the work for me.
Notes: It mbr logs in remotely - you would have to store or create .meteorsession from account details After a couple of hours testing I hit the quota... So the publish for arch farm should have special quotas You might want to coordinate with @ekatek - making sure its possible to get special quotas.
Hey @raix, thanks for the pointer!
...I have to admit I'm not sure I have completely understand your remote login and the quota problem. What I'm thinking of is to execute a bash script from a clean virtual machine on travisCI.
Something like, at least:
UNAME_PWD = "$METEOR_USER
$METEOR_PWD"
curl https://install.meteor.com/ | sh
git clone https://github.com/CollectionFS/Meteor-cfs-gridfs.git
cd Meteor-cfs-gridfs
echo "$UNAME_PWD" | meteor login
meteor publish
a fresh meteor install requires about 328Mb so there should be no problems with the disk space on the VM, and the username and pwd should be among the global (protected) env variables after the initial setup to be made directly with the TravisCI API through the packaging.meteor.com application.
Do you think once we'll have frequent publishes (in case this will ever happen...) we might end up making too much meteor login
from the same ip address (the Travis' one)? Is this what you mean with the quota?
@splendido I was thinking publish + publish-for-arch - I guess the packages you would support are 100% non-binary?
https://build.meteor.com/ got pr. user quotas - I'm not sure what they are - but I exceeded those getting the error: Build farm: Sorry, you are over your time limit. Wait a while and try again. [over-quota]
Ah ok, now I see!
...I'm not sure but I guess yes, at least at a first step, only non-binary packages. Perhaps if the game play nicely we'll support also binary ones, possibly getting in touch with MDG to get a more friendly quota ;-)
would be nice :) btw. some of the features in mbr would be nice to have in such a service - I'm currently implementing an autogenerator for changelog.md - just one of many things in there - you are welcome to pluck etc.
tnx @raix! I'll happily pluck in case something will come in handy ;-)
...as for your automatic changelog, I know @geritol tried something similar that was deployed here but it doesn't seem to work anymore. Here is the repository.
@splendido it looks like: https://github.com/CollectionFS/Meteor-cfs-file/blob/master/CHANGELOG.md And its generated when versions are bumped ready for release in mbr.
Wow, awesome work! ...apparently you don't need help at all! ...but this could be somehow expected ;-)
https://build.meteor.com/ got pr. user quotas - I'm not sure what they are - but I exceeded those getting the error: Build farm: Sorry, you are over your time limit. Wait a while and try again. [over-quota]
I'm pretty confident that if we exceed some quota, I can talk to a few folks at MDG and get an exemption.
@splendido, keep up the good work, and @raix, thanks for the help!
BTW Atmosphere now supports organization pages (thanks @tmeasday & co), so I'm populating those that I've reserved. Check out https://atmospherejs.com/fortawesome :)
@splendido I cant see the publish.meteor.com code yet? :)
You cannot see it because it's empty! ;-)
...for now I played a bit with TravisCI trying to publish the publish-test package. I got it published
but with a non clean flow see the logs.
Basically you can run a deploy hook after new releases and I'm running a before_deploy
script to get the package published: this works, but the deploy itself (since it is a fake one) fails, so you get back a deploy fail
email from TravisCI which is not a good thing :(
This pattern might be easy enough for those who have some real deploy step on Travis, but for the others I'm starting to think the right thing to do is leveraging GithHub WebHooks and get our publish.meteor.com
app automatically notified about new releases. You basically get a post request on the url you specify witha payload including all necessary information.
This app should keep a list of publish action to perform inside a collection. Every time the server route is hit a new object is inserted into this collection (after needed checks) and a server-side observer triggers some script to do new publishing at each new notification.
This is what the repo for publish.meteor.com
is intended for, but I wanted to check with automated flows on TravisCI before reinventing the wheel ;-)
Ok, apparently publish.meteor.com is already taken, even if nothing is being loaded. Was it you @dandv?
Otherwise autopublish.meteor.com seems still untaken? Should I reserve it right now?
...done! nothing awesome but autopublish.meteor.com ;-)
...and eventually we have a working .travis.yml to automatically publish meteor packages from TravisCI after successful tests!
Lets have a look at this log: if you got to the end and clink on after_success
badge ad line 213 you'll see the output of this script which does meteor login
and meteor publish
only for tag commits.
If you instead have a look at this log you'll see the following log for the publish_on_tag.sh
script:
$ ./meteor/publish_on_tag.sh
Checking commit...
No release commit, skipping Meteor publish...
Done!
So the flow is the following:
./meteor/publish_on_tag.sh
which in turn:
meteor login
meteor publish
...I admit this publish_on_tag.sh
script needs some love! I'll try to make it a bit more robust but any improvement is very welcome.
@raix I'm now trying to build the autopublish.meteor.com
app.
At the moment I'm getting notified on a server route by GitHub via a WebHook whenever a push/release is made on a repo.
The problem I'm facing is if I run a script calling meteor login
or meteor whoami
. What I get is
Error: Future resolved more than once
from which I guess it might be impossible to run meteor
from a running meteor server?
What would you suggest?
I'm starting to think your mrtbulkrelease
will come in handy!
...do you think it would be possible to exploit it from a meteor server app?
One problem I see is the login to meteor locally (which is actually already a remote machine) to get the remote build machine before calling mbr
.
I think we're going to use always the same user, which credentials can be passed through the --settings
option, but still we'll need to keep a valid login session on the machine were the app is deployed.
I'm not the right person to solve this tricky topic, so any hint will be appreciated.
P.S. As for the quota you where mentioning I've noticed that you can pass in the amount of minutes you want the build machine reserved, and I saw no minutes
argument inside your code.
I guess there's a default reservation time which you can hopefully increase just enough for your job to complete.
Lets see the output of meteor help admin get-machine
, even if I guess you already know about it! ;-)
Ok, apparently publish.meteor.com is already taken, even if nothing is being loaded. Was it you @dandv?
Nope. Probably something trying to deploy a lame demo about publish functions.
Otherwise autopublish.meteor.com seems still untaken? Should I reserve it right now?
A simpler explanation for spacejam & CI than the MeteorHacks article is at Practical Meteor.
Yeah, .travis.yml is a bit obscure with that curl. Here's what I came up with after all this direct Meteor integration stuff:
language: node_js
node_js:
- "0.10.33" # because 0.10.34 is broken by some SSL cert issue, and we don't care about Node versions that much anyway
before_script:
- npm install -g grunt-cli
# Install meteor
- curl https://install.meteor.com | /bin/sh
# Install spacejam, Meteor's CI helper
- npm install -g spacejam
script:
- grunt meteor-test
Re. the future resolved more than once... launch meteor through another intermediary .sh
script? Add a crontab entry to launch it one minute from now?
Just some random thoughts:
@mitar I've also thought about adding a GitHub sevice called like Meteor Autopublish
but It seems they're not going to accept new services proposals. Lets have a look at the very end of the Service Hooks docs:
Note: If you are building a new integration, you should build it as webhook. We suggest creating an OAuth application to automatically install and manage your users’ webhooks. We will no longer be accepting new services to the github-services repository.
The application I'm trying to shape for autopublish.meteor.com
should look more or less like the TravisCI interface: listing your repos and letting you choose which one to enable for autopublishing.
Once you enable a repo, you should get a webhook automatically setup to point to autopublish.meteor.com/publish
.
When the app get notified about a new release should trigger a test-publish sequence on the remote build machine.
So I think this is exactly in line with what you're proposing.
For the test-publish on TravisCI lets see my above post: I actually used secure env vars to get username/password to login to meteor.com and the publish is attempt only after successful testing.
I'll keep you posted on the progresses, even if they might be a little slow due to lack of spare time...
I think what GitHub is saying that you could use OAuth to ask user to permission to setup something they now call a "web hook", instead of creating old "service hook" and enabling that. So we create an OAuth application and then use Github API to setup a web hook which calls us at every event we are interested in. What I am saying that this should be in my opinion some other service and not overloading Travis CI.
Yeah, exactly! This is what I tried to explain above.
I've managed to got 'auto publish' with TravisCI, in case anyone would be interested in using it
But I think we're more focusing in developing a completely autonomous service exploiting GitHub OAuth permissions, a meteor app running at autopublish.meteor.com
where you can enable your repos for autopublishing, and then from there trigger a build-test-publish on a meteor build machine every time the app get notified.
We're saying the same things I guess! ...I think it's me not being able to well explain myself with my poor English, sorry!
Oh, no, don't worry. Your English is great. I think I just stayed in one mindset and didn't recognize that you are talking about two things. :-)
All this sounds amazing!
The only thing I am missing is some integration of CLA bot into all this workflow. So to get users to sign CLAs. Then to test (Travis CI). And then to publish (what you are doing here).
Not expert about this, but I guess we'll have room to integrate it so to get a stronger position wrt repositories maintainers.
...btw, wouldn't it be enough to get them sign up to the autopublish.meteor.com
to get their permissions to auto-publish for Meteor? We might also add some tems of use or whatever we think useful not to get into troubles...
Then when you get notified about a new tag/release you simply publish the new package. I don't think single commits or pull requests should fall within this workflow... Thoughts?
Oh, no, CLA bot is not linked to the idea of autopublish. I just listed it as an idea what is useful to have as a service on GitHub.
So created MDC as a org on meteor for Meteor Development Community if we maybe wanna create some collaborative packages in the future under a common namespace
:+1:
Ok guys, we're almost there! Lets have a look at http://autopublish.meteor.com/ :-)
Things to be done:
The thing is: I'm using a droplet of mine on digital ocean :(
It is not possible to reserve a build machine with meteor admin get-machine ...
since there's no meteor installed on the production server...
If we could make it to convince MDG this is a valuable project and get access to three always available build machined that would be it!
... I'm using the user meteorpublish
which was added to the packaging
organization to test the project with this dummy package.
Feel free to experiment with it!!! ...more discussion on this issue
Let me know what you think! Luca
@splendido I see you use the github login - but perhaps it should also use the Meteor login (to generalte .meteorsession on the get-machine? - the get-machine would git clone and do the heavy lifting)
I think this should be discussed more...
At the moment I'm using the meteorpublish
meteor user which is already logged on my remote droplet I'm using to git clone / meteor publish.
This requires the meteorpublish
user to be among the package maintainers (or part of the organization) in order to be able to publish: this might be enough for 3rd party libraries, but I appreciate it won't fit for private packages...
In case the thing will reveals valuable, we might think about letting each user choose whether to publish with their own meteor user (which would require both the github and meteor-developer login I might provide with the useraccounts package...) or through the meteorpublish
user. ..
We'd also need a Meteor javascript API to be able to add users/maintainers to org/packages?
So far for the autopublish to work the user meteorpublish
should be manually added to the maintainers even before subscribing on the website. ..
With a Meteor API this could be also done on user choice, if this makes any sense...
....to be honest, there's still a bit to do.
@raix your mbr work is awesome! I took out most of it as you suggested ;-)
I think mpm can replace npm, bower, etc but it can't replace CDN & most of the popular third party libraries (like bootstrap, fontawesome, moment, etc) available in CDN so I think use them that's better way in production app, but in the case of offline dev version we need them locally so try to download them from CDN (like metrorhacks:npm), there is a another advantage - that is we can get any previous version of that third party library from CDN
we try to do this in jsdelivr:cdnify meteor package
Some of the official packages could actually use NODE_ENV in it's packages to check if it's development or production and unless it's development load files from cdn.
Thinking about bootstrap maybe nate and twitter could merge their packages. Using nate's approach in production mode.
also differential:vulcanize using VULCANIZE as env variable to compress bower_components in production version
Also I'm not sure to be in the right thread for this, but yesterday I wanted to use jquery-textcomplete and didn't really have the time to open a pull request, explain the meteor package system, and wait it to be accepted and merged. So I created a meteor wrapper. I haven't read the whole thread, but it seems you have standard messages for this kind of PR. So if someone want to do it just take my package.js
and ping me in the PR message. Thanks!
If you're looking for a popular 3rd party library on Atmosphere, chances are you'll find several wrappers, in various states of maintenance. To avoid this problem, several Meteor community members have started to provide authoritative, high-quality, Meteor integrations of 3rd party libraries, complete with tests and grunt scripts. These are forks of the library rather than Git submodules, because forks allow modifying the source to support Meteor (if necessary) and one can still pull upstream changes without having to re-apply the integration changes.
These forks live in the MeteorPackaging GitHub organization. This org was created so that multiple developers can collaborate on maintaining an integration in a centralized location. Now that Meteor's Package API is stable, we can submit PRs to popular JS library maintainers to add official Meteor integration. Some maintainers will accept the PR - great, we have integration straight from the source, with same-day updates on Atmosphere. Some won't accept the PR, in which case we continue with the MeteorPackaging fork.
I started this effort in November 2014, spoke about it at DevShop SF, and here's where we are now.
Going forward
As of September 2015, see http://autopublish.meteor.com. It's a system authored by @splendido that automatically publishes new versions of a wrapper package automatically on Atmosphere (once you've registered your repo with it), as soon as the upstream package has a new release. See the How it works page. For an example repo, see https://github.com/MeteorPackaging/numeraljs-core-wrapper.
The instructions from here on may be somewhat outdated.
Contribute (minimal effort)
If you have created an MD organization for a library, please let us know. Namespace is precious. Currently we're looking for:
Who has these?
Popular libraries with official Meteor support
PR acceptance pending
Refused to accept the PR
How to do this
Overview
meteor publish
Below are the detailed steps.
Get the integration working before even contacting the maintainer
git clone <your_fork_url>
cd <repo_directory>
git remote add upstream <repo_url_of_the_library>
git fetch upstream
git checkout -b meteor-integration upstream/
that_branchmeteor
directory in the root of the project and copy there the Hammer.js Meteor package files, or the Font Awesome Meteor package files, depending on whether you integrate a library or CSS/image/font resources.Go through every file and make changes to reflect the actual library you're working on.
in particular, give a GOOD summary in
package.js
(for SEO on Atmosphere), instead of the library's tagline. Example:For Atmosphere, a repository of JavaScript libraries, this is suboptimal: it repeats the library name, and it says "library" and "JavaScript" (well duh). You only have 100 characters. Don't waste them. Better:
Export the library global(s)
$
, there's nothing to export.Otherwise, add a
meteor/export.js file
like this:Run the tests using the build process, e.g.
grunt meteor-test
. If you want to run tests manually, execute this:Pick a namespace and publish the package under it
Please be very careful with this, as renaming packages is impossible.
Log into your Meteor Developer account and create an organization that represents the package. Most often it will be the name of library, without any special characters (no
-
,.
). No need to include theJS
part, since we're always dealing with JavaScript libraries.If the organization name is not available, contact me, @dandv. I've registered a bunch of them, so potentially evil individuals don't :)
meteor publish
the package to Atmosphere for the first time. This prevents the maintainer from running into an error that the package doesn't exist when they publish (meteor would complain about a--create flag
).After you've ensured publishing works (namespace is available, the
name
inpackage.js
is valid etc.), push to your git fork:Now you can go to the GitHub page of the library and issue a pull request!
Contact the library maintainers
Here is the recommended PR message.
See d3 and Font Awesome above for addressing objections.
Contact the maintainers of other Meteor wrapper libraries
Ask them to deprecate the library by creating a PR overwriting the README.md of their package with the text
The commit message should be "Deprecating package in favor of official integration".
The name of the PR issue (after you click "Create pull request) should be "Kind request to deprecate package in favor of official integration".
The text of the PR should be similar to this:
meteor
doesn't warn the user when runningmeteor add
on a deprecated package, so we hope users will check on Atmosphere. However, each package needs several flags from different users, in order to show as flagged globally.TODO
moment
(pinging @estark37)