So I'm going off Bower

[jackfranklin]

The fact that it downloads entire repositories annoys me. When I run bower install jQuery, all I want is the jQuery source, minified. Not the entire repository.

I've been doing some work on my nodefetch project that will allow a user to specify a project local JSON file, called nodefetch.json that might look like:

{
    "destination": "js/vendors",
    "dependencies": [
        "jquery",
        "requirejs",
        "handlebars"
    ]
}

nodefetch is designed to be much more simple than other projects, so there's no versioning or anything. Current version on my github.

Or @phuu suggested https://github.com/component/component, and I was also pointed towards http://volojs.org/.

Thoughts?

[benhowdle89]

I did try Bower once, and was also confused why it would give you the full repo!? If there's a more lightweight one, then lets do that...

[jackfranklin]

Both component and volo seem a bit full on, they do a lot more than we need.

I might try my own nodefetch and see how it goes.

[tgvashworth]

I would say it's better to go with something external than nodefetch (sorry @jackfranklin, no offence meant), and component does seem to be under active development and has a growing library. If we're going to force users into using a web dependency manager, I think it should be a well maintained one with good documentation (again, no offence meant :wink:).

I would argue for picking one of bower, component and volo and sticking with it.

[jackfranklin]

@phuu don't worry, no offence taken!

I'm really not keen on Bower, as I noted in the first post, so I'll take a look at both Component and Volo and try them out. From a quick glance, I'm more keen on Component than Volo, which seems to attempt much more than what we need.

[tgvashworth]

Agreed on component over volo, but I've actually never used either so could be wrong!

[jackfranklin]

Having had a look at component it doesn't seem to be too well supported, unless I'm looking in the wrong place.

Going to take a look at Volo.

[jackfranklin]

So I had a play

Component

Just was difficult to set up - some simple modules it just wouldn't find (I couldn't get it to install Underscore, for example!).

Volo

Better, but when it doesn't know about a specific repository - eg, the repo doesn't contain the JSON settings needed for Volo to play nicely - (eg Handlebars), it just downloads the entire repository. However, for stuff it knew about (eg volo add jquery) it happily just downloaded the jQuery file.

This is why Bower made the call to act just as a wrapper around git clone. Either we work around the Bower issues by trying to configure R.js not to minify every single file within there, or we use something else. I still reckon nodefetch could be a potential option in terms of dead simple management but I do agree with @phuu's reservations of it.

TBH I'm kind of lost...help!

[tgvashworth]

Maybe we stick with bower then? That said, if all nodefetch does is grab a file from a location then that should work fine too (I'm working so haven't had a proper chance to look, sorry).

[jackfranklin]

I've been having a play.

If we restructure the app so everything that needs to be run through RequireJS apart from the Bower installs exists within its own folder, then we're fine sticking with Bower.

For example, if we make the folder structure look like so (or similar)

app/
    index.html
    assets/
        css/
        js/
            modules/
            main.js
lib/
    ...bower installs...
component.json
package.json
....

And update the RequireJS configuration to point to the app folder, we achieve the following:

• everything within the app folder is run through RequireJS
• the RequireJS optimizer doesn't run on the lib folder or the node_modules folder
• the resulting dist/ folder is slimmed down to actually be what we need.

Thoughts?

[tgvashworth]

Yeah I see where you're going with that – I'm not so hot on the directory structure but otherwise yeah, nice.

Just to throw a spanner in the works, what about Jam?

[46bit]

Interesting debate. Do you know whether Bower is using the --depth 1 option when cloning?

[jackfranklin]

Yeah I see where you're going with that – I'm not so hot on the directory structure but otherwise yeah, nice.

We can keep the same one but nest it one level deeper.

package.json
component.json
README.md
...
site/ { better name required! }
    app/
    index.html

Or something like that. If that lets us use Bower + R.js properly, I think it's a price worth paying.

Just to throw a spanner in the works, what about Jam?

Jam comes tightly integrated with RequireJS which I'm not so keen on. For usage, components also have to be "registered" to the Jam JS library, and it's not massive. For example, the RequireJS text plugin we use isn't in there, and I'd bet a lot of smaller libraries are not either.

Interesting debate. Do you know whether Bower is using the --depth 1 option when cloning?

I've no idea but I'd bet @sindresorhus might.

[tgvashworth]

For usage, components also have to be "registered" to the Jam JS library, and it's not massive

Yeah I saw that too, ah well.

I'm just wary of this happening. Perhaps stick with Bower and work around it?

[jackfranklin]

@phuu yeah me too.

Part of me thinks we're putting in too much effort...perhaps we shouldn't, and leave the choice up to the user? I'm keen to encourage people to do things "right" (for want of a better word) but if we can't decide on one, should we be suggesting one? (This is something Ben said on Twitter which got me thinking).

At least if we use Bower, it's very easy for someone else to ditch it and use something else.

The same could also be said for nodefetch, and Volo, and Component, but not Jam.

Out of all of them, Bower is my favourite, and I think the work around isn't too bad.

[tgvashworth]

Yep, agreed. I think your solution works. As a continuation of that idea, does a structure like the one below work? I guess you'd have to have require ignore certain directories and files (must be possible?), and then build only the required files (it does that anyway, right?)

app/
  ...
assets/
  ...
components/
  ...bower installs...
component.json
package.json
...

[tgvashworth]

Also, it would seem that Bower supports Component components (because it's just a git clone).

bower install component/super-cool-thing

[tgvashworth]

@jackfranklin Wicked. Sorry I'm not doing any of the playing myself, got a shitload of work to do!

[jackfranklin]

@phuu actually, I don't think that will.

The file exclusion is very poor in the optimiser, you can only do it through a regexp. If you exclude the components dir, the build file can't see it when it tries to optimise the require calls.

This is why the entire directory we want to build has to be in a different folder to the components. Here's how it works, I think:

• have an app folder that has everything we want to build in (assets, HTML, JS, modules)
• have a components folder that has Bower installed stuff
• in the Rjs build, set the appDir to the app folder. This means it will minify all the stuff within that folder, but can still pick up the Bower installs from the components folder. But because the components folder isn't within the appDir, they won't get minified and Rjs wont include them in the final dist/ output.

[46bit]

@jackfranklin I should have asked: do you dislike the git clones for being slow/large sizes or is it something else?

Judging by https://github.com/twitter/bower/blob/master/lib/core/package.js#L522, Bower is cloning the full history of the repos - hence much larger downloads and longer times. We might be able to get that changed.

EDIT: Reading through the issues on Bower for a sense of future direction, doubt this change would be that easy.

[jackfranklin]

@46bit the issue is that Bower by default clones the entire repo, and in the current set up, the RequireJS optimiser minifies and includes every single file in each repository Bower cloned in the output.

It's a combination of those two things that cause the problem. My problem isn't purely with Bower.

[tgvashworth]

@jackfranklin Hmm, that sounds roughly like what I was thinking.

A solution to the minify issue might be to add a step to the Grunt build process to minify the code and copy it into dist/.

[jackfranklin]

A solution to the minify issue might be to add a step to the Grunt build process to minify the code and copy it into dist/.

I am sure I'm missing something here, but what do you mean by "minify issue" ?

[tgvashworth]

But because the components folder isn't within the appDir, they won't get minified and Rjs wont include them in the final dist/ output.

I think I misread this. The optimiser will pick up and minify dependencies if they're in the components folder but ignore the rest?

[jackfranklin]

I think I misread this. The optimiser will pick up and minify dependencies if they're in the components folder but ignore the rest?

Yes.

By default, RequireJS will minify every single file that it can find within the appDir directory, whatever that might be.

This is why the components shouldn't be within appDir. Then, it will minify any files within components that are required through a require([]...), but not blindly minify & copy all the files over into dist.

Does that clear it up? Sorry, it's a little confusing and jarring.

I think the best way to do this might be for me to make these changes in a separate branch so you can check it out an see for yourself :)

[tgvashworth]

Yep, clears it up nicely, thanks. So we have a solution? (and a package manager independent one it seems?)

[jackfranklin]

Yes. It's going to require the folder structure change I mentioned above that in an absolutely ideal world I'd like to avoid, but it's not too bad.

I also think it would be nice to have wiki tutorials on how to swap Bower out for another package manager of your choice.

[jackfranklin]

So I thought we had a solution here but in actual fact I don't think we do.

The extra nesting of the folders causes problems with paths, and it's just generally a complete pain.

[jackfranklin]

So pulldown usage landed. Closing. Great convo all, thanks :+1:

[sindresorhus]

The fact that it downloads entire repositories annoys me. When I run bower install jQuery, all I want is the jQuery source, minified. Not the entire repository.

Me too, but the alternative is only support components tailored for Bower, which is unrealistic this early in. Just cloning repos means it can support a project without forcing it to do any changes.

A .bowerignore file is planned: https://github.com/twitter/bower/issues/88

RequireJS optimiser minifies and includes every single file in each repository Bower cloned in the output

Put it outside your 'appDir' and only reference what you use in your RJS config.

The extra nesting of the folders causes problems with paths, and it's just generally a complete pain.

Why? Might be something we could solve. I hate to throw it away just because of an path issue. Care to elaborate?

[jackfranklin]

@sindresorhus thanks for your comment :)

Me too, but the alternative is only support components tailored for Bower, which is unrealistic this early in. Just cloning repos means it can support a project without forcing it to do any changes.

I completely agree and fully understand why Bower does what it does - as you say there's no real way round it as this stage.

Put it outside your 'appDir' and only reference what you use in your RJS config.

I had tried this but couldn't get it all to work nicely. However, if we can use Bower I want to so I will take another look at this.

Why? Might be something we could solve. I hate to throw it away just because of an path issue. Care to elaborate?

So because we want RequireJS to fully minify all the assets and create a new "dist" folder with everything in (index.html, minified JS & CSS), we put all of that into a new folder, which we use as appDir. So then on the top level we have:

app/
  index.html
  js/
    main.js
  css/
components/
  ...bower installs to here...

It means that the paths to referencing something from Bower can be quite extreme, eg ../../components/jquery/jquery.min.js. I could also not quite get R.js to behave properly so it would only minify as I wanted it to. Then again I have a niggling feeling I might have not been doing something quite right.

Lets reopen this as I'm keen to revisit personally.

[OliverJAsh]

This is of no help, rather just my own curiosity.

RequireJS optimiser minifies and includes every single file in each repository Bower cloned in the output

RequireJS also places files like app.js, helpers.js, and introduction.js into the dist (build directory) which are not needed as they are concatenated into main.js. I've always been curious as to how you should go about "cleaning up" build files like this – you're circumventing the problem here by places lib outside of appDir I see, but how should it work for other files? Sorry to go off topic.

[jackfranklin]

@OliverJAsh happy to discuss, but could we move that into its own issue, just to not clutter this one up so much? :)

[OliverJAsh]

Put it outside your 'appDir' and only reference what you use in your RJS config.

I had tried this but couldn't get it all to work nicely. However, if we can use Bower I want to so I will take another look at this.

I just tried this and it works nicely. Components are stored in app/components and RequireJS only builds app/scripts, so it only concatenates the files referenced in modules! :)

[jackfranklin]

@OliverJAsh awesome! I didn't get round to tackling it yesterday. Pull Request it? ;)