ignore via .gitignore

[calvinmetcalf]

it would be nice if harp could ignore files/folders/paths in the .gitignore, as things like node_modules can't easily have their name changed

[kennethormandy]

If this was done in a way where this was a nice add-on if you are using git, that would be fine. But there might be other things for us to consider. If we supported Browserify, what would be the implications of not serving or compiling the node_modules folder? Should it only skip it during compile? Then there could be different source files for harp server than harp compile, which we’ve been trying to avoid.

It seems worth thinking about, though. .gitignore would be the sensible convention to build upon.

Thanks for opening an issue! If you happen to have a project where this is an issue for you, it would be great for us to see to make sure we’re covering your use case.

[calvinmetcalf]

so for the record with browserify there would be no consequences to not serving or compiling the node_modules folder, browserify handles it and if it can't find the right thing will look in the node_modules folder in the directory above where it is

as for my use case, more generally I'm looking for an app that will let me preview gh-pages jekyll site before I push it up to github but using node instead of ruby, I'm not sure if harp is going to be the right one, but when I was testing it out it took forever due to it trying to copy over the node_modules folder every single time with no way to turn that off.

On Thu, Apr 3, 2014 at 2:22 PM, Kenneth Ormandy notifications@github.comwrote:

If this was done in a way where this was a nice add-on if you are using git, that would be fine. But there might be other things for us to consider. If we supported Browserify, what would be the implications of not serving or compiling the node_modules folder? Should it only skip it during compile? Then there could be different source files for harp serverthan harp compile, which we've been trying to avoid.

It seems worth thinking about, though. .gitignore would be the sensible convention to build upon.

Thanks for opening an issue! If you happen to have a project where this is an issue for you, it would be great for us to see to make sure we're covering your use case.

Reply to this email directly or view it on GitHubhttps://github.com/sintaxi/harp/issues/275#issuecomment-39486483 .

-Calvin W. Metcalf

[zeke]

Does browserify rely on node_modules at runtime?

[calvinmetcalf]

when bundles to create a static asset it uses the node_modules folder yes

On Thu, Apr 10, 2014 at 3:16 PM, Zeke Sikelianos notifications@github.comwrote:

Does browserify rely on node_modules at runtime?

Reply to this email directly or view it on GitHubhttps://github.com/sintaxi/harp/issues/275#issuecomment-40126498 .

-Calvin W. Metcalf

[rodolfo42]

Hi

I believe this is necessary for the problem I'm going through. Here is the SO question.

This is a great project btw, I use it all the time. My thanks to the contributors!

I've had an idea: why not parse a .harpignore file at the root? That way we could let the developer choose whether to use .gitignore (by creating a symlink to it with the name .harpignore) or to have a custom set of files or directories ignored but just for harp's eyes only.

What do you think?

[kennethormandy]

I think if we do make it configurable, it will be similar to the suggestion in the harp.json in #305, but I’m not sure if that’s the best approach either. We want to support the Browserify use case, which could probably done in a way that covers people who are installing Component or Bower modules too.

Potentially you could have an array of directories in the harp.json file that act like _’d directories even through they can’t be renamed. I am still keen on having it either through the .gitignore or implicit (ex. if you have a component.json then components/ would be ignored, if you have a package.json then node_modules would be ignored)

[jevets]

@kennethormandy I like the verboseness of an array in harp.json.

But one clear problem with relying on .gitignore: What if the developer doesn't use git? What if there's simply no need, or maybe he's working in a situation where git can't be used or he must use some other VCS? Or maybe he's required to commit his components or bower_components for some organization-level reason, as stupid as it may be.

I'd think decoupling is best. Harp should be able to handle this completely on its own. And I think it should be totally up to the developer. When we (as developers) find ourselves repeating ourselves, then we typically make a template or boilerplate for future use anyway.

And it's probably best not to assume that one's component.json file is actually tied to components/. A developer's component.json file may not even be used for Component, for example. Maybe component.json is a public data file for front-end use, some ajax UI widget or something, and maybe the developer has no choice and can't name it something other than component.json.

Harp probably shouldn't make any assumptions about my component.json (or any other file outside of Harp's lovely rules). A solution to this is simply to allow developers to specify explicitly which files should be ignored, or during harp server/harp compile: copied directly and not parsed (i.e. bower_components/bootstrap/less/*.less).

As an alternative idea to a verbose dotfile or having Harp assume anything about my files, what if harp (or maybe terraform would need to do this) provided developers with some kind of hook for us to extend/modify the list of parseable/copyable files/dirs, maybe via a user-provided callback. Maybe as simple as a CLI argument to load some local javascript file. Or maybe we could extend or override harp methods, without having to hack the core. (Just more ideas, food for thought.)

Of course, we can always just use a build system like grunt/gulp/etc. to do most of this, but that takes away from the lovely simplicity that Harp provides. Once you get into build systems, you may as well handle parsing and compiling everything there, which isn't as fun or quick or elegantly simple as just using Harp.

[kennethormandy]

I definitely agree with the issues you’ve raised here, and I see the benefits of decoupling. You’re right that a lot of people don’t use Git with Harp, I guess I was assuming that most people that use Bower or npm probably do but that’s not necessarily the case either.

The reason I’d rather make it implicit is for the exact reason you described: if you’re having to configure your build system, chances are you’d rather use Gulp, Grunt, or another SSG where you can configure everything. Harp’s benefit comes from not having to configure that stuff.

I’m trying to avoid someone having to look up what to drop in their harp.json just because they ran bower, component or npm install whatever.

I think this is a similar outcome to what you’ve described about overriding defaults. What if something like this was configurable in the harp.json:

{
  "ignore": ["css/fonts", "README.md"]
}

but by default (ie. if you never specified an "ignore" array in the harp.json) it would act as if it was already set to:

{
  "ignore": ["bower_components", "node_modules", ".git", ".gitignore"]
}

We already do this with .git and .gitignore so in a way it’s just exposing and expanding on what’s there already.

[turizoft]

Any progress on this? I like the idea of having the option to ignore some directories as part of the harp.json, it looks like it will be lead to less confusions and errors, than having it depending of the .gitignore.

[jevets]

@kennethormandy That sounds like a great solution to me, very usable and easy, still fits in with Harp's simplistic approach, and allows us just the little bit of "filtering" on the ignore list that we need to handle these use cases. And seems plenty flexible and simple enough to handle unknown future use cases, too.

My interpretation of the ignore array is that during harp compile, these ignored files would simply be copied directly over to the build directory (www/), exactly as-is. Is this what you're thinking? (Just to confirm.)

Same (pretty much) would go for harp server - the files would be served as-is and not parsed by Harp. Agrees with your thinking?

Another idea I had, may be worth considering, is an additional rule for naming. Maybe prefix two underscores for files/directories that shouldn't be parsed, but should be copied during harp compile and served during harp server? Or some similar naming rule? E.g. (not necessarily specific to Bower or anything):

harp.json
.gitignore
public/
    _data.json
    _partial.ejs
    _layout.ejs
    index.ejs
    __bower_components/

Just an idea. It does seem counterintuitive, though, when considering it alongside a file prefixed with a single underscore. But thought it was worth throwing into the discussion, at least.

@turizoft I don't believe any progress on actual implementation has been made yet. We're still discussing how it should work.

[kennethormandy]

Yeah, I’m imagining they’d be served as-is and never compiled (unless you were using them with Browserify which would have some extra stuff happen). This would also solve #170, #259, and #262.

While _bower_components works in the meantime by modifying your .bowerrc file, node_modules and components can’t be renamed intentionally. (There is no way to npm install --save whatever and have it go into _node_modules for example.) Otherwise following the _ convention would be ideal.

[jevets]

Yeah, I wasn't too thrilled with the additional naming convention idea, just threw it into the discussion for what it's worth. Sometimes bad ideas spark better ideas.

...and never compiled (unless...

• The ignored files would be copied over during harp compile, right? If not, how about a flag for the CLI? harp compile --copy-ignores or something?

A developer may very well need the entire bower_components/ copied during the build for the sake of deploying. (I'm sure there are other use-cases for this, even something like serving a .less file as plain text that's not intended for use in the Harp app. Or an .md or .markdown file that's intended for consumption on the front-end or from a 3rd party app, outside of Harp's build process.

[turizoft]

@jevets As no progress was being done, I took the simplest approach: Add a ignorable vendor path via options. The PR is up. See #312 for more details.

[iancrowther]

sorry to be slow here but do I just add README.md to the ignore: [...] ?

[kennethormandy]

Ah, I’m so sorry for the confusion. This doesn’t actually work yet, we’re discussing the syntax and there’s an in-progress pull request at #312. In theory, I think it might work like this:

{
  "ignore": ["README.md"]
}

[iancrowther]

ok great.. that seems to be the most obvious approach. +1 :-)

[iancrowther]

hey @kennethormandy @sintaxi

I just tried to adding README.md to an ignore block ^^ with no joy.

What currently is the best approach for this?

[og2t]

+1 :)

[alonextou]

It seems really trivial to add an ignore directive to harp.json. I'd like to write a tutorial using Harp with Grunt and Foundation Zurb, but without ignoring the node_modules and bower_modules, my workaround method feels like a total hack. Any progress on ignoring specified directories?

Thanks for a great server!

[og2t]

I am also thinking of creating a blog engine based off @kennethormandy article (http://kennethormandy.com/journal/start-a-blog-with-harp) that would rather use folders for articles, to eliminate the need of having _data.json file. Each folder would then contain its .json config file (easier to manage) with own assets, scripts etc. But for that I also need to be able to exclude/ignore some source files.

[kevinsimper]

@og2t You should look at Metalsmith, it sounds like what you look for. https://www.npmjs.com/package/metalsmith

[edrex]

There seem to be two use cases being discussed here:

1. pass-through without processing (what #312 implements via a flag to compile)
2. omit from processing and don't copy

I've run into both cases, although 2 is what I'm currently wishing for. If you are using a root-style app at the top level of a repository, this is necessary in order to keep .git and other detritus out of your output.

I would propose introducing two new keys in _harp.json to cover these two use cases:

    "copy": ["components", ...],
    "ignore": [".git", "Makefile", ...],

[chkal]

I agree with @edrex. These are two distinct use case. I like the suggested syntax. IMO this is much nicer compared to what #312 implements.

[rocketnova]

:+1: I much prefer the solution proposed in #438 of using something like a .harpignore or the one proposed here to ignore using .gitignore. This is much preferable to just ignoring default directories by name, such as "vendor" or "bower_components", as proposed in #312 beacuse it always the developer more flexibility.

[og2t]

.harpignore would be bliss. Since Harp is a zero-conf framework, this gives enough flexibility without adding any ignore JSON config files. Another benefit would be getting rid of the need to "underscore" _files and _folders, and still keep them ignored behind the scenes.