the creeping scourge of tooling config files in project root directories

[boneskull]

Splitting this out from #71 which was rather broad. There are (at least) two separate issues there; one is where user-specific config files should live, and another is where project-specific config files should live. This issue is about project-specific config files.

The files we're talking about are configuration files for development dependencies and are committed to VCS.

Many projects suffer from the problem of too many config files in the project root, as @iansu tweeted:

We put config files in the project root because we put config files in the project root. There is no reason other than a lack of an alternative convention.

We can do better than this. The idea is we aim for a "critical mass" of popular tooling authors moving their project-specific configuration to a subdirectory (e.g., .config/). If we can agree on a subdirectory, and change our tools to support the new subdirectory, we will significantly reduce this problem. It's our hope (though not a strict requirement) that the convention we agree upon will be flexible enough to reach beyond the JS ecosystem. For instance, your .travis.yml and netlify.toml could live in this directory as well, if this idea becomes popular enough.

We should get buy-in from maintainers of popular tools; if these tools adopt the new subdirectory, it's likely the ecosystem will slowly follow suit.

cc @nodejs/tooling @nodejs/package-maintenance

[t0suj4]

I think this could be solved via overlayfs and editor plugin.

All you need is a FS driver and time to develop the plug-in that would mount the config directory and hide the files again.

It's IMO much simpler than demanding cooperation across projects if all you want is to get configs off your sight.

[DylanVann]

If popular tools would standardize on something like cosmiconfig support for a config folder could be done there, in one place.

Personally I like to put configuration in package.json.

[mslinn]

A documented series of fallback directories is a standard that *nix has used for decades. X11 has probably taken this concept too far.

At a minimum, tooling could look for config files in the following order. In this example I use the token {root} to mean "the top-level directory of this project's directory tree":

config/
{root}

Get fancy with per-user config to augment per-project config.

config-user/
config/
{root}

Per-user config would not be checked into git due to the following entry in .gitignore:

config-user/

[fregante]

Please don't make configuration tools more complex. It can't get any easier than this:

• configuration files are at the root; everything below is affected by them

It has always worked like this, with very few exceptions using ~/.config/* instead of just ~/.*/; they're the exception.

There's already a convention for this issue: config at the root and app files in sub-directory/ies.

---

If you're annoyed by config files on GitHub, have a look at https://github.com/sindresorhus/hide-files-on-github

[coreyfarrell]

Would this new subdirectory also include package.json config?

Moving package.json to .config folder seem like an odd choice, given that some of its fields are more of a metadata in a sense.

First off, who is the intended target for this new .config folder? Is it only for tooling/development, or is it also a place to put files necessary for runtime?

I suppose given the title this would've meant for tooling/development, so here's what I propose. Keep the necessary fields like name dependencies and post/preinstall scripts on the project root folder, but for the rest like publishConfig or build scripts, move it into package-dev.json

Having a package-dev.json might be worth considering. In this scheme tools which will load config from package.json could look for package-dev.json as well, prefer to grab config from package-dev.json if found. A project could configure supported tools in that single file without bloating the published size of the package as happens when putting a large number of development settings in package.json.

A project moving fields used by npm into the package-dev.json would require caution as doing so would force projects to require contributors to use some future version of npm to develop (but not matter to end-users). CC @ruyadorno any thoughts from the npm side about this?

[ntotten]

This would be very weird to support in tools like ESLint, Prettier, etc. that have always assumed folder hierarchy inheritance. (I am the maintainer of the VS Code Prettier Plugin).

Additionally, this feels more like personal preference and something that could be done in editors rather than globally. For example, it would be fairly easy to write a VS Code extension that organizes configs for you. You could use the settings to exclude the config files from the main explorer:

"files.exclude": {
    ".prettierrc": true,
    ".eslintrc": true
}

And then write an extension that provides a custom tree view that organizes the config files in whatever way you want - could be grouped by type, etc.

[samheuck]

What problem is being solved? Why is this such an issue? Can anyone break this down beyond "It smells" or "It makes my project root look cluttered"?

As a general rule of thumb, I'm always skeptical of hiding things that are already there just to reduce visual noise. I think that if a code repository accumulates "too many" .config files, that's a thing you would want to be very visible. Especially for a programmer who may be unfamiliar with that code base. It gives you a quick way to scan for what tools may be used by other contributors. That said, sometimes rules of thumb need to be broken, but I think understanding why is extremely important first. I think Chesterton's Fence applies here:

https://en.wikipedia.org/wiki/Wikipedia:Chesterton%27s_fence https://josephwoodward.co.uk/2020/04/software-the-chestertons-fence-principle

[agustif]

This would be very weird to support in tools like ESLint, Prettier, etc. that have always assumed folder hierarchy inheritance. (I am the maintainer of the VS Code Prettier Plugin).

Additionally, this feels more like personal preference and something that could be done in editors rather than globally. For example, it would be fairly easy to write a VS Code extension that organizes configs for you. You could use the settings to exclude the config files from the main explorer:

"files.exclude": {
    ".prettierrc": true,
    ".eslintrc": true
}

And then write an extension that provides a custom tree view that organizes the config files in whatever way you want - could be grouped by type, etc.

I use this to exclude node_modules and .vscode/ settings folder from the view.

[mleonhard]

The issue fails to show that a problem exists. It essentially just says "X is bad." I believe that changes should be made only for reasons. Please give reasons.

[mleonhard]

There are strong advantages to having config files in the project's root directory:

1. They show that this is the project root directory. This is useful for programmers and tools.
2. They show that the project uses particular tools. This is useful for new project members and old project members who haven't touched that stuff for a while.
3. In programming and system design, putting related things close together makes maintenance easier. Config files usually apply to multiple sub-directory-trees. The project root directory is therefore the closest position to the places where the config files are used (the sub-directory-trees). Putting config files there will reduce maintenance effort. Maintenance is 80% of the effort put into software projects. Reducing maintenance effort improves productivity and morale for engineers working on the project.
4. They support overrides in an intuitive way. For example, /proj-root/.gitignore contains rules that apply to all sub-directory-trees and /proj-root/abc/.gitignore contains rules that are specific to the /proj-root/abc/ sub-directory-tree. This obvious-ness is lost if the files are in /proj-root/config/.gitignore and /proj-root/abc/config/.gitignore. This also adds extra directories. If you make the config/ directory optional, you have introduced multiple allowed locations for config files. Having multiple ways of doing things makes maintenance much much harder.

In absence of compelling reasons to make the suggested change, it would be good to close this issue.

[darcyclarke]

There's been some great ideas, insights & criticisms surfaced already... so, why not throw my 2 cents in for good measure.

First & foremost, I think its honorable for us to try to tackle this issue holistically (*virtual pat on back to our community).

Second, I think it would be helpful to redefine/clarify the questions we're looking to answer & problem we'd like to solve for from the original post (especially to root out any assumptions - pun intended 😉):

Questions:

1. Where does project-specific config live today?
2. Is the current location of project-specific config a problem? If so...
3. Would standardizing project-specific config's location help? If so...
4. Where should project-specific configs live tomorrow?

Thoughts:

1. Anecdotally, we agree that the root of a project has become the primary - not sole - dumping ground for config today (with a mixture of options, overrides & extensions supported by the array of environments/tools)
2. I think this is the most contentious part. From a maintainers perspective (specifically npm): I don't think it's much of a problem today for us or many of our users/use-cases; Hypocritically, from the perspective of a user of multiple tools in my own projects: I'd appreciate a more substantial level of standardization/organization (thus, let's explore #3)
3. I'm concerned we're going to get into a situation where this only gets partial adoption & we now have a net-new competing standard/implementation (https://xkcd.com/927/); This would probably mean users fallback to whatever is easiest & most widely supported (ie. what is happening today). Let's assume for a minute that we could get widespread adoption of a standard, then...
4. .config/ for project-specific configs makes sense to me based on XDG (as has been proposed here by @boneskull / @shannonmoeller) but it doesn't resolve any concerns/reservations I have for #3

I'm open to discussing this further, including outlining any next steps, IF we can get some more interest/input by more tooling maintainers. Otherwise, I think we might want to spend more time/energy on other issues.

[MrStonedOne]

What problem is being solved? Why is this such an issue? Can anyone break this down beyond "It smells" or "It makes my project root look cluttered"?

It doesn't scale.

There is a reason github started supporting the .github directory.

Its as plain and simple as that.

Here is the not-stupid suggestion. .project-dev-config/ this way it doesn't require a directory conflict with the projects actual config files, and applies universally across programming languages and frameworks.

eslint and vslint and friends are not going to have any actual trouble moving their current paradigm to using this. if you can check for a file in all parents, you can check for a file in a subfolder of all parents just as easily. and if you want to lint your .project-dev-config folder, you simply put that config inside .project-dev-config/.project-dev-config.

This is a system that will scale, every potential issue raised stems from a lack of vision, given how easily they can be solved.

This would be very weird to support in tools like ESLint, Prettier, etc. that have always assumed folder hierarchy inheritance. (I am the maintainer of the VS Code Prettier Plugin).

If you are intelligent enough to make a linter, you are intelligent enough to abstract away the folder a config is actually in, and the folder a config applies to. I really don't know why everybody is trying to bring up excuses.

[patrickcurl]

My preference would be something like:

.config/
-- 
- package.json

// package.json 

"default_config_path": ".config",
"config_overrides": [
    {"eslint": ".eslintrc.js"},
    {"babel": "submodule/babel.config.js"}
]

This allows some nice defaults.... if you leave default_config_path off, if the config doesn't exist in the root, it'll check .config, otherwise it'll ignore anything not in .config, unless overridden.

[iansu]

I don't think we're going to be able to get consensus on a new config directory. Getting all tools to allow you to customize the location of the config file seems more realistic. We'll direct our efforts at specific tools that don't allow this today.

[Z3TA]

I don't see the problem...

File-list too long to fit on screen ?

Annoying?

On unix inspired systems files and folders will be hidden if the first character of the name is a . dot Windows also has a way to mark file/folders as hidden.

Or have the tree folder become a "billboard" ? For example Github listing the files on the project front page.

[ssbarnea]

If you already got enough of clutter in project directory and you are the maintainer of any tool with config, check https://dot-config.github.io/ and see yourself as welcomed to join the movement. Start would be hard and I suspect that early adopters will be less mainstream tools, but once we pass a dozen I suspect it will likely explode as I doubt that the number of those loving 100 files in project root is big.

[ssbarnea]

@wmstack I think you are missing the point about .config. Is not about the lengh of the folder, is about building on top on an already established and documented standard (XDG config) which is followed by thousands. I would personally try to avoid a "NIH" approach, even if it saves few characters. While node is very popular, it should not consider itself as a trend setter.

Sadly the cool feature that vscode introduced to collapse set of files like a folder will make is less likely to agree on getting the root cleaned. Unfortunately they decided to call this feature file nesting instead of using the far more popular terms like folding, collapsing or grouping, making it very hard to find and configure.

[faigdavid]

The only possible way to reduce config clutter is to create a config-manager tool that parses well-organized config folders/files and compiles them to the correct location (project root) when needed.. Potentially, it could reduce the number of root config files from like 16 to 3 or 4.

But we also need to consider: Would it really be neater?

Splitting up your configurations between root .config could potentially just make it harder for people reading your project to figure out what's going on. Either the config-manager's config is extremely neat or it would have to auto-generate some "project configuration description" that totally describes everything going on in all the config files.

The problem is, the way things are now seems to work pretty well. Sure it looks bad, but it just works. That said, it might be one of those "once you feel it you never go back" situations where someone makes a config-manager and suddenly everyone in the industry NEEDS it.

[ladasfeed]

Hey guys.

I have been trying to find a solution to minimise count of configs for 2 days and here is my conclusion.

Currently there is no opportunity to specify config file for the most of popular IDE plugins like eslint or jest, and the most popular IDEs able to work with different tool's configs only following strict hierarchy ( i mean tsconfig, jsconfig, babel etc... )

But

You can easily move everything related to webpack configuration, or any other configs/sub-configs which are used inside the root configs ( i mean - you are able to specify the exact path there ). For example

// package.json
"jest": {
"setupFiles": [
      "./configs/jest-setup.js" // you are able to store `jest-setup.js` everywhere!
   ]
}

So you can easily relieve your root.

BUT

You do not need it anymore. The main purpose was to get rid of noise. So catch some plugins for ides that allows you to hide everything you wanna hide:

https://marketplace.visualstudio.com/items?itemName=JeremyFunk.hidefiles https://plugins.jetbrains.com/plugin/17288-foldable-projectview