renovatebot / renovate

Home of the Renovate CLI: Cross-platform Dependency Automation by Mend.io
https://mend.io/renovate
GNU Affero General Public License v3.0
17.57k stars 2.31k forks source link

Refactor/reconsider all file locations #3466

Closed rarkins closed 5 years ago

rarkins commented 5 years ago

Thinking through the various directories:

  1. OS_TMPDIR: some of the third party tools we use are going to write to this no matter what. I think we should recommend for Docker that users volume map the tmp dir from the host disk to the Renovate image so that Renovate isn't continuously writing "layers" inside the container file system

  2. Working dir: This is where we should clone repos to. It should be predictable.

  3. Renovate cache: This is where we write persistent cached data between runs, e.g. when we cache datasources for 30 minutes. It should be considered non-private "shared" data between repos

  4. Package manager caches: Most like npm, composer, etc all let you control where they write the cache to and read it back from later. A self-hosted bot owner would usually benefit from having a common, persistent cache, while in other cases it's beneficial to wipe the cache between runs, so we should make it configurable.

rarkins commented 5 years ago

As mentioned by @ViceIce in #3455, the working directory needs to be different for each repo in order for the persistent git feature to be useful.

Let's take a simple example where the OS's tmp dir is /tmp. We could then define Renovate's tmpdir as /tmp/renovate.

Repository working dirs could have a platform abbreviation to reduce chance of collision, e.g.

/tmp/renovate/bb/project/repo and /tmp/renovate/gh/owner/repo

Renovate's own cache can be /tmp/renovate/cache/*

And each manager can be assigned a directory/cache like: /tmp/renovate/managers/npm/, /tmp/renovate/managers/composer/, etc.

If the bot admin prefers to nuke the cache between repos, then we just rm -rf /tmp/renovate/managers while leaving /tmp/renovate/cache in place.

rarkins commented 5 years ago

Currently we have logic that checks if RENOVATE_TMPDIR is set and uses it, otherwise the OS's TMPDIR. We then add ./renovate/ to the path to be safe.

rarkins commented 5 years ago

I can also see a need to separate out the cache location from the working directory location. So the default should be like:

/tmp/renovate/workdir <-- workingDir /tmp/renovate/cache <-- cacheDir /tmp/renovate/cache/internal /tmp/renovate/cache/managers

const renovateDir = path.join(process.env.RENOVATE_TMPDIR || os.tmpdir(), 'renovate');
config.workDir = config.workDir || path.join(renovateDir, 'workdir');
config.cacheDir = config.cacheDir || path.join(renovateDir, 'cache');
renovate-bot commented 5 years ago

:tada: This issue has been resolved in version 15.10.0 :tada:

The release is available on:

Your semantic-release bot :package::rocket: