New package manager: earthfile

[rarkins]

New package manager questionnaire

Did you read our documentation on adding a package manager?

• [X] I've read the adding a package manager documentation.

Basics

What's the name of the package manager?

earthfile

What language(s) does this package manager support?

Earthly

How popular is this package manager?

Earthly is an OS tool with close to 11,000 Github stars

Does this language have other (competing?) package managers?

• [x] Yes dockerfile
• [ ] No.

What are the big selling points for this package manager?

It will help support many users to keep their dependencies up to date in Earthly build files.

Explain how this package manager is different from existing ones. The Earthfile syntax is similar to a Dockerfile but is different in a few ways. Creating a dedicated manager to support those differences will provide better support for Earthly users.

Detecting package files

What kind of package files, and names, does this package manager use?

Earthfile

Which fileMatch pattern(s) should Renovate use?

fileMatch: ['(^|/)Earthfile$', '(^|/|).+?\\.earth$']

Do many users need to extend the fileMatch pattern for custom file names?

• [ ] Yes, provide details.
• [x] No.

Is the fileMatch pattern going to get many "false hits" for files that have nothing to do with package management?

No

Parsing and Extraction

Can package files have "local" links to each other that need to be resolved?

Package file parsing method

The package files should be:

• [ ] Parsed together (in serial).
• [x] Parsed independently.

Which format/syntax does the package file use?

• [ ] JSON
• [ ] TOML
• [ ] YAML
• [x] Custom (explain below)

The Earthfile syntax is similar to that of a Dockerfile

How should we parse the package files?

• [ ] Off the shelf parser.
• [x] Using regex.
• [ ] Custom-parsed line by line.
• [ ] Other.

Does the package file have different "types" of dependencies?

• [ ] Yes, production and development dependencies.
• [x] No, all dependencies are treated the same.

List all the sources/syntaxes of dependencies that can be extracted

The package manager will rely on the docker datasource. It's possible it will rely on a custom datasource to maintain the version of the Earthfile

Describe which types of dependencies above are supported and which will be implemented in future

docker is already implemented. To support the Earthfile version we might need to implement a new datasource to retrieve the supported versions (This can currently be done with a regex manager)

Versioning

What versioning scheme does the package file(s) use?

Any versioning scheme that are supported by docker

Does this versioning scheme support range constraints, like ^1.0.0 or 1.x?

• [ ] Supports range constraints (for example: ^1.0.0 or 1.x), provide details.
• [x] No.

Lookup

Is a new datasource required?

• [ ] Yes, provide details.
• [x] No.

Will users want (or need to) set a custom host or custom registry for Renovate's lookup?

• [x] Yes, if they want to access docker images from private registries, but otherwise they will use public registries
• [ ] No.

Where can Renovate find the custom host/registry?

• [ ] No custom host or registry is needed.
• [ ] In the package file(s), provide details.
• [ ] In some other file inside the repository, provide details.
• [x] User needs to configure Renovate where to find the information

Same configuration as done today to access docker images from a private registry.

Are there any constraints in the package files that Renovate should use in the lookup procedure?

• [ ] Yes, there are constraints on the parent language (for example: supports only Python v3.x), provide details.
• [ ] Yes, there are constraints on the parent platform (for example: only supports Linux, Windows, etc.), provide details.
• [ ] Yes, some other kind of constraint, provide details.
• [x] No constraints.

Will users need the ability to configure language or other constraints using Renovate config?

• [ ] Yes, provide details.
• [x] No.

Artifacts

Does the package manager use a lock file or checksum file?

• [ ] Yes, uses lock file.
• [ ] Yes, uses checksum file.
• [ ] Yes, uses lock file and checksum file.
• [x] No lock file or checksum.

Is the locksum or checksum mandatory?

• [ ] Yes, locksum is mandatory.
• [ ] Yes, checksum is mandatory.
• [ ] Yes, lock file and checksum are mandatory.
• [x] No mandatory locksum or checksum.
• [ ] Package manager does not use locksums or checksums.

If lockfiles or checksums are used: what tool and exact commands should Renovate use to update one (or more) package versions in a dependency file?

Package manager cache

Does the package manager use a cache?

• [ ] Yes, provide details.
• [x] No.

If the package manager uses a cache, how can Renovate control the cache?

• [x] Package manager does not use a cache.
• [ ] Controlled via command line interface, provide details.
• [ ] Controlled via environment variables, provide details.

Should Renovate keep a cache?

• [ ] Yes, ignore/disable the cache.
• [x] No.

Generating a lockfile from scratch

Renovate can perform "lock file maintenance" by getting the package manager to generate a lockfile from scratch. Can the package manager generate a lockfile from scratch?

• [ ] Yes, explain which command Renovate should use to generate the lockfile.
• [ ] No, the package manager does not generate a lockfile from scratch.
• [x] No, the package manager does not use lockfiles.

Other

What else should we know about this package manager?

I think it should be straightforward to implement by using the same implementation as the dockerfile manager but make some changes based on the syntax differences, and I'll be happy to submit a PR for it.

Originally posted by @idodod in https://github.com/renovatebot/renovate/discussions/28359#discussioncomment-9091642

[idodod]

Thank you @rarkins. Does this mean I can submit a PR for this manager once it's ready?

[secustor]

Yes, this means we will accept PRs for this feature.

[HonkingGoose]

@idodod I'll assign you to the issue so we know you're going to work on this. If you don't want to work on this, let us know, and we'll remove your assignment. :smile:

[aslafy-z]

Is there any news on this new manager impl?

[HonkingGoose]

Hi @idodod, do you still want to work on this issue?