Support git deps with build tool

[lpil]

Gleam only, no need for rebar3 currently

https://github.com/elixir-lang/elixir/blob/79cd891eb86ecb7654a7acdb63769cfdd950a5c0/lib/mix/lib/mix/scm/git.ex

• [x] Update gleam.toml to include git deps. Will require a fancy serde deserializer.
• [ ] Download top level git deps
• [ ] Parse gleam.toml for them
• [ ] Add info to manifest
• [ ] Integrate into the version resolver
• [ ] Assert there are no git deps when publishing

Anything else? Can't think of anything.

Config

[deps]
short_hex_package = "~> 1.0"
long_hex_package = { version = "~> 1.0" }
git_package = { git = "git@github.com:gleam-lang/stdlib.git", ref = "284b9f0" }

# We could have this shortcut too
github_package = { github = "gleam-lang/stdlib", ref = "284b9f0" }

[HarryET]

I'd like to have a look at this, but would do #1409 first as that is the basis for this issue.

[PgBiel]

I have been experimenting with this feature locally, and would like to discuss some of my findings so we can better define what to do next.

First, I noticed that there are two moments within gleam-cli/src/dependencies.rs in which we may need to clone or update a Git repository.

1. When creating the manifest, we need access to the updated Git repository to be able to check for its transitive dependencies and add them to the manifest, as well as check if there are no version conflicts.
2. After the manifest is ready, we still need to ensure all Git dependencies were updated, since it's possible the manifest didn't have to be updated but no dependencies were fetched yet (e.g. we ran gleam clean). Therefore, in the "downloading packages" step, we also need to consider Git dependencies.

Therefore, in my initial approach, I simply cloned the Git dependency to build/packages/<name> on those two occasions whenever I detected the need for it, and it sort of works.

However, I still haven't decided on the best solution for the "detecting the need for a clone" part. This is because the happy path I was considering is when build/packages/<name> is empty, or at least already has a repo clone with the same remote URL (so deleting the directory isn't necessary). Problems arise when that isn't the case, which can happen on the first manifest update after switching from a Hex dependency to a Git dependency, for example (at that directory, the Hex package will be there, not the cloned Git package, and build/packages/packages.toml is only updated after the manifest is created, as well as the extra dependency removal, so the Hex package will still be there at this stage). Basically, we need to decide how to properly handle existing build/packages/<name> while building the manifest, since it's possible we might need to delete it if we want to clone there.

I've thought of two approaches so far:

1. While building the manifest, clone the repository to a temporary location, e.g. build/git. That way we avoid conflicts with Hex dependencies, though we'd still have to delete everything inside this temporary location after we're done building the manifest, so we'd need to put some proper locking system in place (which probably isn't too hard, but worth mentioning). Additionally, this would result in two repository clones unnecessarily, as we'd have to clone to build/packages again later.
2. While building the manifest, we lock the build and packages directories and, for each Git package being provided, if it is already in build/packages/<name> but as either a Hex package or a Git package from a different repo, we erase the build cache for it for each target as well as delete build/packages/<name> (as is also done in remove_extra_packages) so we can clone there (note: when it's from the same repo but a different commit, as an optimization, we only have to erase the build cache, but we don't have to clone again, as we can just checkout to the new commit, fetching only if needed).

Approach 2 seems the most sane to me, but it does kind of entangle LocalPackages and the manifest construction together, where before LocalPackages used to expect the manifest to already be built. It still seems plausible, however.

Some additional questions:

1. Alongside optionally pinning a commit, should we allow people to pin a branch or tag on gleam.toml? (Personally, I'd say yes)
2. If so, should we just have a single option (such as ref or rev) which allows specifying any "commit-like" thing (you could specify a commit hash, a branch name or a tag name in a single option)? It seems to be enough for Git purposes, but I wanted to confirm if this is something we want.

Also, regarding #2899, it's worth noting that I don't think we'd have much trouble making the two feature requests work together, at least based on my POC. So the main focus on the discussion should probably be the points above, in principle.

[versecafe]

Maybe also support

git_package = { git = "git@github.com:gleam-lang/stdlib.git", branch = "otp-28-support" }