search

input-output-hk / foliage

🌿 Foliage is a tool to create custom Haskell package repositories, in a fully reproducible way.
MIT License
42 stars 9 forks source link

Git repositories using submodules do not work #88

Open andreabedini opened 1 year ago

andreabedini commented 1 year ago

input-output-hk/hydra is in this unfortunate situation.

We fetch a tarball for the repo using GitHub API. Like

❯ curl -L \
  -H "Accept: application/vnd.github+json" \
  -H "X-GitHub-Api-Version: 2022-11-28" \
  https://github.com/input-output-hk/hydra/tarball/9f1027e0fdff6765f5233f19c4639fdaa3558bfa -o tmp.tgz

but that tarball does not contain the git submodules. It does include .gitmodules

❯ cat .gitmodules
[submodule "testnet/cardano-configurations"]
        path = hydra-cluster/config/cardano-configurations
        url = https://github.com/input-output-hk/cardano-configurations.git
[submodule "hydra-node/json-schemas/cardanonical"]
        path = hydra-node/json-schemas/cardanonical
        url = https://github.com/CardanoSolutions/cardanonical

but AFAIU the information about what commit they are supposed to be checkout out at is stored in the repository's .git folder, and therefore not part of the tarball.

It seems possible to obtain that information with a single api query

❯ curl -L \
  -H "Accept: application/vnd.github+json" \
  -H "X-GitHub-Api-Version: 2022-11-28" \
  https://api.github.com/repos/input-output-hk/hydra/contents/hydra-cluster/config/cardano-configurations\?ref=9f1027e0fdff6765f5233f19c4639fdaa3558bfa
{
  "name": "cardano-configurations",
  "path": "hydra-cluster/config/cardano-configurations",
  "sha": "f414e7b59a0f100d340b7368bc474e023c26c9f5",
  "size": 0,
  "url": "https://api.github.com/repos/input-output-hk/hydra/contents/hydra-cluster/config/cardano-configurations?ref=9f1027e0fdff6765f5233f19c4639fdaa3558bfa",
  "html_url": "https://github.com/input-output-hk/cardano-configurations/tree/f414e7b59a0f100d340b7368bc474e023c26c9f5",
  "git_url": "https://api.github.com/repos/input-output-hk/cardano-configurations/git/trees/f414e7b59a0f100d340b7368bc474e023c26c9f5",
  "download_url": null,
  "type": "submodule",
  "submodule_git_url": "https://github.com/input-output-hk/cardano-configurations.git",
  "_links": {
    "self": "https://api.github.com/repos/input-output-hk/hydra/contents/hydra-cluster/config/cardano-configurations?ref=9f1027e0fdff6765f5233f19c4639fdaa3558bfa",
    "git": "https://api.github.com/repos/input-output-hk/cardano-configurations/git/trees/f414e7b59a0f100d340b7368bc474e023c26c9f5",
    "html": "https://github.com/input-output-hk/cardano-configurations/tree/f414e7b59a0f100d340b7368bc474e023c26c9f5"
  }
}

but #80 is also an alternative.