Failed to peel the archive from ruff

[jaudiger]

I recently tried to package the ruff tool. While peeling the directory, an error occurred:

Error: tried peeling directory with multiple entries

I then hacked brioche to print the content of the directory (as seen by brioche) since I don't have easily access to a debugger with my development environment:

Directory { entries: {"64f43441c8d3a0e9315e5de1d5e644b0d097e7cc.data": WithMeta { value: RecipeHash(Hash("0fda9b496eb36343c511b5caf5339a8dfcb6a1626586425fdd2daa67f796b2c3")), meta: Meta { source: None } }, "ruff-0.5.3": WithMeta { value: RecipeHash(Hash("545ec949c9a242c508a690a5f2a14b63361008417cba1ff67cdaa6692c815ccd")), meta: Meta { source: None } }} }

But looking manually inside the archive with the tar command, I wasn't able to see the file 64f43441c8d3a0e9315e5de1d5e644b0d097e7cc.data:

> tar -tf ruff-0.5.3.tar.gz  
ruff-0.5.3/
ruff-0.5.3/.cargo/
....
ruff-0.5.3/scripts/release/bump.sh
ruff-0.5.3/scripts/transform_readme.py
ruff-0.5.3/scripts/update_ambiguous_characters.py
ruff-0.5.3/scripts/update_schemastore.py

This is the project I used locally:

import * as std from "std";
import { cargoBuild } from "rust";

export const project = {
  name: "ruff",
  version: "0.5.3",
};

const source = std
  .download({
    url: `https://github.com/astral-sh/ruff/archive/refs/tags/${project.version}.tar.gz`,
    hash: std.sha256Hash(
      "7d3e1d6405a5c0e9bf13b947b80327ba7330f010060aaba514feecfd6d585251",
    ),
  })
  .unarchive("tar", "gzip")
  .peel();

export default () => {
  return cargoBuild({
    source,
    path: "crates/ruff",
    runnable: "bin/ruff",
  });
};

[kylewlacy]

Okay, so what I've figured out so far is that the file 64f43441c8d3a0e9315e5de1d5e644b0d097e7cc.data is meant to be the file ruff_linter__rules__flake8_quotes__tests__require_docstring_doubles_over_docstring_doubles_module_singleline.py.snap.

I figured this out by building tar from source with some extra print statements. I also verified it by comparing the list of files from tar xvf ruff-0.5.3.tar.gz from what you get when building this Brioche project:

import * as std from "std";

export default () => {
  return std
    .download({
      url: `https://github.com/astral-sh/ruff/archive/refs/tags/0.5.3.tar.gz`,
      hash: std.sha256Hash(
        "7d3e1d6405a5c0e9bf13b947b80327ba7330f010060aaba514feecfd6d585251",
      ),
    })
    .unarchive("tar", "gzip");
};

What I've pieced together is that the name 64f43441c8d3a0e9315e5de1d5e644b0d097e7cc.data is in the tarfile as the entry name, but then I think because the filename is so long, it's using a GNU extension to embed the longer filename? That makes me think this is an upstream issue, so I'll do a little digging to see if I can work out if this affects the tar, async-tar, or tokio-tar crates.

[kylewlacy]

As a temporary workaround, you can also use GNU Tar directly to extract the archive like this:

export const project = {
  name: "ruff",
  version: "0.5.3",
};

const sourceTar = std.download({
  url: `https://github.com/astral-sh/ruff/archive/refs/tags/${project.version}.tar.gz`,
  hash: std.sha256Hash(
    "7d3e1d6405a5c0e9bf13b947b80327ba7330f010060aaba514feecfd6d585251",
  ),
});
const source = std.process({
  command: "tar",
  args: ["-xf", sourceTar, "--strip-components=1", "-C", std.outputPath],
  outputScaffold: std.directory(),
  dependencies: [std.tools()],
});

[kylewlacy]

I made a small Rust projects to try out tokio-tar, then compared it to tar-rs, and that confirmed this bug is only in tokio-tar. Based on he issue tracker, it looks like tokio-tar is effectively unmaintained (and they don't allow opening issues on the repo), so I think the best course of action will be to move away from tokio-tar

[jaudiger]

Thanks for the investigation! Indeed, if this library is unmaintained, it's reasonable to use an alternative. I opened, in the meantime, the PR to add Ruff based on your suggestion.