Rename ?format=car URL params to match IPIP-402

[lidel]

This issue is based on discussions I had with @aschmahmann and with @hannahhoward (slack) about around depth=0|1|all. cc https://github.com/ipfs/specs/issues/348

Problem

The depth=0|1|all parameter was provisionally introduced as part of Rhea project and we quickly realized we need to improve if we want to upstream it to https://specs.ipfs.tech/http-gateways/trustless-gateway/

The name is unfortunate, because after all Graph API interations we've ended up with "depth" that lost original meaning ("how deep to fetch a DAG (or path)"), and ended up meaning "logical depth from the perspective of end user thinking in terms of a single block, a file or a directory enumeration".

In that framing, the name makes very little sense:

• a block can be resolved only via depth=0
• things that support byte-range seeking, like unixfs files, can have max depth=1
• hypothetical depth=2 makes sense only for directories and DAGs built out of IPLD things like DAG-JSON/CBOR documents (but we don't use this for Rhea nor need for gateway atm).

I don’t think we should have an integer range where there are only two possible values with well-defined behavior.

I agree, we ended up with three states:

• 0 - blocks for path + only the root block for terminal CID
  • Gateway uses this for IPFSBackend.GetBlock and for IPFSBackend.ResolvePath
• 1 - blocks for path + all blocks for a file, or a minimal set of blocks to enumerate directory or HAMT
  • Gateway uses this for all IPFSBackend.Get requests, as we don't want to over-fetch directories, and we don't know which path ends with a dir
• all - blocks for path + all blocks for entire DAG
  • Gateway uses this only for TAR (IPFSBackend.GetAll)and CAR (IPFSBackend.GetCAR) responses, but this is also the implicit default for ?format=car, so we don't really need this.

I think we both want to clean this up. and we agree "depth" is simply an invalid term here. I also agree with you, strings are more meaningful than numbers for this abstraction layer.

What we want

• this is not "depth" → this is more a predefined "scope" with specific meaning
• hard to reason what integers mean, or what the parameter is related to → include "car" in name, avoid magical IPLD/ADL terms, and other PL-specific words
• we don't accept integers >1 → make it opaque string
• we may want to use "depth" for selecting actual DAG depth in the future → rename

Solution

Based on my notes and discussion with Hannah, we agreed car-scope=block|file|all is a better framing:

• depth=0 → car-scope=block
• depth=1 → car-scope=file (non-files, like directories, and IPLD Maps only return blocks necessary for their enumeration)
  • we use file here to make more sense to non-PL user. "file" is a synonym for "the blocks needed to interact with the ADL in the transformed view", and the pathing on /ipfs/ defines the implicit ADLs to use.
• depth=all → car-scope=all (implicit default when car-scope is unset, send everything)

This is way easier to reason about, no need to read docs, hard to make a mistake, and a better fit for a future IPIP that updates https://specs.ipfs.tech/http-gateways/trustless-gateway/ which we want to do this year.

Transition plan.

• [x] bifrost-gateway starts sending requests with both depth and car-scope
  • done: https://github.com/ipfs/bifrost-gateway/pull/81
• [x] Saturn L1 forwards car-scope to lassie https://github.com/filecoin-saturn/L1-node/issues/323#issuecomment-1499478883
• [x] Lassie implements car-scope → https://github.com/filecoin-saturn/L1-node/pull/324
• [ ] confirm Lassie implements car-scope=block → https://github.com/filecoin-saturn/L1-node/pull/324/files#r1160386359
• [ ] biforst-gateway stops sending depth (https://github.com/ipfs/bifrost-gateway/pull/144)
• [x] draft IPIP with car-scope → https://github.com/ipfs/specs/pull/402#issuecomment-1544819745
• [ ] live with it for some time

Updated plan

• [x] bifrost-gateway starts sending requests with depth, car-scope and dag-scope AND both bytes and entity-bytes as noted in IPIP-402 – https://github.com/ipfs/bifrost-gateway/pull/108
• [x] new parameters released and deployed to Rhea prod
• [ ] Lassie implements new parameters https://github.com/filecoin-project/lassie/issues/238
• [ ] Saturn L1 implements new parameters
• [ ] dotStorage implements new parameters
• [x] Live with it for some time
• [ ] remove old params from bifrost-gateway https://github.com/ipfs/bifrost-gateway/pull/144

[lidel]

Eyeballed metrics on staging, unsurprisingly the file scope is requested the most often:

[lidel]

We had another round of renames in https://github.com/ipfs/specs/pull/402#issuecomment-1544819745:

• name changes based on rough consensus. (Juan, Rod, and Adin all suggested removing ambiguity and name dependency on CAR and UnixFS formats, making things more future-proof):
  • car-scope=file → dag-scope=entity
  • bytes → entity-bytes

Repurposing this issue to align bifrost-gateway implementation with specs.