[freebsd-ports] 支持 Git 部分克隆

[cherinyy]

目前 freebsd-ports/ports.git 服务器不支持部分克隆（Partial Clone），用户为了减小体积仅能通过设置深度（Depth）为 1 来完成，希望能够支持更加适用的部分克隆特性。

git clone --filter=blob:none https://mirrors.ustc.edu.cn/freebsd-ports/ports.git
Cloning into 'ports'...
warning: filtering not recognized by server, ignoring
...

[taoky]

已添加配置。

[cherinyy]

已添加配置。

似乎仍然不支持：

git clone --filter=blob:none https://mirrors.ustc.edu.cn/freebsd-ports/ports.git /usr/ports
Cloning into '/usr/ports'...
remote: Enumerating objects: 3585596, done.
remote: Total 3585596 (delta 0), reused 0 (delta 0), pack-reused 3585596
Receiving objects: 100% (3585596/3585596), 754.63 MiB | 11.09 MiB/s, done.
Resolving deltas: 100% (1876274/1876274), done.
error: RPC failed; HTTP 413 curl 22 The requested URL returned error: 413
fatal: unable to write request to remote: Broken pipe
fatal: could not fetch 2c80d58b7b230ec502a1535a46f26b7ea27f494a from promisor remote
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'

[taoky]

看了一下 nginx 错误日志：

2023/08/21 17:08:39 [warn] 32383#32383: *2316165779 a client request body is buffered to a temporary file <redacted>, client: <redacted>, server: mirrors.ustc.edu.cn, request: "POST /freebsd-ports/ports.git/git-upload-pack HTTP/2.0", host: "mirrors.ustc.edu.cn"
2023/08/21 17:08:41 [error] 32383#32383: *2316165779 client intended to send too large chunked body: 1049509 bytes, client: <redacted>, server: mirrors.ustc.edu.cn, request: "POST /freebsd-ports/ports.git/git-upload-pack HTTP/2.0", host: "mirrors.ustc.edu.cn"

从我的角度看，得先搞清楚 git 为什么向 server 发了 1M 多的 POST request，然后再考虑要不要拉高 nginx 默认的限制。先 reopen 了。

[taoky]

看来用 filter=blob:none 之后最后一步还是要下载 blob，而且似乎最后获取 blob 的单个 POST 请求的大小和 blob 的数量成正比。

另外是本地跑的不同方式 clone 的大小测试结果：

> git clone --filter=blob:none file:///home/taoky/tmp/freebsd-ports/ports.git/
Cloning into 'ports'...
remote: Enumerating objects: 3431157, done.
remote: Counting objects: 100% (39662/39662), done.
remote: Compressing objects: 100% (22196/22196), done.
Receiving objects: 100% (3431157/3431157), 693.28 MiB | 75.66 MiB/s, done.
remote: Total 3431157 (delta 20616), reused 34674 (delta 17185), pack-reused 3391495
Resolving deltas: 100% (1790166/1790166), done.
remote: Enumerating objects: 146835, done.
remote: Counting objects: 100% (91063/91063), done.
remote: Compressing objects: 100% (87801/87801), done.
remote: Total 146835 (delta 7353), reused 8750 (delta 3047), pack-reused 55772
Receiving objects: 100% (146835/146835), 73.04 MiB | 11.49 MiB/s, done.
Resolving deltas: 100% (9784/9784), done.
Updating files: 100% (151793/151793), done.
> du -sh ports
1.8G    ports
> git clone file:///home/taoky/tmp/freebsd-ports/ports.git/ ports-full
Cloning into 'ports-full'...
remote: Enumerating objects: 5668653, done.
remote: Counting objects: 100% (63800/63800), done.
remote: Compressing objects: 100% (42095/42095), done.
Receiving objects: 100% (5668653/5668653), 1.91 GiB | 51.37 MiB/s, done.
remote: Total 5668653 (delta 29445), reused 47056 (delta 21205), pack-reused 5604853
Resolving deltas: 100% (3258193/3258193), done.
Updating files: 100% (151793/151793), done.
> du -sh ports-full/
3.0G    ports-full/
> git clone --depth=1 file:///home/taoky/tmp/freebsd-ports/ports.git/ ports-shallow
Cloning into 'ports-shallow'...
remote: Enumerating objects: 187944, done.
remote: Counting objects: 100% (187944/187944), done.
remote: Compressing objects: 100% (175697/175697), done.
Receiving objects: 100% (187944/187944), 79.21 MiB | 9.86 MiB/s, done.
remote: Total 187944 (delta 11293), reused 101531 (delta 5898), pack-reused 0
Resolving deltas: 100% (11293/11293), done.
Updating files: 100% (151793/151793), done.
> du -sh ports-shallow/
996M    ports-shallow/

部分克隆得到的仓库比 --depth=1 还是大接近一倍。

[taoky]

client_max_body_size 加到了 16m，我测试 filter blob none 情况下同步 ports 没问题。如果之后要把这个值设置到太高的话我会更倾向于关掉部分克隆支持。