500 Error When Accessing Crates

[Micha-L-Davis]

Hello-- I have a Panamax mirror set up and I'm able to use the rustup portion without problems. For Cargo, however, it's giving me trouble when accessing the /git/crates.io-index endpoint.

I've tried running this both with and without the net.git-fetch-with-cli config set to true, with no difference.

Panamax is running on port 80, with a fully synced mirror.

Here's the relevant log entry:

04:25:11 Step 10/10: Run x.py (PowerShell) 04:25:11 PowerShell Executable: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe 04:25:11 Working directory: D:\Home\teamcity\work\Rust 04:25:11 Command: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe 04:25:11 PowerShell arguments: -NoProfile, -NonInteractive, -ExecutionPolicy, ByPass, -File, D:\Home\teamcity\temp\buildTmp\powershell4071428807607952288.ps1 04:25:12 downloading http://[panamax-mirror]/dist/2022-09-22/rust-std-1.64.0-x86_64-pc-windows-msvc.tar.xz 04:25:12
04:25:12 #################### 28.3% 04:25:12 ############################################## 64.9% 04:25:12
04:25:12 ######################################################################## 100.0% 04:25:18 extracting D:\Home\teamcity\work\Rust\rust-repo\build\cache\2022-09-22\rust-std-1.64.0-x86_64-pc-windows-msvc.tar.xz 04:25:18 downloading http://[panamax-mirror]/dist/2022-09-22/rustc-1.64.0-x86_64-pc-windows-msvc.tar.xz 04:25:18
04:25:18 ####### 10.0% 04:25:18
04:25:18 ################## 25.8% 04:25:18 ############################# 41.6% 04:25:18 ######################################### 57.3% 04:25:18
04:25:18 ################################################### 72.1% 04:25:18 ############################################################## 87.1% 04:25:18 ######################################################################## 100.0% 04:25:31 extracting D:\Home\teamcity\work\Rust\rust-repo\build\cache\2022-09-22\rustc-1.64.0-x86_64-pc-windows-msvc.tar.xz 04:25:31 downloading http://[panamax-mirror]/dist/2022-09-22/cargo-1.64.0-x86_64-pc-windows-msvc.tar.xz 04:25:31
04:25:31 ######################################################################## 100.0% 04:25:32 extracting D:\Home\teamcity\work\Rust\rust-repo\build\cache\2022-09-22\cargo-1.64.0-x86_64-pc-windows-msvc.tar.xz 04:25:32 Building rustbuild 04:25:33 Updating panamax index 04:25:33 error: Unable to update registry crates-io 04:25:33
04:25:33 Caused by: 04:25:33 failed to update replaced source registry crates-io 04:25:33
04:25:33 Caused by: 04:25:33 failed to fetch http://[panamax-mirror]/git/crates.io-index 04:25:33 04:25:33 Caused by: 04:25:33 process didn't exit successfully:git fetch --force --update-head-ok http://[panamax-mirror]/git/crates.io-index +HEAD:refs/remotes/origin/HEAD` (exit code: 128) 04:25:33 --- stderr 04:25:33 remote: Unhandled rejection: Io(Os { code: 2, kind: NotFound, message: "No such file or directory" }) 04:25:33 fatal: unable to access 'http://[panamax-mirror]/git/crates.io-index/': The requested URL returned error: 500 04:25:33 failed to run: D:\Home\teamcity\work\Rust\rust-repo\build\x86_64-pc-windows-msvc\stage0\bin\cargo.exe build --manifest-path D:\Home\teamcity\work\Rust\rust-repo\src/bootstrap/Cargo.toml 04:25:33 Build completed unsuccessfully in 0:00:20 04:25:33 Process exited with code 0

[k3d3]

If you're running panamax on a Windows machine, this might be due to some bugs with running Panamax on Windows due to NTFS being case-insensitive.

[Micha-L-Davis]

It's running in Ubuntu, so that's not an issue.

[panzerballett]

similar issues (not sure the root of the problem is the same)...

i used to have rewrite rules enable in nginx. these seem no longer necessary? i disabled them and can get most of the crates. but not all of them.

there seem to be some inconsistencies in the data i downloaded - e.g. in the crates directory i have:

$ ls 3/syn/
0.10.0 ... 1.0.107

but (or: "and also"); note the additional s:

$ ls 3/s/syn/
0.10.0 ... 1.0.109

in the second directory there seem to be more recent crates.

but that is not consistent - e.g.:

$ ls 3/x
xcp  xot

but:

$ ls -d 3/x??
3/x11  3/x1b  3/xal ...  3/xvi  3/xxv  3/xyn

for the moment i do not know how to get files from the crates consistently...

any advice?

[panzerballett]

oh, that may be the solution.

• issue 83

i will try.

[panzerballett]

the rewrite rule from issue 83 does not fix the problem...

[k3d3]

Oh yeah, the rewrite rules for nginx are currently not mantained by me, as I've been using the serve command. They probably need to be changed. Do you still get the same issue when using serve, @panzerballett ?

As for the original issue, unfortunately I'm not running into the same issue on a fresh Ubuntu install. Even running the git fetch command, it seems to succeed:

git fetch --force --update-head-ok http://127.0.0.1:5585/git/crates.io-index +HEAD:refs/remotes/origin/HEAD
From http://127.0.0.1:5585/git/crates.io-index
   34beb0f9cf..bfb0397660             -> origin/HEAD

I'm not sure what the issue could be - does the server have git installed?

[k3d3]

@Micha-L-Davis So it seems that uninstalling git on the server and rerunning the command gives me:

~/.local/bin/git fetch --force --update-head-ok http://127.0.0.1:5585/git/crates.io-index +HEAD:refs/remotes/origin/HEAD
remote: Unhandled rejection: Io(Os { code: 2, kind: NotFound, message: "No such file or directory" })
fatal: unable to access 'http://127.0.0.1:5585/git/crates.io-index/': The requested URL returned error: 500

The reason git needs to be installed is because panamax itself doesn't handle the git protocol directly. Instead, it calls out to the git http-backend command which then handles the protocol instead. Without git installed, this will fail in the same way you're seeing.

I'll need to change that error message to make it more clear.

So let me know if installing git fixes your problem!

[panzerballett]

git

git is installed on the server. the nginx config includes a the following lines:

        location ~ /crates.io-index(/.*) {
                fastcgi_param GIT_PROJECT_ROOT    /mnt/rust/crates.io-index;
                include       fastcgi_params;
                fastcgi_pass  unix:/var/run/fcgiwrap.socket;
                fastcgi_param SCRIPT_FILENAME     /usr/lib/git-core/git-http-backend;
                fastcgi_param GIT_HTTP_EXPORT_ALL "";
                fastcgi_param PATH_INFO           $1;
        }

what directory did you mean i should run git fetch in?

panamax serve

using panamax serve results in the same issue. e.g.:

$ cargo build
    Updating `panamax` index
  Downloaded io-lifetimes v1.0.6 (registry `panamax`)
error: failed to download from `http://panamax.internal/crates/3/l/log/0.4.17/log-0.4.17.crate`

Caused by:
  failed to get successful HTTP response from `http://rust.cry/crates/3/l/log/0.4.17/log-0.4.17.crate`, got 404

again, the file is available in /mnt/rust/crates/3/log/0.4.17 but not in /mnt/rust/crates/3/l/log/0.4.17 (note the l in the middle).

[k3d3]

Sorry, that git question was directed at @Micha-L-Davis.

@panzerballett you're running into a different problem and should open up a new issue.

[Micha-L-Davis]

@k3d3 --

Thank you, that was indeed the issue. Mirror is working great now.