search

opensciencegrid / stashcp

Apache License 2.0
0 stars 10 forks source link

Crash when specifying a cache with --cache #48

Closed matyasselmeci closed 2 years ago

matyasselmeci commented 2 years ago

The OSG test suite sets up a cache on localhost:8001 and tries to download a file through it with the command

stashcp -d --cache=http://localhost:8001/ /osgtest/PUBLIC/testfile3 /tmp/tmpiord6aip

on 6.8.0 this fails with errors and a stack trace

time="2022-08-09T10:53:43-05:00" level=debug msg="Len of source: 2"
time="2022-08-09T10:53:43-05:00" level=debug msg="Sources: [/osgtest/PUBLIC/testfile3]"
time="2022-08-09T10:53:43-05:00" level=debug msg="Destination: /tmp/tmpiord6aip"
time="2022-08-09T10:53:43-05:00" level=error msg="Failed to download namespaces: 404 NOT FOUND"
time="2022-08-09T10:53:43-05:00" level=debug msg="Failed to download namespaces: Failed to download namespaces: 404 NOT FOUND, continueing using built-in namespace configuration"
<dump of namespaces file>
time="2022-08-09T10:53:43-05:00" level=info msg="Trying CVMFS..."
time="2022-08-09T10:53:43-05:00" level=debug msg="Checking if the CVMFS file exists: /cvmfs/stash.osgstorage.org/osgtest/PUBLIC/testfile3"
time="2022-08-09T10:53:43-05:00" level=debug msg="CVMFS File does not exist"
time="2022-08-09T10:53:43-05:00" level=info msg="Trying XROOTD..."
time="2022-08-09T10:53:43-05:00" level=info msg="Trying HTTP..."
time="2022-08-09T10:53:43-05:00" level=debug msg="Nearest cache list: [http://localhost:8001/]"
time="2022-08-09T10:53:43-05:00" level=debug msg="Cache list name: []"
time="2022-08-09T10:53:43-05:00" level=debug msg="Matched caches: []"
time="2022-08-09T10:53:43-05:00" level=debug msg="Trying the caches: []"
panic: runtime error: index out of range [0] with length 0

goroutine 1 [running]:
github.com/opensciencegrid/stashcp/v6.download_http({0x7ffda24e3f32, 0x0}, {0x7ffda24e3f4c, 0x10}, 0x0, {{0x0, 0x0, 0x0}, {0x0, 0x0}, ...}, ...)
    /home/runner/work/stashcp/stashcp/handle_http.go:214 +0xa8c
github.com/opensciencegrid/stashcp/v6.DoStashCPSingle({0x7ffda24e3f32, 0x19}, {0x7ffda24e3f4c, 0x10}, {0xc00005b560, 0x3, 0x0}, 0x0)
    /home/runner/work/stashcp/stashcp/main.go:293 +0xd49
main.main()
    /home/runner/work/stashcp/stashcp/cmd/stashcp/main.go:222 +0xf11

perhaps because the cache is not listed in the namespaces JSON?

I think if a cache is explicitly specified then it should always be used instead of doing any sort of lookup. (Also don't try CVMFS if it's an HTTP URL.)