search

hep-gc / shoal

A squid cache publishing and advertising tool designed to work in fast changing environments
Apache License 2.0
4 stars 8 forks source link

update Shoal verification URLs #113

Closed rptaylor closed 7 years ago

rptaylor commented 9 years ago

The URLs for verification need updating: https://github.com/hep-gc/shoal/blob/master/shoal-server/shoal_server/config.py#L48 There are two problems: port 8000 is used (not sure where that came from, must be a very old configuration), and the repo names aren't fully qualified.

The best way to get the valid server URLs is to do this on a node with the latest version of CVMFS installed: grep CVMFS_SERVER_URL /etc/cvmfs/domain.d/cern.ch.conf and make all the permutations for each repository, by replacing @fqrn with <reponame>/.cvmfswhitelist.

So the list of validation URLs should be:

http://cvmfs-stratum-one.cern.ch/cvmfs/atlas-condb.cern.ch/.cvmfswhitelist
http://cvmfs-stratum-one.cern.ch/cvmfs/sft.cern.ch/.cvmfswhitelist
http://cvmfs-stratum-one.cern.ch/cvmfs/grid.cern.ch/.cvmfswhitelist
http://cernvmfs.gridpp.rl.ac.uk/cvmfs/atlas.cern.ch/.cvmfswhitelist
http://cernvmfs.gridpp.rl.ac.uk/cvmfs/atlas-condb.cern.ch/.cvmfswhitelist
http://cernvmfs.gridpp.rl.ac.uk/cvmfs/sft.cern.ch/.cvmfswhitelist
http://cernvmfs.gridpp.rl.ac.uk/cvmfs/grid.cern.ch/.cvmfswhitelist
http://cvmfs.racf.bnl.gov/cvmfs/atlas.cern.ch/.cvmfswhitelist
http://cvmfs.racf.bnl.gov/cvmfs/atlas-condb.cern.ch/.cvmfswhitelist
http://cvmfs.racf.bnl.gov/cvmfs/sft.cern.ch/.cvmfswhitelist
http://cvmfs.racf.bnl.gov/cvmfs/grid.cern.ch/.cvmfswhitelist
http://cvmfs.fnal.gov/cvmfs/atlas.cern.ch/.cvmfswhitelist
http://cvmfs.fnal.gov/cvmfs/atlas-condb.cern.ch/.cvmfswhitelist
http://cvmfs.fnal.gov/cvmfs/sft.cern.ch/.cvmfswhitelist
http://cvmfs.fnal.gov/cvmfs/grid.cern.ch/.cvmfswhitelist
http://cvmfs02.grid.sinica.edu.tw/cvmfs/atlas.cern.ch/.cvmfswhitelist
http://cvmfs02.grid.sinica.edu.tw/cvmfs/atlas-condb.cern.ch/.cvmfswhitelist
http://cvmfs02.grid.sinica.edu.tw/cvmfs/sft.cern.ch/.cvmfswhitelist
http://cvmfs02.grid.sinica.edu.tw/cvmfs/grid.cern.ch/.cvmfswhitelist
rptaylor commented 9 years ago

I used this script:


for i in http://cvmfs-stratum-one.cern.ch/cvmfs/ http://cernvmfs.gridpp.rl.ac.uk/cvmfs/ http://cvmfs.racf.bnl.gov/cvmfs/ http://cvmfs.fnal.gov/cvmfs/ http://cvmfs02.grid.sinica.edu.tw/cvmfs/
 do for j in atlas.cern.ch atlas-condb.cern.ch sft.cern.ch grid.cern.ch
   do echo "${i}${j}/.cvmfswhitelist"
 done
done```

You can also look in ```/usr/bin/cvmfs_config``` on a CVMFS client and look for a comment "Check for network" that shows a chunk of code for getting and testing all the proxy/server combinations.
colsond commented 9 years ago

All the urls are in the server config file: https://github.com/hep-gc/shoal/blob/master/shoal-server/shoal_server/config.py#L48

rptaylor commented 9 years ago

Alright, I updated the config file.

Hmm, it looks like the verifications are okay now: http://shoal.heprc.uvic.ca/ @rd37 did you update them yesterday?

colsond commented 7 years ago

@rptaylor the verification thread seems to be functioning fine, is it ok to close this issue at this time?

rptaylor commented 7 years ago

Yes, looks okay and can be closed for now. Future updates may be needed from time to time, keep an eye on changes in the cvmfs-config-default RPM. Notably, the TW stratum 1 was removed from the default CVMFS config (and from the Shoal verification URLS).