search

hep-gc / shoal

A squid cache publishing and advertising tool designed to work in fast changing environments
Apache License 2.0
4 stars 8 forks source link

Global access of squids is generally not good #169

Open DrDaveD opened 3 years ago

DrDaveD commented 3 years ago

I think that global access to squids is generally a bad idea. If global caching is needed, using the Cloudflare CDN through something like openhtc.io is much more effective and simpler to operate.

Could disclaimers be put into the shoal documentation and configuration files to this effect? (I assume you would not be willing to disable the designation completely at this point.)

The WLCG does operate two pairs of squids at FNAL and CERN that are globally accessible but they are not for normal operation, they are for catching failures of squids elsewhere, so they don't get put into any auto discovery service.

MarcusEbert commented 3 years ago

How would you access the cvmfs content via openhtc.io?

To have a global access for a squid is not necessarily a bad idea since what can be retrieved through the squid is limited in the squid configuration, usually limited to the major experiments cvmfs content and blocked for all other requests.

It is helpful to have a view global squids that can be used when there is nothing else close by, especially when running on different clouds. While it is preferred to have a squid in each cloud, squids on VMs running in such clouds need to be always up but are not due to different issues with the cloud infrastructure or specific hypervisors.

Since the squid system and cvmfs in general is used by more than just CERN experiments, global squids provide a general fallback solution in case nearby or site squids are down. The fallback squids at FNAL and CERN are not configured by default in cernvm or cvmfs as far as I am aware of, but are a fallback coded inside of experiments framework, aren't they? Are they available to non-CERN experiments?

DrDaveD commented 3 years ago

How would you access the cvmfs content via openhtc.io?

We define names for all the stratum 1s there and set CVMFS_SERVER_URL to those aliases. Then Cloudflare caches all the content in their huge network of caches worldwide. We use the Cloudflare free tier. @rptaylor has a separate Cloudflare domain for Compute Canada.

See for example the global configuration for cern.ch.

To have a global access for a squid is not necessarily a bad idea since what can be retrieved through the squid is limited in the squid configuration, usually limited to the major experiments cvmfs content and blocked for all other requests.

It is helpful to have a view global squids that can be used when there is nothing else close by, especially when running on different clouds. While it is preferred to have a squid in each cloud, squids on VMs running in such clouds need to be always up but are not due to different issues with the cloud infrastructure or specific hypervisors.

Cloudflare is much better than any squids we can provide for when nothing is close by.

Since the squid system and cvmfs in general is used by more than just CERN experiments, global squids provide a general fallback solution in case nearby or site squids are down.

Fallbacks for when local squids are down are a different situation. I agree that you don't want to use Cloudflare for that, because you want to be able to know when the local squids aren't working, and Cloudflare gives no detailed usage breakdown. In order for that to be helpful, however, those backup squids need to be monitored for failover, and the owners of the squids need to be notified when there's something going wrong. That's what we do on the WLCG Squid Monitor CVMFS failover monitor. We have a few people that watch that and contact the squid owners.

The fallback squids at FNAL and CERN are not configured by default in cernvm or cvmfs as far as I am aware of, but are a fallback coded inside of experiments framework, aren't they? Are they available to non-CERN experiments?

The are configured by default for known domains by the cvmfs-config-default rpm and in its configuration repository for other domains it knows about (here is it's source in github). Yes they can and are also used for CVMFS by non-CERN experiments, they're for any grid computing. They allow connections to any well-known cvmfs stratum 1, as defined by the MAJOR_CVMFS acl in the frontier-squid source code. It looks like we don't yet have Compute Canada stratum 1s listed there but I could add them if it's desired. I am updating frontier-squid right now to shoal-agent-1.0.0 so it is a good time to make another change. @rptaylor if you want to be able to use them as CVMFS fallback proxies let me know the names of the stratum 1s and their Cloudflare aliases.

rptaylor commented 3 years ago

@DrDaveD sure it would be useful to have CC CVMFS servers included in the MAJOR_CVMFS ACL. The naming scheme is cvmfs-s1*.computecanada.ca and cvmfs-s1*.computecanada.net

However other than that CC CVMFS is not really related to the use of Shoal or this ticket.