Added ardc.edu.au domain

[djw8605]

The goal for this repo is 2 things:

1. Replicate this stratum 1 to the OSG stratum 1 servers. I hope that is accomplished in this file: https://github.com/opensciencegrid/topology/blob/master/virtual-organizations/Neurodesk.yaml
2. Add the repo to the config-repo so it's more widely accessible.

This goal is to increase the accessibility of this repo for users outside of Australia.

[DrDaveD]

I see. At this point do we only care about getting it into the osg distribution for use in North America, or is there a need for it worldwide?

At minimum, the OSG stratum 1s should be added to the list of CVMFS_SERVER_URLs. For that you could use the example at https://github.com/cvmfs-contrib/config-repo/blob/osg/etc/cvmfs/domain.d/sdcc.bnl.gov.conf#L7. Include the setting of CVMFS_USE_CDN at the top of the file. Oh, it's a problem for CDN mode that the original server is an IP address only and not a DNS alias that can be kept constant. I suggest that we instead include at least the swinburne stratum 1 and ask the administrator of that stratum 1 to add this domain, once it is replicated to the OSG stratum 1. You can find the names for that stratum 1 for example in https://github.com/cvmfs-contrib/config-repo/blob/osg/etc/cvmfs/domain.d/opensciencegrid.org.conf.

[djw8605]

I'll ask if they have a DNS entry for it. Odd that they only gave us an IP.

[DrDaveD]

A reverse DNS lookup gets a name with the IP address encoded in it so it's not much better.

[stebo85]

Dear all, thank you for the feedback.

I now added a DNS name for our Stratum 0 server: http://stratum0.neurodesk.cloud.edu.au/

Thank you Steffen

[DrDaveD]

Ok then I added an openhtc.io (Cloudflare CDN) alias using that DNS name. The alias I used was s1ardc-cvmfs.openhtc.io. So @hzhu16 you can use http://s1ardc-cvmfs.openhtc.io/cvmfs/@fqrn@ with CDN and http://stratum0.neurodesk.cloud.edu.au/cvmfs/@fqrn@ without CDN for the CVMFS_SERVER_URL.

Can anyone say if this is to be replicated to only OSG stratum 1s, or do we want it to go further? Where will jobs be run that use it?

[stebo85]

Dear @DrDaveD,

That's fantastic! Thank you so much.

Our www.neurodesk.org users are geographically quite distributed, so if we can get this repository on stratum 1s close to them that would be wonderful:

[hzhu16]

Thank you all for providing information and instructions. The config file was updated and please let me know if it is right.

[djw8605]

Hi @stebo85, is this repo already replicated to some stratum 1 somewhere already?

[stebo85]

Yes, our neurodesk stratum 0 repository is replicated to 3 stratum 1 servers in Australia: cvmfs-brisbane.neurodesk.org cvmfs-perth.neurodesk.org cvmfs-sydney.neurodesk.org

We currently also run Stratum 1 servers in Europe and in the US, but we would like to stop hosting them ourselves and use the OSG servers instead to reduce our costs.

Thank you so much for everyone's help with this!!!

[DrDaveD]

Oh then we definitely shouldn't list your stratum 0 in CVMFS_SERVER_URL. Would you like us to list your 3 Australian stratum 1s, or do you hope to decommission some of those as well?

[stebo85]

Can you list our "primary" Stratum 1: cvmfs-brisbane.neurodesk.org - this one will be guaranteed to be available for the next 5 years.

Thank you!

[DrDaveD]

Perfect. I have now deleted the s1ardc-cvmfs.openhtc.io alias since we won't be using that and instead created s1brisbane-cvmfs.openhtc.io.

@hzhu16 Please make that change to your configuration and squash your commits down to one, then it will be ready to merge.

[hzhu16]

Hi Dave, please review the changes. Thank you!

[DrDaveD]

Please make a new PR to cherry-pick this to the osg branch