incorrect T1_US_FNAL_Disk LFN-to-PFN translation for root protocol

[stlammel]

Please open a ticket. But I don’t quite understand what you’re saying here. Were you able to modify the code to do this correctly or…

On Nov 29, 2022, at 6:29 PM, Stephan Lammel [lammel@fnal.gov](mailto:lammel@fnal.gov) wrote:

Ok, i followed Eric's suggestion and the modification is done explicitly (tfc_lfn2pfn is fine but cmstfc does this). I don't know the reason behind the URI modification. I suspect it's to replace any http/https with davs in the protocol part of the PFN returned from tfc_lfn2pfn, etc. lines 40 to 47 of cmstfc.py. The code also replaces double slashes with single slash which is the right thing to do for all but the root protocol. I also don't know what 'web_service_path' does, i guess wildly that it's for the new tape prestaging API? I don't know the function of the 'prefix' either. It adds an additional slash.

I think we should change things such that multi-host TFC rules are supported. I would suggest the change but without knowing the history and about 'prefix' and 'web_service_path' function, can't.

Eric, shall i open an enhancement request or how do we want to proceed? Thanks, cheers, Stephan

---

From: Eric W Vaandering Sent: Tuesday, November 29, 2022 14:40 To: Stephan Lammel Cc: Rahul Chauhan; cms-comp-ops-transfer-team@cern.ch; cms-US_FNAL-admin@cern.ch Subject: Re: T1_US_FNAL_Disk root tr anslation screwed up

So the problem is with the root specification.

That goes through a code path that attempts to mimic what the TFC does with the following inputs:

Code is here: https://raw.githubusercontent.com/dmwm/CMSRucio/master/docker/CMSRucioClient/scripts/cmstfc.py

root domains: '{"lan": {"read": 0, "write": 0, "delete": 0}, "wan": {"read": 3, "write": 3, "delete": 3, "third_party_copy_r ead": 3, "third_party_copy_write": 3}}' extended_attributes: {'tfc_proto': 'xrootd', 'tfc': [{'proto': 'xrootd', 'path': '/+store/temp/user/(.)', 'out': 'root ://cmseos.fnal.gov//eos/uscms/store/temp/user/$1'http://cmseos.fnal.gov//eos/uscms/store/temp/user/$1'}, {'proto': 'xrootd', 'path': '/+store/(.)', 'out': 'root://cmsdcadisk.f nal.gov:1094//dcache/uscmsdisk/store/$1'http://nal.gov:1094//dcache/uscmsdisk/store/$1'}]} hostname: cmsdcadisk.fnal.govhttp://cmsdcadisk.fnal.gov/ impl: rucio.rse.protocols.gfal.Default port: 1094 prefix: / scheme: root

On Nov 29, 2022, at 2:34 PM, Stephan Lammel [lammel@fnal.gov](mailto:lammel@fnal.gov)<mailto:[lammel@fnal.gov](mailto:lammel@fnal.gov)> wrote:

Hallo Rahul, thanks for taking a look. I believe we are doing things very similar. Below my python commands:

import rucio.client.rseclient rseClient = rucio.client.rseclient.RSEClient(account="lammel") rseName = "T1_US_FNAL_Disk" rseProtocols = rseClient.get_protocols(rseName) rseWriteURL = next(iter( rseClient.lfns2pfns(rseName, ["cms:/store/temp/user/cmssam/"], protocol_domain="wan", operation="write", scheme="root").values() )) rseWriteURL 'root://cmsdcadisk.fnal.gov:1094/root:/cmseos.fnal.gov/eos/uscms/store/temp/user/cmssam/' rseClient.lfns2pfns(rseName, ["cms:/store/temp/user/cmssam/"], protocol_domain="wan", operation="write", scheme="root").values() dict_values(['root://cmsdcadisk.fnal.gov:1094/root:/cmseos.fnal.gov/eos/uscms/store/temp/user/cmssam/'])

Thanks, cheers, Stephan

---

From: Rahul Chauhan [rahul.chauhan@cern.chmailto:[rahul.chauhan@cern.ch](mailto:rahul.chauhan@cern.ch)] Sent: Tuesday, November 29, 2022 12:14 To: Stephan Lammel; Eric W Vaandering; cms-comp-ops-transfer-team@cern.chmailto:[cms-comp-ops-transfer-team@cern.ch](mailto:cms-comp-ops-transfer-team@cern.ch) Cc: cms-US_FNAL-admin@cern.chmailto:[cms-US_FNAL-admin@cern.ch](mailto:cms-US_FNAL-admin@cern.ch) Subject: Re: T1_US_FNAL_Disk root tr anslation screwed up

Hi Stephan, I am unable to reproduce this behaviour. Can you share more information, on how are you requesting the translation?

  from rucio.client import Client
  c = Client()
lfn = "cms:/store/temp/user/cmssam/"
rse = "T1_US_FNAL_Disk"
c.lfns2pfns(rse = rse, lfns = [lfn])
{'cms:/store/temp/user/cmssam/':
'davs://cmsdcadisk.fnal.gov:2880/dcache/uscmsdisk/store/temp/user/cmssam/'}

Cheers

Rahul

On 29/11/2022 17:49, Stephan Lammel wrote:

Hallo Eric, transfer team, if i ask Rucio for a LFN to PFN translation of cms:/store/temp/user/cmssam/ for T1_US_FNAL_Disk it returns root://cmsdcadisk.fnal.gov:1094/root:/cmseos.fnal.gov/eos/uscms/store/temp/user/cmssam/ which seems to be wrong and rather odd with the "root:" in the path. I expected to get root://cmseos.fnal.gov//eos/uscms/store/temp/user/cmssam based on the storage.json of T1_US_FNAL. Could someone please take a look/comment? Thanks, cheers, Stephan

[dciangot]

@ericvaandering is this issue pending? should we put it into todo project table?

[ericvaandering]

I am going to look at this today and hopefully solve it.

[ericvaandering]

This should be fixed

[stlammel]

Hallo Eric, so, the path translation is correct for Fermilab xrootd but still wrong for Fermilab SRM/gsiftp where /store/temp/user tranlates to "/srm/managerv2?SFN=/gsiftp:/cmseos-gridftp.fnal.gov/eos/uscms/store/temp/user/" Could you please take another look? Thanks, cheers, Stephan

[ericvaandering]

This appears fixed. @stlammel can you check again with exactly what you do?

[stlammel]

Looks good to me. Thanks Eric! - Stephan