search

DUNE / data-mgmt-ops

3 stars 3 forks source link

Ops: apparently no shared 3rd party copy between RAL-PP and QMUL #181

Closed StevenCTimm closed 2 years ago

StevenCTimm commented 2 years ago

FTS3 has thus far shown no transfers between RAL-PP and QMUL even though the full contents of RAL-PP have been set up to be transferred to QMUL. Instead Rucio seems to be trying to pull the lot off of tape at Fermilab. RAL-PP Protocols:

gsiftp domains: '{"lan": {"read": 0, "write": 0, "delete": 0}, "wan": {"read": 1, "write": 2, "delete": 2, "third_party_copy": 1}}' extended_attributes: None hostname: mover.pp.rl.ac.uk impl: rucio.rse.protocols.gfalv2.Default port: 2811 prefix: /pnfs/pp.rl.ac.uk/data/dune scheme: gsiftp https domains: '{"lan": {"read": 0, "write": 0, "delete": 0}, "wan": {"read": 2, "write": 1, "delete": 1, "third_party_copy": 1}}' extended_attributes: None hostname: mover.pp.rl.ac.uk impl: rucio.rse.protocols.gfalv2.Default port: 2880 prefix: /pnfs/pp.rl.ac.uk/data/dune scheme: https root domains: '{"lan": {"read": 0, "write": 0, "delete": 0}, "wan": {"read": 1, "write": 0, "delete": 0, "third_party_copy": 1}}' extended_attributes: None hostname: mover.pp.rl.ac.uk impl: rucio.rse.protocols.gfalv2.Default port: 0 prefix: /pnfs/pp.rl.ac.uk/data/dune scheme: root srm domains: '{"lan": {"read": 0, "write": 0, "delete": 0}, "wan": {"read": 3, "write": 3, "delete": 3, "third_party_copy": 1}}' extended_attributes: {'web_service_path': '/srm/managerv2?SFN='} hostname: heplnx204.pp.rl.ac.uk impl: rucio.rse.protocols.gfalv2.Default port: 8443 prefix: /pnfs/pp.rl.ac.uk/data/dune scheme: srm

QMUL

========== davs domains: '{"lan": {"read": 0, "write": 0, "delete": 0}, "wan": {"read": 2, "write": 2, "delete": 2, "third_party_copy": 1}}' extended_attributes: None hostname: se03.esc.qmul.ac.uk impl: rucio.rse.protocols.gfalv2.Default port: 8443 prefix: /dune/RSE scheme: davs srm domains: '{"lan": {"read": 0, "write": 0, "delete": 0}, "wan": {"read": 1, "write": 1, "delete": 1, "third_party_copy": 1}}' extended_attributes: {'web_service_path': '/srm/managerv2?SFN='} hostname: se03.esc.qmul.ac.uk impl: rucio.rse.protocols.gfalv2.Default port: 8444 prefix: /dune/RSE scheme: srm

StevenCTimm commented 2 years ago

Now seeing a few successful davs->davs 3rd party transfers.

srm->srm being attempted and failing gridftp->srm also being attempted and failing.

srm input on QMUL is known to work from other sites.

StevenCTimm commented 2 years ago

TRANSFER [13] TRANSFER globus_ftp_client: the server responded with an error 530 530-Login incorrect. : globus_gss_assist: Error invoking callout 530-globus_callout_module: The callout returned an error 530-an unknown error occurred 530 End.

That's the gsiftp to srm error

Same error in srm to srm. Davs to davs is fine.

wyuan-uoe commented 2 years ago

I deleted QMUL root protocol for few days when testing rucio upload stuff. Now I add it back.

StevenCTimm commented 2 years ago

Copies started up soon after wenlong added the root protocol back.. we are still seeing the gsiftp to gsiftp errors Some davs to davs copying going well too. transfers in aggregate about 95% done.. taking slower than normal because srm is preferred protocol and much more transfers being tried that way first and failing.

StevenCTimm commented 2 years ago

QMUL now fixed the gsiftp errors too. Closing this.