search

openalto / ietf-hackathon

3 stars 7 forks source link

Design Rucio queue status as a RFC7285 service #42

Open yryang opened 2 years ago

yryang commented 2 years ago

Make Rucio queue service as a cost map (end point cost service)

yryang commented 2 years ago
yryang commented 2 years ago

More reference: https://datatracker.ietf.org/doc/rfc8189/

yryang commented 2 years ago

https://datatracker.ietf.org/doc/draft-ietf-alto-performance-metrics/

yryang commented 2 years ago

Conduct initial design how this will change Rucio code.

yryang commented 2 years ago

Q1. Where is the code that generated the queue status file? Upstream of the modification Q2. How this info is used? // downstream of the modification

emiapwil commented 2 years ago

How C3PO uses the JSON data: https://github.com/rucio/rucio/blob/master/lib/rucio/daemons/c3po/collectors/network_metrics.py#L31

A talk on C3PO: https://indico.cern.ch/event/1028600/contributions/4318999/attachments/2227099/3772807/Dynamic%20Data%20Placement%20in%20Rucio.pdf

on behalf of @MattSlm

emiapwil commented 2 years ago

https://github.com/rucio/rucio/blob/6e27de4e1894a68a6fdeffd6e703b54590336a3d/lib/rucio/daemons/conveyor/submitter.py#L111 -> https://github.com/rucio/rucio/blob/6e27de4e1894a68a6fdeffd6e703b54590336a3d/lib/rucio/daemons/conveyor/common.py#L432 -> https://github.com/rucio/rucio/blob/6e27de4e1894a68a6fdeffd6e703b54590336a3d/lib/rucio/daemons/conveyor/common.py#L464 -> https://github.com/rucio/rucio/blob/6e27de4e1894a68a6fdeffd6e703b54590336a3d/lib/rucio/daemons/conveyor/common.py#L480

What is actually invoked:

https://github.com/rucio/rucio/blob/6e27de4e1894a68a6fdeffd6e703b54590336a3d/lib/rucio/transfertool/fts3.py#L760