search

stfc / rucio

Rucio - Scientific Data Management
http://rucio.cern.ch
Apache License 2.0
0 stars 0 forks source link

Odd behaviour for list_bad_replicas_status #70

Closed patrick-austin closed 4 years ago

patrick-austin commented 4 years ago

Motivation

test_bad_replica_methods_for_UI() relies on list_bad_replicas_status(), which has two different modes depending on the Boolean argument list_pfns. When False, we simply return all rows from the table bad_replicas that match the other function arguments.

When True, we instead use the rows from bad_replicas to generate a list of DIDs which are then passed to list_replicas(), which returns pfns for all replicas of the DID (not just the bad ones). This can optionally be filtered by specifying an rse_id, but we do not do this in tests.

test_bad_replica_methods_for_UI() runs before test_replica_recoverer() and usually passes. However if it is run afterwards (without clearing the database), it fails due to data generated by test_replica_recoverer(), which declares 3 files as suspicious 3 times each (+9 rows in bad_replicas). It also adds these 3 files (DIDs) at 2 RSEs, and so when listing using list_pfns = True we have +6 elements in the returned list.

As test_bad_replica_methods_for_UI() uses a combination of list_pfns = False and list_pfns = True and asserts based on the results, this means we get a discrepancy of 9 - 6 = 3 entries, causing the test to fail.

For a mutli-VO test suite, this failure can be prevented by restricting the list of replicas to the current VO (this is currently absent, which is an oversight). However as a general point it seems like either the tests or the functions themselves may need to change to prevent failures generally, and to ensure the two versions of the function are equivalent.

In terms of potential impact, the only places I can find the list_bad_replicas_status() function used is in tests and the WebUI.

Modification

First restrict the list of replicas based on VO. This will be covered as part of issue #25 anyway.

In terms of what to do to the tests/functions themselves, potentially try discussing with Martin (although is low priority compared to getting the main PR checked)?

patrick-austin commented 4 years ago

Multi-VO aspect of this (not filtering replicas by VO) has been merged as part of #25, any remaining work on the tests/replica functions is not a multi-VO issue so should be moved to mainline.