search

ssl-hep / ServiceX

ServiceX - a data delivery service pilot for IRIS-HEP DOMA
BSD 3-Clause "New" or "Revised" License
19 stars 21 forks source link

API to Query Number of Files #708

Open gordonwatts opened 6 months ago

gordonwatts commented 6 months ago

Story

As a user or library writer I want to know how many files are in a dataset so I can configure parallelism correctly

Assumptions

  1. Dataset exists!

Acceptance Criteria

  1. I can make an API query and get back the number of files available in a particular dataset, accepting all modifiers (nfiles, if good, etc.).

Discussion

This is a fairly specific API call. We've talked in the past about exporting all the DID Finder results. This could be a version of that. Make an API query to SX and get back a complete list of all replicas for a dataset that you can then use locally however you'd like. A flavor of this (with a flag?) could return just the integer number of files.

ponyisi commented 2 months ago

Is there a use case for this before the transformation is launched?