Store named queries for datasets as well as files

[hschellman]

Can we make named queries for datasets as well as files?

[ivmfnal]

how would that be used ?

[hschellman]

Right now we’re using the gui to generate lists of data collections.

Here is how we describe the recent production:

https://metacat.fnal.gov:9443/dune_meta_prod/app/gui/query?default_namespace=schellma&query=datasets+matching+schellma%3A*+having+datasetpar.deftag%3Dprod_v4&view_meta_as=&action=run DUNE/ProtoDUNE MetaCat Productionhttps://metacat.fnal.gov:9443/dune_meta_prod/app/gui/query?default_namespace=schellma&query=datasets+matching+schellma%3A*+having+datasetpar.deftag%3Dprod_v4&view_meta_as=&action=run metacat.fnal.govhttps://metacat.fnal.gov:9443/dune_meta_prod/app/gui/query?default_namespace=schellma&query=datasets+matching+schellma%3A*+having+datasetpar.deftag%3Dprod_v4&view_meta_as=&action=run [favicon.png]https://metacat.fnal.gov:9443/dune_meta_prod/app/gui/query?default_namespace=schellma&query=datasets+matching+schellma%3A*+having+datasetpar.deftag%3Dprod_v4&view_meta_as=&action=run

On Sep 26, 2023, at 9:30 AM, Igor Mandrichenko @.***> wrote:

[This email originated from outside of OSU. Use caution with links and attachments.]

how would that be used ?

— Reply to this email directly, view it on GitHubhttps://github.com/ivmfnal/metacat/issues/39#issuecomment-1735893464, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIA37DLFGJZEE3G57I5GAX3X4L7JHANCNFSM6AAAAAA46OJQFY. You are receiving this because you authored the thread.Message ID: @.***>

[ivmfnal]

Ok I understand there is a need to have dataset queries. How would having them have names help ? And why these named queries have to be in MetaCat ? Why they can not be stored in something like a collaboration wiki ?

[hschellman]

Well, being able to refer to them clearly. I realize this is things inside of things inside of things..

On Sep 26, 2023, at 9:37 AM, Igor Mandrichenko @.***> wrote:

[This email originated from outside of OSU. Use caution with links and attachments.]

Ok I understand there is a need to have dataset queries. How would having them have names help ?

— Reply to this email directly, view it on GitHubhttps://github.com/ivmfnal/metacat/issues/39#issuecomment-1735904624, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIA37DOBTVZYKZFV5LP5M5DX4MAF5ANCNFSM6AAAAAA46OJQFY. You are receiving this because you authored the thread.Message ID: @.***>

[hschellman]

Mainly convenience in referring to them.

On Sep 26, 2023, at 9:37 AM, Igor Mandrichenko @.***> wrote:

[This email originated from outside of OSU. Use caution with links and attachments.]

Ok I understand there is a need to have dataset queries. How would having them have names help ?

— Reply to this email directly, view it on GitHubhttps://github.com/ivmfnal/metacat/issues/39#issuecomment-1735904624, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIA37DOBTVZYKZFV5LP5M5DX4MAF5ANCNFSM6AAAAAA46OJQFY. You are receiving this because you authored the thread.Message ID: @.***>

[ivmfnal]

Named file queries can be used inside other file queries. That is why it makes sense to have them named. There is no functionality like that for dataset queries, so I do not see much of a benefit of having dataset queries have names other than keeping them organized, but I think wiki or similar mechanism can do much better job here.

[hschellman]

Let me think about how one might used named file queries to make a synthetic named data set query.

Likely

metacat create [options] schellma:mynameddatasetquery “files from schellma:mydatasets*”

How do I use a named query in a file query

Also a bit of a concern - users may use

metacat create …… without realizing it is specific to named queries.

Suggest

metacat named_query create

syntax to protect “create"

On Sep 26, 2023, at 9:49 AM, Igor Mandrichenko @.***> wrote:

[This email originated from outside of OSU. Use caution with links and attachments.]

Named file queries can be used inside other file queries. That is why it makes sense to have them named. There is no functionality like that for dataset queries, so I do not see much of a benefit of having dataset queries have names other than keeping them organized, but I think wiki or similar mechanism can do much better job here. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[marcmengel]

So going forward, we will track this in https://github.com/fermitools/metacat/issues/2