dmwm / dbs2go

DBS server written in Go
MIT License
5 stars 4 forks source link

Fix for Issue #22 #23

Closed d-ylee closed 2 years ago

d-ylee commented 2 years ago

Initial fix for Issue #22.

@vkuznet While working on this, I have a few questions:

  1. Does datasets GET API require dataset_access_type parameter for each request? Only way to make dataset GET tests work is by including dataset_access_type for each of the tests: https://github.com/d-ylee/dbs2go/blob/4e37beb2f4e73c4283ee218607d3c226fb3a9008/test/int_datasets.go#L220
  2. Problems when using parent_dataset as a parameter: I get a SQL malformed error when trying to use parent_dataset as a parameter around the WHILE clause: https://github.com/d-ylee/dbs2go/blob/4e37beb2f4e73c4283ee218607d3c226fb3a9008/dbs/datasets.go#L99
vkuznet commented 2 years ago

@d-ylee to answer your questions:

  1. no, there is no mandatory parameters for datasets API, it should work with and without dataset_access_type parameter. In this case the datasets API will return all datasets regardless of their status
  2. we should fix datasets.sql template to account for parent_dataset lookup like this
    ...
    {{else}}
        D.DATASET
    {{end}}
    {{if .ParentDataset}}
        ,PDS.DATASET PARENT_DATASET
    {{end}}

    i.e. put parent dataset at the end of select clause and then remove PARENT_DATASET from conds statement like this:

    conds, args = AddParam("parent_dataset", "PDS.DATASET", a.Params, conds, args)

    Please make these adjustments and test the code.

d-ylee commented 2 years ago

@vkuznet In my integration tests, I have been using PRODUCTION as the dataset_acces_type. https://github.com/dmwm/dbs2go/blob/50f05b9495b098adae976e52c91777996b99c6ac/test/data/integration/integration_data.json#L8

Does this logic have dataset_access_type default to only finding rows that have it set to VALID and not show rows that have it set to other values, such as PRODUCTION, DELETED, DEPRECATED, or INVALID, as listed when accessing https://cmsweb.cern.ch/dbs/prod/global/DBSReader/datasetaccesstypes https://github.com/dmwm/dbs2go/blob/50f05b9495b098adae976e52c91777996b99c6ac/dbs/datasets.go#L78-L98

vkuznet commented 2 years ago

yes, by default this API should only list VALID datasets, and I suggest that you check this in a test with and w/o dataset_access_type varaible