Redefine inconsistent use of dataset specific responses

[mbaudis]

The includeDatasetResponses parameter - picked up from the v1 schema - allows o specify if dataset specific responses should be provided:

• https://github.com/ga4gh-beacon/specification-v2/blob/a9fb4c9e293de2d9ea3fc65831d0e8d0c92bdfa6/beacon.yaml#L1869

However, only for variants dataset specific responses exists (BeaconDatasetAlleleResponse) but this isn't a consistent feature of the other endpoints (biosamples ...). Here, AFAIK it currently is not supported to deliver the results in per-dataset objects.

Overall, the "datasetAlleleResponse" etc. is a bit confusing. The straightforward options would be:

• besides the general "exists..." response, all data responses (variants, biosamples etc.) should be contained in dataset specific objects

OR

• data responses (biosamples, variants ...) should always be for a single dataset or aggregated response

Maybe it is just me missing some simplicity here...

[Tom-Shorter]

I agree with dataset specific objects idea.

Not only does this idea help to resolve the above but it can also help with AAI and pagination. This is an area I had planned to discuss in the next meeting (please remind me!) so I'll save writing any lengthy comments here until after the meeting.

[mbaudis]

The whole concept of "datasets" as a special grouping item is tricky. We have now a large interest in e.g. cohorts, which may span datasets. The datasets concept itself derives a lot from the "single VCF/collection with pre-computed frequencies" etc. which is extremely limiting.

So:

• there should not be a separate format for datasetAlleleResponses
• there should be a general wrapper format for responses for aggregates (datasets, cohorts, studies...)
• one could use the same format (i.e. a list of aggregates) as a default wrapper, with a fallback to a "default" value if no aggregation request (datasetIds ...) had been provided.

But this may just reflect some lack of understanding from my side how a /biosamples response for 2 selected datasets would look.

[jrambla]

This has been addressed in the Beacon v2 Model.