Datasets API does not return the expected file number

[tdelhomme]

When I ask for a limit of 10 files, (it is the same behaviour if I ask 1000 files), the api seems to don't take this into account:

> lung_bam = a$api(path = "query", body = body, method = "POST", limit = 10)
> lung_bam$count
[1] 100

[nanxstats]

Hi @tdelhomme, - thanks for reporting this. I didn't really see the parameter limit was supported by the datasets API. Maybe that is subject to change in the future since the dataset API is still in the advance access program (which is why it is not officially supported yet by the API client package). Even though it's not supported, I guess you could easily select the first 10 files from the returned list in R (though not optimal).

[tdelhomme]

Hi @road2stat, thanks a lot for your answer! Actually in the definition of the api method there is the limit parameter... Unfortunately I do need to have the total number of items, the first ten was just to try an example where limit does not work. Do you know another way to get all the files of my query? Here is an example of what I would like to get:

> body = list(
+     entity = "files",
+     hasExperimentalStrategy = "WXS",
+     hasDataType = "Raw sequencing data",
+     hasDataSubtype = "Aligned reads",
+     hasDiseaseType = c("Lung Adenocarcinoma"),
+     hasSample = list("hasSampleType" = c("Blood Derived Normal"))#, "Solid Tissue Normal"))
+ )
> 
> a$api(path = "query/total", body = body, method = "POST")
$total
[1] 459

Thanks!

[tengfei]

hi @tdelhomme yes, actually the limit and offset are two parameters always there.

By design, there is complete = TRUE parameter to return everything, and if you want to know total number there is other way, for each query, in header there is field "x-total-matching-query" as.numeric(headers(response(req))[["x-total-matching-query"]])

But like Nan said, it's advanced access with updating, I just tried couple lines of the previous code, it doesn't work properly, let us try to fix the problem with some time.

Thanks for reporting.

[tdelhomme]

@tengfei thanks for your answer, looking forward for the fixing. awesome api btw!

[tdelhomme]

@tengfei

> t = a$api(path = "query", body = body, method = "POST", complete = TRUE)
Error in a$api(path = "query", body = body, method = "POST", complete = TRUE) : 
  object '.item' not found