Currently, the "zoom" mechanism triggers a full data fetch, because the cache key is based only on exact chr/start/end match.
It is difficult to write a generic subsetter because each API has different assumptions (like fetching data in a different order not sorted by position, or not using specific field names).
The LZ-specific data sources can feasibly add a new feature for more granular subsetting of data, insofar as the payload format and ordering behavior should be known or easily guessed.
Eg, a cache hit would be satisfied by either an exact match, or a smaller region than what was already fetched; the payload data could then be subsetted using annotateData before further parsing.
We will need to think about backwards-compatible behavior, some sites use the association adapter with minor tweaks to talk to different APIs. We don't want those sites to break as we add nicer feature for our own APIs.
Adapters that would benefit: anything where each position was a single point:
Assoc
Recomb
PheWAS
LD
Tabix source
GWAS catalog
Sources that would require further thought: anything where each datapoint represents an interval (half in, half out of the region)
Currently, the "zoom" mechanism triggers a full data fetch, because the cache key is based only on exact chr/start/end match.
It is difficult to write a generic subsetter because each API has different assumptions (like fetching data in a different order not sorted by position, or not using specific field names).
The LZ-specific data sources can feasibly add a new feature for more granular subsetting of data, insofar as the payload format and ordering behavior should be known or easily guessed.
Eg, a cache hit would be satisfied by either an exact match, or a smaller region than what was already fetched; the payload data could then be subsetted using
annotateDatabefore further parsing.We will need to think about backwards-compatible behavior, some sites use the association adapter with minor tweaks to talk to different APIs. We don't want those sites to break as we add nicer feature for our own APIs.
Adapters that would benefit: anything where each position was a single point:
Sources that would require further thought: anything where each datapoint represents an interval (half in, half out of the region)