Users need to discover data across the CFDE

hms-dbmi / hidive-dmp-requirements

0 stars 0 forks source link

Users need to discover data across the CFDE #28

Open keller-mark opened 8 months ago

keller-mark commented 8 months ago

Based on the comments from @manzt about needing more scientific use cases that are discussed in the RFA

User wants to know which CFDE data is available in the NIH/common fund ecosystem:

Data from which large-scale projects is available? (e.g., HuBMAP, Gtex, ENCODE, 4DN, KPMP, ...)
Within each CFDE project, which data is available - query based on metadata
For each available file, is it ready to go for visualization? or does the user need to request data processing so that it will be ready for visualization?

keller-mark commented 8 months ago

For developers, there might need to be a concept of developing an "adapter" to a CFDE ecosystem project. These adapters should have the ability to consume "ready" data (e.g., in an object store, in a good file format) as well as "raw" data (e.g., needs SQL request, needs conversion, etc). A contribution of this project could be what a standard "adapter" interface would need to look like.

manzt commented 8 months ago

A contribution of this project could be what a standard "adapter" interface would need to look like.

Could take inspiration from a project like: https://pangeo-forge.org/

keller-mark commented 8 months ago

Ah yes that seems like a really great analogy from another domain - it might even help to frame the project as "pangeo-forge for biology" (plus the visualization aspect)