consider goals of schema

[gkiar]

What subset of the following do we want? Perhaps we should then design the schema specifically around them. Checked off are the values we are currently satisfying with the initial efforts this week.

• [x] 1. easily human readable (i.e. can open in vim/etc. and read text)
• [x] 2a. easily human parseable (i.e. can interpret easily, such as with a nested structure)
• [x] 2b. mimics filesystem on disc (i.e. like the above, but explicitly the same structure as on disc structure)
• [ ] 3. flexibly queryable (i.e. arbitrary attributes can be queried with approximately same performance/complexity)
• [ ] 4. enable graph querying (i.e. when derivatives are incorporated, NIDM-W-esque provenance tracking)
• [ ] 5. others?

Note: 2/2b and 3 are perhaps mutually exclusive, which is why I think we should try to define the goal-posts up front. My current guess is that we ultimately want 1 and 3, but eager to hear your thoughts, @glatard and @jbpoline.

[jbpoline]

Hey - to me the human readability/parseability is not mandatory but nice to have. 2b certainly does not seem to buy us much. 3 and 4 seems more important, or say we should think of how this will be extended. So, extendability and querying capacity / efficiency sounds the most important to me.

[chrisgorgo]

Could not agree more :)

On Sep 1, 2017 10:35 PM, "Jean-Baptiste Poline" notifications@github.com wrote:

Hey - to me the human readability/parseability is not mandatory but nice to have. 2b certainly does not seem to buy us much. 3 and 4 seems more important, or say we should think of how this will be extended. So, extendability and querying capacity / efficiency sounds the most important to me.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/INCF/apine/issues/7#issuecomment-326723562, or mute the thread https://github.com/notifications/unsubscribe-auth/AAOkp_ISE_zaqlPhFpXPo3RTZ3pTat00ks5seOkLgaJpZM4PKy32 .

[gkiar]

Great, I completely agree. Satisfying 3 is simple enough - I can have each run be represented on the same level with all the metadata flattened inside of it. How should we approach 4? We don't want to reinvent NIDM-W, so should we simply enable that derived data have a field which points to a NIDM-W object, and eventually we may parse it?

I'm planning to keep this project and the development of our cloud API a high priority in the coming months :)

[jbpoline]

Not sure yet - having a first example and playing with it will help though. I'll set up an e-meeting to discuss it further.

[glatard]

We should invite CARMIN people to this meeting: @axlbonnet @sapetnioc @michaelkain, this is apine (APIs for NEurostuff), a just-launched initiative to design a common API to manipulate (query, transfer, process) brain imaging datasets.

[sapetnioc]

I am very new to this project and I have very little information, excuse me if I misunderstood the whole thing.

I think schema of the metadata (data that are used to navigate through data and processes) is very important. The API should expose the data but also the schema. Therefore, it is important to clearly identify the features wanted for the schema, for the data and for the processing. For instance there is no doubt that querying the data is important but querying the processes and even the schema could be interesting when it grows and becomes complex.

Following the idea of schema management via the API, it would be great to be able to build, copy, modify and inspect schemas with the API. If one wants to have a custom data field that is documented or that can be efficiently queried, he must add it to the schema via the API (internally each API implementation would do all databasing stuff necessary to speed-up queries on that field). Of course, not all schemas would be writable by everybody, there is a need for at least one standard schema.