search

FAIRiCUBE / Knowledge-Base

This repository is intended for sharing our current understanding and getting feedback on the proposed architecture, structure and functionalities of the KB as well as related look & feel. Comments and issues related to the Knowledge Base shall be provided via issue tracker of this repository
0 stars 0 forks source link

Query Tool upgrade to query both a/p resources and datasets metadata #4

Open cozzolinoac11 opened 1 month ago

cozzolinoac11 commented 1 month ago

As an outcome of the common topic seminar held on 21.05.2024 (see presentation), below the proposed requirements for Query Tool upgrade to query both a/p resources and datasets metadata.

Concluding remark of the seminar was to document objective/verifiable reasons to discard option b), e.g. inefficiency of querying the catalog via STAC API, especially if querying two or more different collections and using many parameters. Should option a) be approved, the related requirements and implementation plan are listed in the table below.

Req. Id Requirement Plan Deadline
1 Create an ingestion procedure to ingest dataset metadata in the Knowledge Base database. 1. Read json dataset metadata from GitHub

2. Mapping from json dataset metadata to database table

3. Definition of mechanism to run the procedure		 31.10.2024
2 Upgrade current Query Tool to allow complex queries. 1. Definition of additional queries

2. Definition of layout changes

3. Implementation		 31.12.2024

Feedback welcome!

KathiSchleidt commented 1 month ago

I see the decision between option a and option b as the first step, when can we expect this?

Assuming for steps 2.1 & 2.2 (updating the design for the Query Tool), you'll be getting feedback from partners?

gmartirano commented 1 month ago

I see the decision between option a and option b as the first step, when can we expect this?

I would say after leaving to the coordinators and to the interested partners a reasonable time to provide their feedback. 1 or 2 weeks from now (also depending on possible feedback)?

Assuming for steps 2.1 & 2.2 (updating the design for the Query Tool), you'll be getting feedback from partners?

It would be good to jointly finalise the requirement, e.g. agreeing on what type of "complex queries" may be useful to implement.

KathiSchleidt commented 1 month ago

@gmartirano trying to weigh the strengths and weaknesses of options A vs. B.

Are you sure that option B is that much more complex to query? Is the query functionality on the STAC API so bad?

On option A, how to you foresee maintaining alignment between the data catalog and your internal DB, nightly updates?

On steps 2.1 & 2.2 (updating the design for the Query Tool), I'd appreciate a dedicated workshop on this functionality. As we also need a workshop with the UC partners on dataset metadata requirements, could make sense to do this jointly

gmartirano commented 1 month ago

@gmartirano trying to weigh the strengths and weaknesses of options A vs. B.

Are you sure that option B is that much more complex to query? Is the query functionality on the STAC API so bad?

Not entirely sure, but quite confident :) However, because disregarding option B, should be this the final decision, will have to be based on some evidence, we will investigate a bit more. On option A, how to you foresee maintaining alignment between the data catalog and your internal DB, nightly updates?

Let's say regular updates. Nightly could be an option. On steps 2.1 & 2.2 (updating the design for the Query Tool), I'd appreciate a dedicated workshop on this functionality. As we also need a workshop with the UC partners on dataset metadata requirements, could make sense to do this jointly

Ok, good idea!