datalad / datalad-catalog

Create a user-friendly data catalog from structured metadata
https://datalad-catalog.netlify.app
MIT License
15 stars 12 forks source link

NF: add number of subdatasets to dataset view and on each item in subdatasets tab #280

Closed jsheunis closed 1 year ago

jsheunis commented 1 year ago

So that:

jsheunis commented 1 year ago

does not have to click on a subdataset in order to see how many subdatasets it has in turn

This will require a change with costs that we likely do not want to incur. Currently the following happens when entering a dataset page:

  1. try to fetch all of the dataset's subdatasets
  2. if success: mark subdataset as available and grab some subdataset metadata to store on dataset structure (name, authors, keywords, etc)
  3. if failure: mark subdataset as unavailable (i.e. it is listed as a submodule of the current dataset, but it does not have its own catalog entry)

With this process, we can count the number of subdatasets (both available and unavailable in the catalog). Note: I think it would make sense from a UX perspective to display the number of available datasets (not the total).

BUT, if we also want to display the number of subdatasets of each subdataset (i.e. in the list of subdatasets where we currently display subdataset name, authors, keywords, etc) we will have to recurse into subsubdatasets at step 2 above, and this might introduce unwanted delays, i.e. wait for info on a dataset page to load just because one of its subdatasets has a thousand subdatasets.

So unless we want to implement something else to achieve the same (e.g. run some service that routinely checks subsubdatasets and updates the number on all dataset entries correctly) I think we should not add the subsubdataset count.

However, showing the subdataset count of the current dataset is feasible, e.g.:

Screenshot 2023-04-13 at 16 12 45
mih commented 1 year ago

It sounds like this is not worth the investment. I am not familiar enough with the processing logic, but it seems to make sense that such a "reach-down" would break encapsulation and make things more complicated.

jsheunis commented 1 year ago

I agree