search

sci-ndp / scidx-api

sciDX API
Apache License 2.0
1 stars 2 forks source link

`processing` during URL registration #94

Open philip-davis opened 2 months ago

philip-davis commented 2 months ago

We have changed the processing argument to optional in the python bindings, but it is needed in the validator for URLRequest:

https://github.com/sci-ndp/scidx-api/blob/5d48fac1fa144a55b181007d55613198782c5f10/api/models/urlrequest_model.py#L159

even though processing is optional in the model itself.

I'm trying to register a netcdf url, and it's not clear to me what the correct value for group should be, and what are the implications of any particular choice for that value:

image

I guess the ways forward I can imagine are:

  1. A trivial, default file_type that maps to an empty processing dict. I personally think file_type is an intuitive and important attribute of a url registration, so I don't prefer that option.
  2. Tolerate an empty processing dict for some or all values of file_type. I don't know how processing is used, so I don't know what would be lost by doing that.
  3. Add logic to set reasonable defaults for any given file_type in the client bindings or API.
  4. Compel the user to provide processing in the client bindings, and sufficiently document the correct values to use. If processing is only narrowly useful, I'm not sure that's worthwhile.

My intuition is towards either 2 or 3 (with the logic in the API). But I really don't know what processing is used for, so I could easily be missing the right option.

rbardaji commented 2 months ago

Honestly, I’m not sure what the purpose of this argument is. I know it's something Andreu code needs. I could remove it from the API validation, but first, I need to confirm whether this would impact Andreu. I’m assigning this issue to Andreu for further clarification.

Andreufb commented 2 months ago

This problem has been solved in the branch I am currently working on. I need to finsih some more functionalitites and test them before asking for a merge but I will try to make it soon.