search

fractal-analytics-platform / fractal-web

Web client for Fractal
https://fractal-analytics-platform.github.io/fractal-web/
BSD 3-Clause "New" or "Revised" License
6 stars 0 forks source link

Support new `pydantic_v2` manifest and JSON Schemas #531

Closed tcompa closed 3 months ago

tcompa commented 4 months ago

As already explored/discussed in https://github.com/fractal-analytics-platform/fractal-web/issues/479, we are currently not supporting the anyOf pattern for nullable JSON Schema properties - although we may want to do it in the future. Thus are working in fractal-tasks-core to move to Pydantic V2 while keeping the generated manifest as close as possible to the V1 one (see e.g. https://github.com/fractal-analytics-platform/fractal-tasks-core/pull/801).

Here is an example of the old/new manifests for fractal-tasks-core (perhaps up to some last-minute additions):

Manifest with Pydantic V1: https://raw.githubusercontent.com/fractal-analytics-platform/fractal-tasks-core/f40116aedd211dac4a1b6d3eea3dbbb7734b1e86/fractal_tasks_core/__FRACTAL_MANIFEST__.json

Manifest with Pydantic v2: manifest_pydantic_v2.json

We should review the remaining differences, and handle them in fractal-web.

One that I already noticed is that "definitions" is now replaced by "$defs". I'll mention here any other change that I find.

tcompa commented 3 months ago

I took one of the new manifests, and I replaced $defs with definitions by hand (sed -i -e 's/$defs/definitions/g' fractal_tasks_core/__FRACTAL_MANIFEST__.json). The remaining diff seems quite harmless:

diff --git a/fractal_tasks_core/__FRACTAL_MANIFEST__.json b/fractal_tasks_core/__FRACTAL_MANIFEST__.json
index 824f18e3..73e2c599 100644
--- a/fractal_tasks_core/__FRACTAL_MANIFEST__.json
+++ b/fractal_tasks_core/__FRACTAL_MANIFEST__.json
@@ -1,5 +1,5 @@
 {
-  "args_schema_version": "pydantic_v1",
+  "args_schema_version": "pydantic_v2",
   "has_args_schemas": true,
   "manifest_version": "2",
   "task_list": [
@@ -49,7 +49,11 @@
                 "type": "string"
               },
               "window": {
-                "$ref": "#/definitions/Window",
+                "allOf": [
+                  {
+                    "$ref": "#/definitions/Window"
+                  }
+                ],
                 "description": "Optional `Window` object to set default display settings for napari.",
                 "title": "Window"
               }
@@ -217,7 +221,7 @@
           "init_args": {
             "$ref": "#/definitions/InitArgsCellVoyager",
             "description": "Intialization arguments provided by `create_cellvoyager_ome_zarr_init`.",
-            "title": "Init_Args"
+            "title": "Init Args"
           },
           "zarr_url": {
             "description": "Path or url to the individual OME-Zarr image to be processed. (standard argument for Fractal tasks, managed by Fractal server).",
@@ -316,7 +320,11 @@
                 "type": "string"
               },
               "window": {
-                "$ref": "#/definitions/Window",
+                "allOf": [
+                  {
+                    "$ref": "#/definitions/Window"
+                  }
+                ],
                 "description": "Optional `Window` object to set default display settings for napari.",
                 "title": "Window"
               }
@@ -478,7 +486,7 @@
           "init_args": {
             "$ref": "#/definitions/InitArgsCellVoyager",
             "description": "Intialization arguments provided by `create_cellvoyager_ome_zarr_init`.",
-            "title": "Init_Args"
+            "title": "Init Args"
           },
           "zarr_url": {
             "description": "Path or url to the individual OME-Zarr image to be processed. (standard argument for Fractal tasks, managed by Fractal server).",
@@ -566,7 +574,7 @@
           "init_args": {
             "$ref": "#/definitions/InitArgsMIP",
             "description": "Intialization arguments provided by `create_cellvoyager_ome_zarr_init`.",
-            "title": "Init_Args"
+            "title": "Init Args"
           },
           "overwrite": {
             "default": false,
@@ -862,7 +870,7 @@
           "advanced_cellpose_model_params": {
             "$ref": "#/definitions/CellposeModelParams",
             "description": "Advanced Cellpose model parameters that are passed to the Cellpose `model.eval` method.",
-            "title": "Advanced_Cellpose_Model_Params"
+            "title": "Advanced Cellpose Model Params"
           },
           "channel": {
             "$ref": "#/definitions/CellposeChannel1InputModel",
@@ -1025,14 +1033,15 @@
               "phase_cross_correlation",
               "chi2_shift"
             ],
-            "title": "RegistrationMethod"
+            "title": "RegistrationMethod",
+            "type": "string"
           }
         },
         "properties": {
           "init_args": {
             "$ref": "#/definitions/InitArgsRegistration",
             "description": "Intialization arguments provided by `image_based_registration_hcs_init`. They contain the reference_zarr_url that is used for registration. (standard argument for Fractal tasks, managed by Fractal server).",
-            "title": "Init_Args"
+            "title": "Init Args"
           },
           "level": {
             "default": 2,
@@ -1145,7 +1154,7 @@
           "init_args": {
             "$ref": "#/definitions/InitArgsRegistrationConsensus",
             "description": "Intialization arguments provided by `init_group_by_well_for_multiplexing`. It contains the zarr_url_list listing all the zarr_urls in the same well as the zarr_url of the reference acquisition that are being processed. (standard argument for Fractal tasks, managed by Fractal server).",
-            "title": "Init_Args"
+            "title": "Init Args"
           },
           "new_roi_table": {
             "description": "Optional name for the new, registered ROI table. If no name is given, it will default to \"registered_\" + `roi_table`",
@@ -1331,7 +1340,11 @@
             "description": "A value of the `input_specs` argument in `napari_workflows_wrapper`.",
             "properties": {
               "channel": {
-                "$ref": "#/definitions/ChannelInputModel",
+                "allOf": [
+                  {
+                    "$ref": "#/definitions/ChannelInputModel"
+                  }
+                ],
                 "description": "`ChannelInputModel` object (for image inputs only).",
                 "title": "Channel"
               },
@@ -1466,7 +1479,7 @@
   ]
 }
 {
-  "args_schema_version": "pydantic_v1",
+  "args_schema_version": "pydantic_v2",
   "has_args_schemas": true,
   "manifest_version": "2",
   "task_list": [
@@ -1516,7 +1529,11 @@
                 "type": "string"
               },
               "window": {
-                "$ref": "#/definitions/Window",
+                "allOf": [
+                  {
+                    "$ref": "#/definitions/Window"
+                  }
+                ],
                 "description": "Optional `Window` object to set default display settings for napari.",
                 "title": "Window"
               }
@@ -1684,7 +1701,7 @@
           "init_args": {
             "$ref": "#/definitions/InitArgsCellVoyager",
             "description": "Intialization arguments provided by `create_cellvoyager_ome_zarr_init`.",
-            "title": "Init_Args"
+            "title": "Init Args"
           },
           "zarr_url": {
             "description": "Path or url to the individual OME-Zarr image to be processed. (standard argument for Fractal tasks, managed by Fractal server).",
@@ -1783,7 +1800,11 @@
                 "type": "string"
               },
               "window": {
-                "$ref": "#/definitions/Window",
+                "allOf": [
+                  {
+                    "$ref": "#/definitions/Window"
+                  }
+                ],
                 "description": "Optional `Window` object to set default display settings for napari.",
                 "title": "Window"
               }
@@ -1945,7 +1966,7 @@
           "init_args": {
             "$ref": "#/definitions/InitArgsCellVoyager",
             "description": "Intialization arguments provided by `create_cellvoyager_ome_zarr_init`.",
-            "title": "Init_Args"
+            "title": "Init Args"
           },
           "zarr_url": {
             "description": "Path or url to the individual OME-Zarr image to be processed. (standard argument for Fractal tasks, managed by Fractal server).",
@@ -2033,7 +2054,7 @@
           "init_args": {
             "$ref": "#/definitions/InitArgsMIP",
             "description": "Intialization arguments provided by `create_cellvoyager_ome_zarr_init`.",
-            "title": "Init_Args"
+            "title": "Init Args"
           },
           "overwrite": {
             "default": false,
@@ -2329,7 +2350,7 @@
           "advanced_cellpose_model_params": {
             "$ref": "#/definitions/CellposeModelParams",
             "description": "Advanced Cellpose model parameters that are passed to the Cellpose `model.eval` method.",
-            "title": "Advanced_Cellpose_Model_Params"
+            "title": "Advanced Cellpose Model Params"
           },
           "channel": {
             "$ref": "#/definitions/CellposeChannel1InputModel",
@@ -2492,14 +2513,15 @@
               "phase_cross_correlation",
               "chi2_shift"
             ],
-            "title": "RegistrationMethod"
+            "title": "RegistrationMethod",
+            "type": "string"
           }
         },
         "properties": {
           "init_args": {
             "$ref": "#/definitions/InitArgsRegistration",
             "description": "Intialization arguments provided by `image_based_registration_hcs_init`. They contain the reference_zarr_url that is used for registration. (standard argument for Fractal tasks, managed by Fractal server).",
-            "title": "Init_Args"
+            "title": "Init Args"
           },
           "level": {
             "default": 2,
@@ -2612,7 +2634,7 @@
           "init_args": {
             "$ref": "#/definitions/InitArgsRegistrationConsensus",
             "description": "Intialization arguments provided by `init_group_by_well_for_multiplexing`. It contains the zarr_url_list listing all the zarr_urls in the same well as the zarr_url of the reference acquisition that are being processed. (standard argument for Fractal tasks, managed by Fractal server).",
-            "title": "Init_Args"
+            "title": "Init Args"
           },
           "new_roi_table": {
             "description": "Optional name for the new, registered ROI table. If no name is given, it will default to \"registered_\" + `roi_table`",
@@ -2798,7 +2820,11 @@
             "description": "A value of the `input_specs` argument in `napari_workflows_wrapper`.",
             "properties": {
               "channel": {
-                "$ref": "#/definitions/ChannelInputModel",
+                "allOf": [
+                  {
+                    "$ref": "#/definitions/ChannelInputModel"
+                  }
+                ],
                 "description": "`ChannelInputModel` object (for image inputs only).",
                 "title": "Channel"
               },
tcompa commented 3 months ago

TL;DR

The only relevant differences are

  1. The definitions renaming
  2. Sometimes we have a dummy single-element allOf, which should be already supported (https://github.com/fractal-analytics-platform/fractal-web/issues/413)

@zonia3000 if all seems reasonable to you, then the required change to fractal-web is just to accept both pydantic_v1 and pydantic_v2.

zonia3000 commented 3 months ago

@tcompa In the Add Single task component, when an args schema file is selected, we are displaying an "Args schema version" input having pydantic_v1 as default value. Should I set the default to pydantic_v2?

image

tcompa commented 3 months ago

In the Add Single task component, when an args schema file is selected, we are displaying an "Args schema version" input having pydantic_v1 as default value. Should I set the default to pydantic_v2?

Yes, this seems the correct approach to me.

For the record, the args_schema_version attribute was introduce only to be future-proof in fractal-web. It is not used or validated in any other Fractal component, as far as I remember. Therefore as soon as fractal-web supports both strings, it is fine to always default to the latest.