bug: enums inputs break input matching

[elisabettai]

Initially reported by @guidon:

I was adding an enum to the inputs of iSolve. Schema is identical to the ascent comp service in osparc.io. The input match freaks out server side. I am wondering if this also affects the API (creation of the pipeline and parsing of inputs)

And also @elisabettai found the same while developing a new comp. service locally (see animation below)

When an enum is added in the input port schema, it breaks input matching for files. This is the error I see in the webserver:

ERROR: [2023-09-26 12:01:28,310/MainProcess] [servicelib.aiohttp.monitoring:middleware_handler(227)]  -  Unexpected server error "<class 'aiohttp.web_exceptions.HTTPInternalServerError'>" from access: 10.0.1.38 "GET /v0/catalog/services/simcore/services/comp/iterative_solver/0.1.0/inputs:match" done in 0.03 secs. Responding with status 500
Traceback (most recent call last):
  File "/home/scu/.venv/lib/python3.10/site-packages/servicelib/aiohttp/rest_middlewares.py", line 76, in _middleware_handler
    return await handler(request)
  File "/home/scu/.venv/lib/python3.10/site-packages/servicelib/aiohttp/rest_middlewares.py", line 166, in _middleware_handler
    resp = await handler(request)
  File "/home/scu/.venv/lib/python3.10/site-packages/simcore_service_webserver/products/_middlewares.py", line 70, in discover_product_middleware
    return await handler(request)
  File "/home/scu/.venv/lib/python3.10/site-packages/servicelib/aiohttp/long_running_tasks/_error_handlers.py", line 14, in base_long_running_error_handler
    return await handler(request)
  File "/home/scu/.venv/lib/python3.10/site-packages/simcore_service_webserver/login/decorators.py", line 45, in _wrapper
    return await handler(request)
  File "/home/scu/.venv/lib/python3.10/site-packages/simcore_service_webserver/security/decorators.py", line 23, in wrapped
    ret = await handler(request)
  File "/home/scu/.venv/lib/python3.10/site-packages/simcore_service_webserver/catalog/_handlers.py", line 186, in get_compatible_inputs_given_source_output
    data = await _api.get_compatible_inputs_given_source_output(
  File "/home/scu/.venv/lib/python3.10/site-packages/simcore_service_webserver/catalog/_api.py", line 189, in get_compatible_inputs_given_source_output
    if can_connect(from_output, to_input, units_registry=ctx.unit_registry):
  File "/home/scu/.venv/lib/python3.10/site-packages/simcore_service_webserver/catalog/_api_units.py", line 73, in can_connect
    to_type = _get_type_name(to_input)
  File "/home/scu/.venv/lib/python3.10/site-packages/simcore_service_webserver/catalog/_api_units.py", line 17, in _get_type_name
    _type = port.content_schema["type"]
KeyError: 'type'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/scu/.venv/lib/python3.10/site-packages/servicelib/aiohttp/monitoring.py", line 183, in middleware_handler
    resp = await handler(request)
  File "/home/scu/.venv/lib/python3.10/site-packages/aiozipkin/aiohttp_helpers.py", line 157, in aiozipkin_middleware
    resp = await handler(request)
  File "/home/scu/.venv/lib/python3.10/site-packages/aiohttp_session/__init__.py", line 215, in factory
    raise cast(web.HTTPException, response)
  File "/home/scu/.venv/lib/python3.10/site-packages/aiohttp_session/__init__.py", line 199, in factory
    response = await handler(request)
  File "/home/scu/.venv/lib/python3.10/site-packages/servicelib/aiohttp/rest_middlewares.py", line 135, in _middleware_handler
    _process_and_raise_unexpected_error(request, err)
  File "/home/scu/.venv/lib/python3.10/site-packages/servicelib/aiohttp/rest_middlewares.py", line 64, in _process_and_raise_unexpected_error
    raise resp
aiohttp.web_exceptions.HTTPInternalServerError: Unexpected Server error
Stack (most recent call last):
  File "/home/scu/.venv/bin/gunicorn", line 8, in <module>
    sys.exit(run())
  File "/home/scu/.venv/lib/python3.10/site-packages/gunicorn/app/wsgiapp.py", line 67, in run
    WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
  File "/home/scu/.venv/lib/python3.10/site-packages/gunicorn/app/base.py", line 231, in run
    super().run()
  File "/home/scu/.venv/lib/python3.10/site-packages/gunicorn/app/base.py", line 72, in run
    Arbiter(self).run()
  File "/home/scu/.venv/lib/python3.10/site-packages/gunicorn/arbiter.py", line 202, in run
    self.manage_workers()
  File "/home/scu/.venv/lib/python3.10/site-packages/gunicorn/arbiter.py", line 551, in manage_workers
    self.spawn_workers()
  File "/home/scu/.venv/lib/python3.10/site-packages/gunicorn/arbiter.py", line 622, in spawn_workers
    self.spawn_worker()
  File "/home/scu/.venv/lib/python3.10/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker
    worker.init_process()
  File "/home/scu/.venv/lib/python3.10/site-packages/aiohttp/worker.py", line 51, in init_process
    super().init_process()
  File "/home/scu/.venv/lib/python3.10/site-packages/gunicorn/workers/base.py", line 142, in init_process
    self.run()
  File "/home/scu/.venv/lib/python3.10/site-packages/aiohttp/worker.py", line 57, in run
    self.loop.run_until_complete(self._task)
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 636, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 603, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 1909, in _run_once
    handle._run()
  File "/home/scu/.venv/lib/python3.10/site-packages/servicelib/aiohttp/monitor_slow_callbacks.py", line 31, in instrumented
    retval = _run(self)
  File "/usr/local/lib/python3.10/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/home/scu/.venv/lib/python3.10/site-packages/aiohttp/web_protocol.py", line 433, in _handle_request
    resp = await request_handler(request)
  File "/home/scu/.venv/lib/python3.10/site-packages/aiohttp/web_app.py", line 504, in _handle
    resp = await handler(request)
  File "/home/scu/.venv/lib/python3.10/site-packages/aiohttp/web_middlewares.py", line 117, in impl
    return await handler(request)
  File "/home/scu/.venv/lib/python3.10/site-packages/servicelib/aiohttp/monitoring.py", line 227, in middleware_handler
    log.error(
INFO: [2023-09-26 12:01:28,311/MainProcess] [gunicorn.access:log(206)]  -  10.0.1.38 [26/Sep/2023:12:01:28 +0000] "GET /v0/catalog/services/simcore%2Fservices%2Fcomp%2Fiterative_solver/0.1.0/inputs:match?fromService=simcore%2Fservices%2Ffrontend%2Ffile-picker&fromVersion=1.0.0&fromOutput=outFile HTTP/1.1" 500 284 [33196us] "http://172.16.9.180.nip.io:9081/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/110.0"

Just for reference, this is the metadata.yml of the service:

inputs:
  input_1:
    displayOrder: 1
    label: Combined Mesh
    description: MATLAB (.mat) file with base mesh data
    type: data:*/*
    fileToKeyMap:
      CombinedMesh.mat: input_1
  input_2:
    displayOrder: 2
    label: Combined Mesh P
    description: MATLAB (.mat) file with neighbor integrals and preconditioner
    type: data:*/*
    fileToKeyMap:
      CombinedMeshP.mat: input_2
  input_3:
    displayOrder: 3
    label: Electrode Data
    description: MATLAB (.mat) file with electrodes imprinted
    type: data:*/*
    fileToKeyMap:
      electrode_data.mat: input_3
  input_4:
    displayOrder: 4
    label: Solution method
    description: Choose between the iterative solution (based on MATLAB GMRES) or the BMM-LU decomposition method
    type: ref_contentSchema
    contentSchema: 
      title: Solution method
      default: Iterative (GMRES)
      enum: [Iterative (GMRES), FMM-LU]

[elisabettai]

@odeimaiz, is it something that be easily fixed or do we need backend intervention?

[odeimaiz]

What @mguidon reported is pure backend. Regarding what I see in the animation, I would need to have look.

[pcrespov]

@elisabettai @mguidon @odeimaiz there is a way around to this problem by using "pure-strings" enums on your schema. For instance, in this case modify the json-schema such that it ads a type:

 {
   "title": "Solution method",
   "description": "An enumeration",
   "enum": [ "Iterative (GMRES)",  "FMM-LU"],
   "type": "string"  # <-- NOTE THIS
}

This is legal in json-schema and you are basically saying that the values of the enum can only be strings. For more insight on the enum schemas you can look at https://github.com/pcrespov/sandbox-python/blob/98fb613a41e9c6a5a54f52be321cb652bfe047da/json-schemas/test_schemas_with_enums.py

[mguidon]

This works like a charm! Thanks.