[Python] Create Event Type HTTP response fails to be parsed

[anlambert]

Since svix v1.29.0 and up to current release, the following error is reported when attempting to create an event type using Python:

swh_webhooks = <swh.webhooks.interface.Webhooks object at 0x7fb740857c90>
origin_create_event_type = EventType(name='origin.create', description='This event is triggered when a new software origin is added to the archiv...string', 'description': 'The URL of the newly created software origin', 'format': 'iri'}}, 'required': ['origin_url']})

    def test_create_valid_event_type(swh_webhooks, origin_create_event_type):
>       swh_webhooks.event_type_create(origin_create_event_type)

swh/webhooks/tests/test_webhooks.py:93: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
build/__editable__.swh.webhooks-0.3.1.dev3+g9b381c6.d20240903-py3-none-any/swh/webhooks/interface.py:251: in event_type_create
    self.svix_api.event_type.create(
../../../.virtualenvs/swh/lib/python3.11/site-packages/svix/api.py:885: in create
    return v1_event_type_create.request_sync(
../../../.virtualenvs/swh/lib/python3.11/site-packages/svix/internal/openapi_client/api/event_type/v1_event_type_create.py:154: in request_sync
    return request_sync_detailed(
../../../.virtualenvs/swh/lib/python3.11/site-packages/svix/internal/openapi_client/api/event_type/v1_event_type_create.py:125: in request_sync_detailed
    return _build_response(client=client, response=response)
../../../.virtualenvs/swh/lib/python3.11/site-packages/svix/internal/openapi_client/api/event_type/v1_event_type_create.py:72: in _build_response
    parsed=_parse_response(client=client, response=response),
../../../.virtualenvs/swh/lib/python3.11/site-packages/svix/internal/openapi_client/api/event_type/v1_event_type_create.py:47: in _parse_response
    response_201 = EventTypeOut.from_dict(response.json())
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

cls = <class 'svix.internal.openapi_client.models.event_type_out.EventTypeOut'>, src_dict = {'archived': False, 'createdAt': '2024-09-03T15:10:03.466102Z', 'description': 'This event is triggered when a new software origin is added to the archive', 'featureFlag': None, ...}

    @classmethod
    def from_dict(cls: Type[T], src_dict: Dict[str, Any]) -> T:
        d = src_dict.copy()
        created_at = isoparse(d.pop("createdAt"))

>       deprecated = d.pop("deprecated")
E       KeyError: 'deprecated'

../../../.virtualenvs/swh/lib/python3.11/site-packages/svix/internal/openapi_client/models/event_type_out.py:85: KeyError

To reproduce the issue, you can clone https://gitlab.softwareheritage.org/swh/devel/swh-webhooks/ repository and apply the following diff:

diff --git a/requirements.txt b/requirements.txt
index 7ed3459..85d697e 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,5 +1,5 @@
 click
 jsonschema[format]
-svix==1.23.0
+svix==1.29.0
 tenacity >= 8.4.2
 typing-extensions
diff --git a/swh/webhooks/tests/docker-compose.yml b/swh/webhooks/tests/docker-compose.yml
index b9eee38..643d751 100644
--- a/swh/webhooks/tests/docker-compose.yml
+++ b/swh/webhooks/tests/docker-compose.yml
@@ -1,7 +1,7 @@
 version: "3.7"
 services:
   svix-backend:
-    image: svix/svix-server
+    image: svix/svix-server:v1.29.0
     network_mode: bridge
     links:
       - svix-postgres

Then install tox in your Python environment with pip and execute tox in swh-webhooks directory (to be noted: docker compose is required to run these tests).

[anlambert]

For the record, the issue can be fixed by reverting ea6f38cf4d82700a94cb67593119f5860a66effe.

But I got hit by another Python client issue after that revert:

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
swh/webhooks/interface.py:629: in sent_events_list_for_endpoint
    for attempt in islice(svix_list(list_attempted_messages_by_endpoint), limit):
swh/webhooks/interface.py:121: in svix_list
    response = svix_list_request(iterator)
swh/webhooks/interface.py:616: in list_attempted_messages_by_endpoint
    return self.svix_api.message_attempt.list_by_endpoint(
../../../dev/svix-webhooks/python/build/__editable__.svix-1.32.0-py3-none-any/svix/api.py:1261: in list_by_endpoint
    return v1_message_attempt_list_by_endpoint.request_sync(
../../../dev/svix-webhooks/python/build/__editable__.svix-1.32.0-py3-none-any/svix/internal/openapi_client/api/message_attempt/v1_message_attempt_list_by_endpoint.py:303: in request_sync
    return request_sync_detailed(
../../../dev/svix-webhooks/python/build/__editable__.svix-1.32.0-py3-none-any/svix/internal/openapi_client/api/message_attempt/v1_message_attempt_list_by_endpoint.py:234: in request_sync_detailed
    return _build_response(client=client, response=response)
../../../dev/svix-webhooks/python/build/__editable__.svix-1.32.0-py3-none-any/svix/internal/openapi_client/api/message_attempt/v1_message_attempt_list_by_endpoint.py:130: in _build_response
    parsed=_parse_response(client=client, response=response),
../../../dev/svix-webhooks/python/build/__editable__.svix-1.32.0-py3-none-any/svix/internal/openapi_client/api/message_attempt/v1_message_attempt_list_by_endpoint.py:105: in _parse_response
    response_200 = ListResponseMessageAttemptOut.from_dict(response.json())
../../../dev/svix-webhooks/python/build/__editable__.svix-1.32.0-py3-none-any/svix/internal/openapi_client/models/list_response_message_attempt_out.py:64: in from_dict
    data_item = MessageAttemptOut.from_dict(data_item_data)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cls = <class 'svix.internal.openapi_client.models.message_attempt_out.MessageAttemptOut'>
src_dict = {'endpointId': 'ep_2lbKuaL4s5lW7oe39Ry3ipiAsMj', 'id': 'atmpt_2lbKucxSGDj6HqFLcSrJhmuJveo', 'msgId': 'msg_2lbKucvIIKDWf9qPyY0rUvfq352', 'response': 'OK', ...}

    @classmethod
    def from_dict(cls: Type[T], src_dict: Dict[str, Any]) -> T:
        from ..models.message_out import MessageOut

        d = src_dict.copy()
        endpoint_id = d.pop("endpointId")

        id = d.pop("id")

        msg_id = d.pop("msgId")

        response = d.pop("response")

>       response_duration_ms = d.pop("responseDurationMs")
E       KeyError: 'responseDurationMs'

../../../dev/svix-webhooks/python/build/__editable__.svix-1.32.0-py3-none-any/svix/internal/openapi_client/models/message_attempt_out.py:106: KeyError

This one can be fixed by reverting 52d1a88fb036f5f1daee0db8dcd0364270b38184.

It seems the OpenAPI specification is not synchronized with the rust implementation of the svix server.

[tasn]

Thanks for the report, we are checking!

[svix-jplatte]

Should be fixed with the two PRs linked above. The root openapi.json is intentionally not the same as server/openapi.json, as it includes extra endpoints only available in our cloud / enterprise offerings and not the OSS version of svix. Any differences that cause the clients to not work with supported endpoints of the OSS server are of course bugs, though.

We've got some other bugs to fix w.r.t. to the libraries, so it might take a couple of days until we release the fixes.