lmmx / tubeulator

TfL open data interface library
https://tubeulator.vercel.app
MIT License
2 stars 0 forks source link

DTOs for all route path parameters #19

Open lmmx opened 1 year ago

lmmx commented 1 year ago

In issue #18 I identified that there are no optional parameters in the current design of this API

It's shown on the API documentation page here as

https://api.tfl.gov.uk/StopPoint/Mode/{modes}[?page]

However the definition in my code for this method is

https://github.com/lmmx/tubeulator/blob/9b5a0e0827c783a3fc4ecbc67509b241a8cf6209/src/tubeulator/api/endpoint/routes/stop.py#L15-L16

There are no optional parameters available!

These are needed now, and hopefully can be done without changing the existing API design (I'd rather keep the current string extraction approach to URL parametrisation than use the schema... an API implementation should not be coupled so tightly to a schema in that way).

The real reason this is an issue is that the current schema uses the path names as the path specification, which is not correct. The path name is only a partial specification, the full specification is in the schema properties alongside it, which means more DTOs needed!

I've so far generated DTOs from the schemas, but there are also paths (API routes), and their call signature can be encoded as DTOs too.

I've discovered there's a tool datamodel-code-generator which generates some info from OpenAPI schemas, but I tried it on the path parameters and it failed:

Models not found in the input data

Maybe it'd be good to follow the idea of that package and break out the processing approach I took in this package for general purpose use.

lmmx commented 1 year ago

I developed godto (first I made a fork of datamodel-code-generator then I ran it on the OpenAPI v3 spec, which is itself a jsonschema) so we can now get a dataclass-based model of any OpenAPI (v3) spec, such as the TfL API schemas.

from pathlib import Path

from godto.openapi.v3 import Model

schema_json = Path("data/openapi/StopPoint/StopPoint.json").read_text()
Model.from_json(schema_json)
>>> from godto.openapi.v3 import Model
>>> from pathlib import Path
>>> p = Path("/home/louis/dev/tubeulator/src/tubeulator/data/openapi/StopPoint/StopPoint.json")
>>> m = Model.from_json(p.read_text())
>>> from pprint import pprint
>>> # pprint(m)
>>> pprint(m.paths["/{id}/Route"].get.parameters)
[{'description': 'A stop point id (station naptan codes e.g. 940GZZLUASL, you '
                 'can use /StopPoint/Search/{query} endpoint to find a stop '
                 'point id from a station name)',
  'in': 'path',
  'name': 'id',
  'required': 'True',
  'schema': "{'type': 'string'}"},
 {'description': 'A comma-separated list of service types to filter on. If not '
                 'specified. Supported values: Regular, Night. Defaulted to '
                 "'Regular' if not specified",
  'in': 'query',
  'name': 'serviceTypes',
  'schema': "{'enum': ['Regular', 'Night'], 'type': 'string'}"}]

godto is available on PyPI and has minimal dependencies (overlapping in this case) on dataclass_wizard. In contrast, datamodel-code-generator has slightly heavier dependencies, on pydantic etc.

Therefore we can import the Model class and parse the entire schema, and then specifically we can use the nice structured access to the path parameters. With this structured info on the parameters, we can make DTOs for them.