microsoft / agogosml

agogosml is a flexible data processing pipeline that addresses the common need for operationalizing ML models at scale
MIT License
34 stars 16 forks source link

Can we leverage connexion in simple app's main.py? #293

Open c-w opened 5 years ago

c-w commented 5 years ago

In the simple app main.py template, a very low level construct is used to serve http requests. The http server validates the incoming requests against a JSON schema, transforms the data and then creates a response.

This is a common pattern in the web world: we want to ensure that some request corresponds to our desired schema, call a method on the request body and respond with some data. A great abstraction for this workflow is connexion which is a framework that lets you create http servers based on OpenAPI/Swagger specifications. The framework takes care of validating that the request matches the Swagger schema, routing the request to a particular Python function and serializing the response. As such, it seems like a great match for our use-case.

For example, to translate the simple app into connexion, the following would be all that's required:

File api.spec.yaml

swagger: '2.0'

info:
  title: My simple API
  version: '0.1'

basePath: '/'

paths:
  '/':
    post:
      summary: Transform the data.
      operationId: datahelper.transform
      consumes:
        - application/json
      parameters:
        - $ref: '#/parameters/Data'
      responses:
        200:
          description: The transformed data.

parameters:
  Data:
    name: data
    description: The data to transform.
    in: body
    schema:
      $ref: '#/definitions/Data'
    required: true

definitions:
  Data:
    properties:
      key:
        description: The key of the input.
        type: string
      intValue:
        description: The value of the input.
        type: number
    required:
      - key
      - intValue

File main.py

from connexion import App

app = App(__name__)
app.add_api('api.spec.yaml')
app.run(port=8080, host='0.0.0.0')

The business logic method datahelper.transform doesn't even have to know that it's dealing with JSON inputs anymore! It simply gets a deserialized object passed into it that has the keys/values specified in the Swagger spec. Another advantage of connexion is that we can chose from a wide range of server backends like flask, tornado, aiohttp, etc. so it's trivial to get good performance out of the app with just a configuration change.

Last but not least, we can abstract the Swagger spec and use templating so that the user only has to provide the type definitions for the data and never edit the entire Swagger spec file, e.g.:

File input.spec.yaml

Data:
  properties:
    key:
      description: The key of the input.
      type: string
    intValue:
      description: The value of the input.
      type: number
  required:
    - key
    - intValue

There's an example of how to bind this fragment into the Swagger spec here.

Another benefit of this approach is that we'll get self-documenting APIs since a Swagger spec will be auto-loaded to ${basePath}/swagger.json

sayar commented 5 years ago

👍 please submit PR

fnocera commented 5 years ago

Lets do it!!