Roadmap

[sobolevn]

Initial ideas

1. swagger should be supported in the core library, we prefer to get v3 support
2. Strict separation of parser / validation / logic / render steps
3. Strict separation of views and business logic
4. Schema validation and logical validation should be two different kinds of validation, not mixed together
5. django-filter and native django.Paginator integration
6. jwt should be the default auth mechanism
7. We should support different patterns of API versioning, but make them as light as possible, since generic solutions do not work well for this kind of stuff. Including: URL versioning and hostname versioning

General

1. We should be using typing everywhere

Steps

Runs once on startup:

1. API declaration

Runs per each request:

1. Decorators (django csrf, auth, permissions, etc)
2. Parser, converts text in json, xml, etc formats to python native structures
3. Schema validation, converts native structures to dataclasses
4. Logical validation, ensures that passed objects are passing business logic checks
5. Controller, executes business logic
6. Renderer, converts dataclass to native structures
7. Returns response

Usage examples

Endpoint registration

from server.your_app.views import UserEndpoint

api = APIDeclaration(**some_swagger_information)
api.add_endpoint('user', UserEndpoint)

urlpatterns = [
    url('api/v1', include(api.endpoints)),
]

Endpoint declaration

from wemake_django_rest.parsers import JSONParser
from wemake_django_rest.renderers import JSONRenderer
from wemake_django_rest.views import APIEndpoint, actions

class _UserController(object):
     def __init__(self, request, parsed_data) -> None: ...
     def __call__(self) -> UserResponseRepresentation: ...

class UserEndpoint(APIEndpoint):
     @actions(  # everything is optional:
         parser=JSONParser, 
         renderer=JSONRenderer,
         logic_validator=SomeLogicValidator,
     )
     def get(self, request: DjangoRequest, parsed_data: UserRepresentation) -> Callable:
            return _UserController(request, parsed_data)

Serializers

I do not like the concept of serializers. I like dataclasses that are now part of the python's standard library. Or maybe TypedDict, but they are only mypy_extensions level only. attrs also should be supported for python3.6.

When using drf currently I always end up writing tons of dataclasses and typed dics, because I always have functions and classes that work with the raw data.

That's how it looks:

from dataclasses import dataclass
from typing import List, Optional

@dataclass
class GroupRepresentation(object):
     id: int
     name: str

@dataclass
class UserRepresentation(object):
     username: str
     email: str
     is_staff: bool
     hometown: Optional[str] = None
     groups: List[GroupRepresentation] = field(default_factory=list)

Any logic can be executed in __post_init__ hook of a dataclass. Also, when some logical validation is required (like unique=True it should be done in logic_validator stage.

Global TODOs

1. Think of how this can be improved to be more declarative. Good examples: eve, rdf
2. Think of how errors should be handled and declared
3. Think of how we can fix the nesting problem of drf. Custom fields like NestedField('data.key.value)'? As we use it now forkira`

[sobolevn]

https://github.com/Microsoft/api-guidelines/blob/vNext/Guidelines.md

[sobolevn]

New ideas:

# representations.py
@dataclass
class ShortUserRepresentation(object):
      id: int
      username: str
      email: str

# views.py
class _AbstractUserMethod(APIMethod, abstract=True):
     queryset = User.objects.filter(is_active=True)
     permissions = [permissions.IsAdmin]

class UserListMethod(_AbstractUserMethod):
      renderer = ShortUserRepresentation

class UserDetailsMethod(_AbstractUserMethod):
      renderer = FullUserRepresentation

class UserCreateMethod(_AbstractUserMethod):
      parser = create_model_parser(User, exclude=['id'])  # introspects model and returns a dataclass with all fields
      renderer = FullUserRepresentation

      # To perform logic, define one of:
      controller: Callable = function_or_callable_class
      def controller(self, request: DjangoRequest, parsed_data: FullUserRepresentationWithoutId) -> Callable:
            return _CallableClassWithLogic(request, parsed_data)

# urls.py
api = APIDeclaration(**some_swagger_information)
api.add_endpoint('user', get=UserListMethod, post=UserCreateMethod)
api.add_endpoint('user/{pk}', get=UserDetailsMethod)

[sobolevn]

class UserCreateMethod(APIMethod):
     validators = [
         unique('email'), 
         unique('username'), 
         should_equal('password1', 'password2'),
     ]
     parser = UserDataclassWithoutId
     renderer = UserDataclassWithId

[sobolevn]

ResponseWrapper concept

ResponseWrapper is something that wraps python data like {'username': 'sobolevn'} into more complex construct: {'data': {'username': 'sobolevn'}, '_meta': {...}, '_pagination': {...}}

Code example:

@HATEOASWrapper
class UserDetailsMethod(_AbstractUserMethod):
      renderer = FullUserRepresentation

[sobolevn]

swagger

There are basically two ways of implementing swagger specific data model and rendering:

1. reuse existing solution
2. write from scratch

I personally prefer the first one. There are several implementations worth mentioning:

• https://github.com/p1c2u/openapi-core/tree/master/openapi_core/schema (v3)
• https://github.com/axnsan12/drf-yasg/blob/master/src/drf_yasg/openapi.py (v2)
• https://github.com/p1c2u/openapi-spec-validator/blob/master/openapi_spec_validator/resources/schemas/v3.0.0/schema.json (v3)

swagger ui

I am not of a fan of the idea to use custom templates for swagger-ui or even have these files and templates in the core library.

May be we can go with using standalone swagger-ui package?

[sobolevn]

Settings

We should not have any settings. I do not like settings. Since people might do something wrong with them.

It is even better not to have settings, so we can use dependency injection and be free from global variables.

That's how I see it:

api = APIDeclaration(
    accepts=['application/json', 'application/xml'],
    extra_parser_fields={Email: EmailParser},
    auth_backend=JWTAuthBackend,  # or = PublicAuthBacked
)

What else needs to be configured?

See:

• https://github.com/freelancer/pavlova#adding-custom-types
• https://www.django-rest-framework.org/api-guide/settings/

[sobolevn]

Naming

I am not sure about what parser should mean. I think of it as some kind of a serialization mechanism that turns raw dicts into dataclasses like pavlova does.

And the renderer is the opposite. Turning dataclasses into dicts, like dataclass.asdict() does.

But, there's another type of renderer and parser I can think of. Which turn dict into json formatter text or xml formatted text and vice-a-versa.

So, maybe we can call the first concept request_payload and response_payload and leave parser and renderer alone? This naming seems better.

[sobolevn]

Exceptions

There should be a regular exception handling mechanism. But it will consist of at least three stages:

1. Before controller (permissions, auth, request_payload, logical validation)
2. Inside controller
3. After controller (response_payload, wrappers)

[sobolevn]

Responses, headers, and errors declaration

We need to declare a lot of things for the swagger spec to be able to work correctly. Including: headers, response codes, descriptions, schemas, etc

class UserDetailMethod(APIMethod):
     response_payload = UserDataclassWithId

     class Meta(APIMethodMetaProtocol):
          """This class is only required to supply meta information, it does not alter behavior."""
         errors = [api.HTTP404(description='User with given pk does not exist'), ]
         response_headers = [OpenAPIHeaders(name='X-RateLimit-Limit', type='integer')]

See:

• https://swagger.io/docs/specification/describing-responses/

Testing swagger specs

We should bundle https://pypi.org/project/swagger-conformance/ with the app. So, we will test that some API methods are incorrect. And its Meta needs to be fixed.

This should be both .testing package feature as well as a documentation feature.

[sobolevn]

Observability by default

We need to provide health-checks by default. Or we can reuse some existing libs that do that.

[sobolevn]

https://apispec.readthedocs.io/en/stable/quickstart.html#basic-usage

[sobolevn]

https://github.com/tiangolo/fastapi/blob/master/fastapi/applications.py