This PR goes part toward resolving #577 by starting to address the N+1 problem on the /creatures endpoint.
This content builds on @tylereed 's work in PR #592, only this time for the /creatures endpoint. One notable addition I made is the action parameter passes to setup_eager_loading method, which allows us to configure different eager loading options for list views (ie. v2/creatures) and single views (ie. v2/srd_goblin).
Follow ups
With the current configuration of the CreatureViewSet, it is possible to dynamically select a serializer based on whether the user has request ONE or MANY creatures. It might be a good idea to set these up to return sensible default fields for different contexts, as the Creature model is quite large.
Tests
Tests were conducted using Django Debug Tools
/creatures without fields
For endpoint v2/creatures/?limit=50, which fetches the first 50 creatures in the dataset. This endpoint currently massively overfetches for the purposes of the /monsters page on the front-end
The N+1 problem still very much exists when the data is fetched this way. The problematic fields are CreatureAction and CreatureAbility. These fields are still being actively worked on, so it feels a bit premature to begin optimising them. I am pointing this out because there is definitely a lot of scope to reduce these response times further.
/creatures with fields
For endpoint
v2/creatures/?fields=name,document,key,size,challenge_rating_decimal,challenge_rating_text&document__fields=name,key&size__fields=name,key&limit=50 which returns data in the following format (everything the /monsters page requires to fully function)
This PR goes part toward resolving #577 by starting to address the N+1 problem on the
/creatures
endpoint.This content builds on @tylereed 's work in PR #592, only this time for the
/creatures
endpoint. One notable addition I made is theaction
parameter passes tosetup_eager_loading
method, which allows us to configure different eager loading options for list views (ie.v2/creatures
) and single views (ie.v2/srd_goblin
).Follow ups
With the current configuration of the
CreatureViewSet
, it is possible to dynamically select a serializer based on whether the user has request ONE or MANY creatures. It might be a good idea to set these up to return sensible default fields for different contexts, as theCreature
model is quite large.Tests
Tests were conducted using Django Debug Tools
/creatures
without fieldsFor endpoint
v2/creatures/?limit=50
, which fetches the first 50 creatures in the dataset. This endpoint currently massively overfetches for the purposes of the/monsters
page on the front-endReduction: 1.126s (4 s.f.) Percent Reduction: 76.79% (4 s.f.)
The N+1 problem still very much exists when the data is fetched this way. The problematic fields are CreatureAction and CreatureAbility. These fields are still being actively worked on, so it feels a bit premature to begin optimising them. I am pointing this out because there is definitely a lot of scope to reduce these response times further.
/creatures
with fieldsFor endpoint
v2/creatures/?fields=name,document,key,size,challenge_rating_decimal,challenge_rating_text&document__fields=name,key&size__fields=name,key&limit=50
which returns data in the following format (everything the/monsters
page requires to fully function)Reduction (ms): 89.378 ms Reduction (%): 66.094% (4 s.f.)
Omitting all fields except those that would be required by , it looks like we can reduce response times by a third.