payloadcms / payload

Payload is the open-source, fullstack Next.js framework, giving you instant backend superpowers. Get a full TypeScript backend and admin panel instantly. Use Payload as a headless CMS or for building powerful applications.
https://payloadcms.com
MIT License
25.73k stars 1.64k forks source link

Duplicate network requests: retrieving docs from Relationship Fields #5160

Open BrianJM opened 8 months ago

BrianJM commented 8 months ago

Link to reproduction

https://github.com/payloadcms/payload/tree/main/test/fields-relationship

Describe the Bug

Two network requests are initiated to retrieve the docs from a Relationship field. Only one should be initiated.

This code may be responsible for duplicate network requests. Identified in #4350 in this comment.

To Reproduce

1) Run test/fields-relationship and open a record from http://localhost:3000/admin/collections/fields-relationship.

2) Inspect the Network tab and observe identical two requests (for each relationship field collection). http://localhost:3000/api/relation-one?depth=0&draft=true&limit=1&where%5Bid%5D%5Bin%5D%5B0%5D=65d90540b34af3efcdc1979a

localhost.har

Payload Version

2.11.1

Adapters and Plugins

db-mongodb

BrianJM commented 8 months ago

This impacts performance, rate limiting, and the user experirence.

The example below shows the networks requests for a doc with four relationship fields that are populated, three to the users collection (lastModifiedBy, createdBy, owner) and one to logs-import.

There are a total of eight relationships on the page (the other four are not populated). When all fields are populated, which is typical in this use case, there would be eight duplicate requests for each doc view. The complexity and depth of the relationships further impact performance (i.e., the related doc has other relationships).

The UI does not indicate that the server responded with a 429. Rather, the user sits in an endless transitory state (e.g., loading a doc, clicking a button) until the user abandons the workflow by refreshing the page until the rate limit resets (because it's fixed, not sliding).

@DanRibbens In production, this is a significant performance defect (only triggered in the UI) that can dramatically increase resource requirements and costs (depending on the use case). Rate limiting may need to be enforced outside of Payload to mitigate this in the interim (e.g., HAProxy, Varnish).

Consider 100 concurrent users working within Payload over a one hour period, using it as a CRM. The average user views 50 docs in an hour, with an average of four relationships populated per doc. That results in 20,000 duplicate queries/transactions per hour (total, for all concurrent users).

image