🐛 Bug Report: Paginated catalog page does not order entities correctly

[drodil]

📜 Description

When using the paginated catalog page, the entities are ordered only on the current page where the user is. There seems to be support for orderFields in the CatalogApi queryEntities but it's not used by the UI.

👍 Expected behavior

Ordering should be done in the backend over all entities matching the filters.

👎 Actual Behavior with Screenshots

Only the current page is ordered by the UI.

👟 Reproduction steps

1. Enable pagination in CatalogPage
2. Try to order items by column

📃 Provide the context for the Bug.

No response

🖥️ Your Environment

OS: Darwin 23.5.0 - darwin/arm64 node: v18.19.0 yarn: 1.22.22 cli: 0.26.11 (installed) backstage: 1.29.1

Dependencies: @backstage/app-defaults 1.5.8 @backstage/backend-app-api 0.8.0 @backstage/backend-common 0.21.7, 0.22.0, 0.23.3 @backstage/backend-defaults 0.3.3, 0.4.1 @backstage/backend-dev-utils 0.1.4 @backstage/backend-openapi-utils 0.1.15 @backstage/backend-plugin-api 0.6.21, 0.7.0 @backstage/backend-tasks 0.5.27 @backstage/backend-test-utils 0.4.4 @backstage/catalog-client 1.6.5 @backstage/catalog-model 1.5.0 @backstage/cli-common 0.1.14 @backstage/cli-node 0.2.7 @backstage/cli 0.26.11 @backstage/config-loader 1.8.1 @backstage/config 1.2.0 @backstage/core-app-api 1.14.0 @backstage/core-compat-api 0.2.7 @backstage/core-components 0.14.9 @backstage/core-plugin-api 1.9.3 @backstage/dev-utils 1.0.35 @backstage/errors 1.2.4 @backstage/eslint-plugin 0.1.8 @backstage/frontend-plugin-api 0.6.7 @backstage/integration-aws-node 0.1.12 @backstage/integration-react 1.1.29 @backstage/integration 1.13.0 @backstage/plugin-api-docs 0.11.7 @backstage/plugin-app-backend 0.3.71 @backstage/plugin-app-node 0.1.22 @backstage/plugin-auth-backend-module-atlassian-provider 0.2.3 @backstage/plugin-auth-backend-module-aws-alb-provider 0.1.14 @backstage/plugin-auth-backend-module-azure-easyauth-provider 0.1.5 @backstage/plugin-auth-backend-module-bitbucket-provider 0.1.5 @backstage/plugin-auth-backend-module-cloudflare-access-provider 0.1.5 @backstage/plugin-auth-backend-module-gcp-iap-provider 0.2.17 @backstage/plugin-auth-backend-module-github-provider 0.1.19 @backstage/plugin-auth-backend-module-gitlab-provider 0.1.19 @backstage/plugin-auth-backend-module-google-provider 0.1.19 @backstage/plugin-auth-backend-module-guest-provider 0.1.8 @backstage/plugin-auth-backend-module-microsoft-provider 0.1.17 @backstage/plugin-auth-backend-module-oauth2-provider 0.2.3 @backstage/plugin-auth-backend-module-oauth2-proxy-provider 0.1.15 @backstage/plugin-auth-backend-module-oidc-provider 0.2.3 @backstage/plugin-auth-backend-module-okta-provider 0.0.15 @backstage/plugin-auth-backend-module-onelogin-provider 0.1.3 @backstage/plugin-auth-backend 0.22.9 @backstage/plugin-auth-node 0.4.17 @backstage/plugin-auth-react 0.1.4 @backstage/plugin-bitbucket-cloud-common 0.2.21 @backstage/plugin-catalog-backend-module-aws 0.3.17 @backstage/plugin-catalog-backend-module-github-org 0.1.17 @backstage/plugin-catalog-backend-module-github 0.6.5 @backstage/plugin-catalog-backend-module-logs 0.0.1 @backstage/plugin-catalog-backend-module-msgraph 0.5.30 @backstage/plugin-catalog-backend-module-scaffolder-entity-model 0.1.20 @backstage/plugin-catalog-backend-module-unprocessed 0.4.9 @backstage/plugin-catalog-backend 1.24.0 @backstage/plugin-catalog-common 1.0.25 @backstage/plugin-catalog-graph 0.4.7 @backstage/plugin-catalog-node 1.12.4 @backstage/plugin-catalog-react 1.12.2 @backstage/plugin-catalog-unprocessed-entities-common 0.0.3 @backstage/plugin-catalog-unprocessed-entities 0.2.6 @backstage/plugin-catalog 1.21.1 @backstage/plugin-devtools-backend 0.3.8 @backstage/plugin-devtools-common 0.1.11 @backstage/plugin-devtools 0.1.16 @backstage/plugin-events-backend-module-github 0.2.8 @backstage/plugin-events-backend 0.3.9 @backstage/plugin-events-node 0.3.8 @backstage/plugin-home-react 0.1.15 @backstage/plugin-home 0.7.7 @backstage/plugin-kubernetes-common 0.8.1 @backstage/plugin-notifications-backend 0.3.3 @backstage/plugin-notifications-common 0.0.5 @backstage/plugin-notifications-node 0.2.3 @backstage/plugin-notifications 0.2.3 @backstage/plugin-org 0.6.27 @backstage/plugin-permission-backend 0.5.46 @backstage/plugin-permission-common 0.7.14, 0.8.0 @backstage/plugin-permission-node 0.7.32, 0.8.0 @backstage/plugin-permission-react 0.4.24 @backstage/plugin-proxy-backend 0.5.3 @backstage/plugin-scaffolder-backend-module-azure 0.1.14 @backstage/plugin-scaffolder-backend-module-bitbucket-cloud 0.1.12 @backstage/plugin-scaffolder-backend-module-bitbucket-server 0.1.12 @backstage/plugin-scaffolder-backend-module-bitbucket 0.2.12 @backstage/plugin-scaffolder-backend-module-confluence-to-markdown 0.2.23 @backstage/plugin-scaffolder-backend-module-cookiecutter 0.2.46 @backstage/plugin-scaffolder-backend-module-gerrit 0.1.14 @backstage/plugin-scaffolder-backend-module-gitea 0.1.12 @backstage/plugin-scaffolder-backend-module-github 0.4.0 @backstage/plugin-scaffolder-backend-module-gitlab 0.4.4 @backstage/plugin-scaffolder-backend-module-notifications 0.0.5 @backstage/plugin-scaffolder-backend 1.23.0 @backstage/plugin-scaffolder-common 1.5.4 @backstage/plugin-scaffolder-node-test-utils 0.1.9 @backstage/plugin-scaffolder-node 0.4.8 @backstage/plugin-scaffolder-react 1.10.0 @backstage/plugin-scaffolder 1.23.0 @backstage/plugin-search-backend-module-catalog 0.1.28 @backstage/plugin-search-backend-module-elasticsearch 1.5.3 @backstage/plugin-search-backend-module-techdocs 0.1.27 @backstage/plugin-search-backend-node 1.2.27 @backstage/plugin-search-backend 1.5.14 @backstage/plugin-search-common 1.2.13 @backstage/plugin-search-react 1.7.13 @backstage/plugin-search 1.4.14 @backstage/plugin-signals-backend 0.1.8 @backstage/plugin-signals-node 0.1.8 @backstage/plugin-signals-react 0.0.4 @backstage/plugin-signals 0.0.8 @backstage/plugin-techdocs-backend 1.10.9 @backstage/plugin-techdocs-node 1.12.8 @backstage/plugin-techdocs-react 1.2.6 @backstage/plugin-techdocs 1.10.7 @backstage/plugin-user-settings-backend 0.2.21 @backstage/plugin-user-settings-common 0.0.1 @backstage/plugin-user-settings 0.8.9 @backstage/release-manifests 0.0.11 @backstage/test-utils 1.5.8 @backstage/theme 0.5.6 @backstage/types 1.1.1 @backstage/version-bridge 1.0.8

👀 Have you spent some time to check if this bug has been raised before?

• [X] I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

• [X] I have read the Code of Conduct

Are you willing to submit PR?

None

[Rugvip]

Thank you! 👍

This is on our radar but not an immediate priority, in the meantime if anyone wants to help out towards fixing this it'll of course be appreciated 🙏

[aramissennyeydd]

Taking a look at this and the big blocker it seems is the use of client-side resolved fields, https://github.com/backstage/backstage/blob/ddfc3aac3f08e7b452091c0e29c487c09c4b488a/plugins/catalog/src/components/CatalogTable/CatalogTable.tsx#L249-L269

Since their state isn't reflected by the server, we'd need to either load everything into memory and sort or remove the usage of the resolved fields. Deprecating/removing usage seems like the better long-term option, but that's likely a huge breaking change for existing filters.

Here's my branch if anyone else wants to take a look.

[drodil]

Loading everything in the memory is not really an option as it would be the same as without pagination. Deprecating should be fine but maybe it would make sense to be able to also introduce optional sortField (or similar) to the TableColumn and send it to the backend for ordering in case it's in place?

[Rugvip]

Without having dug into this much at all, it would seem like a possible way forward is to deprecate the resolved fields, where if they're used in combination with paging you'll end up getting the broken behavior that's behind reported here. We move away from those and make sure the default set of fields are all supported when sorting via the backend.

[vinzscam]

FYI this issue is still tracked in #12247 as sorting requires some refactoring of the existing columns