Back-end dependencies for the new Backpack V2

[sandrade-dcl]

---

layout: adr adr: 206 title: Back-end dependencies for the new Backpack V2 date: 2023-04-04 status: Review type: ADR spdx-license: CC0-1.0 authors:

• sandrade-dcl
• lorux0

• davidejensen

Need

This is a technical proposal for the needs described here: PRD: Backpack V2

The intention of this document is:

• Identifying which teams will be involved and what responsibilities they’ll cover.
• Dividing the new functionality in smaller tasks that will be estimated accordingly in order to create a development roadmap.
• Finding possible corner cases in order to preemptively identify and solve them.
• Reaching a consensus between different affected teams in terms of defining the different functionalities and flows.

Involved teams

This initiative will require a cross team effort that will include the following teams:

• Explorer: for the UI, UI functionalities and integration with the back-end side.
• Platform: for the modification of the already existing lambdas endpoints in order to make them compatible with the new requirements (sorting wearables by different criteria, filtering them by different criteria, etc.) and/or the creation of new ones required.

Back-end dependencies

We currently have several lambdas endpoints (some of them compatible with pagination) that are already being used by the current version of the Backpack to retrieve the information about the wearables. These endpoints are:

• Base Wearables: /lambdas/collections/wearables?collectionId=urn:decentraland:off-chain:base-avatars
• Owned Wearables: /lambdas/users/:userId/wearables?includeDefinitions=true
• Owned Emotes: /lambdas/users/:userId/emotes?includeDefinitions=true
• Third Party Wearables by collection: /lambdas/users/:userId/third-party-wearables/:collectionId&includeDefinitions=true

We need to support the features in the lambdas. It is recommended to create new ones, to keep retro-compatibility, but we leave the decision to the @platform team.

Filter WEARABLES by collection

In order to give the user the possibility of seeing and managing any type of wearables (BASE, OWNED and THIRD PARTY) together in the backpack, we would need to receive all the information from a single endpoint instead of having to request them separately.

These will be the filters available in the UI to filter the wearables:

Show BASE WEARABLES ON/OFF

Select 1 or more COLLECTIONS ("decentraland" would be the owned wearables and any other would belong to TPW)

In order to be able to manage these filters, we will need an endpoint where we can specify a list of collection ids (either "Decentraland", "urn:decentraland:off-chain:base-avatars" or any third party collection) and return the list of wearables that match with those collections.

Examples:

Only owned collection

/lambdas/users/:userId/wearables?includeDefinitions=true&collectionIds=decentraland

Only base collection

/lambdas/users/:userId/wearables?includeDefinitions=true&collectionIds=urn:decentraland:off-chain:base-avatars

Several third party collections

/lambdas/users/:userId/wearables?includeDefinitions=true&collectionIds=urn:decentraland:matic:collections-thirdparty:cryptoavatars,urn:decentraland:matic:collections-thirdparty:kollectiff,...

Any combination

/lambdas/users/:userId/wearables?includeDefinitions=true&collectionIds=decentraland,urn:decentraland:off-chain:base-avatars,urn:decentraland:matic:collections-thirdparty:cryptoavatars,urn:decentraland:matic:collections-thirdparty:kollectiff,...

Filter EMOTES by collection

In order to give the user the possibility of seeing and managing any type of emotes (BASE or OWNED) together in the backpack, we would need to receive all the information from a single endpoint instead of having to request them separately.

This will be the filter available in the UI to filter the emotes:

Show BASE EMOTES ON/OFF

In order to be able to manage these filters, we will need an endpoint where we can specify a list of collection ids (either "decentraland" or "base") and return the list of emotes that match with those collections.

Examples:

Only owned collection

/lambdas/users/:userId/emotes?includeDefinitions=true&collectionIds=decentraland

Only base collection

/lambdas/users/:userId/emotes?includeDefinitions=true&collectionIds=base

Any combination

/lambdas/users/:userId/emotes?includeDefinitions=true&collectionIds=decentraland,base

Filter WEARABLES/EMOTES by text

The user will be able to filter the wearables/emotes by text.

In order to be able to do it, we will need to add a new parameter to the existing endpoints that will be used to specify a text and return the list of wearables that match with that text. The new parameter will be called textFilter and will be a string that will accept any value.

In order to be able to manage this filter, we will need an endpoint (the same described in the previous sections) where we can specify a text and return the list of wearables that match with that text.

Examples:

Filter any combination of wearables by text

/lambdas/users/:userId/wearables?includeDefinitions=true&collectionIds=decentraland,urn:decentraland:off-chain:base-avatars,urn:decentraland:matic:collections-thirdparty:cryptoavatars,urn:decentraland:matic:collections-thirdparty:kollectiff&textFilter=aviator

Filter any combination of emotes by text

/lambdas/users/:userId/wearables?includeDefinitions=true&collectionIds=decentraland,base&textFilter=chic

Filter WEARABLES by category

The user will be able to filter the wearables by category.

In order to be able to manage this filter, we will need an endpoint (the same described in the previous sections) where we can specify the category and return the list of wearables that match with that category.

Examples:

Filter any combination of wearables by category

/lambdas/users/:userId/wearables?includeDefinitions=true&collectionIds=decentraland,urn:decentraland:off-chain:base-avatars,urn:decentraland:matic:collections-thirdparty:cryptoavatars,urn:decentraland:matic:collections-thirdparty:kollectiff&category=hat

Sort WEARABLES/EMOTES

The user will be able to sort the wearables/emotes by different criteria: newest/oldest, rarest/less rare or name A-Z/name Z-A.

In order to be able to manage this sorting, we will need an endpoint (the same described in the previous sections) where we can specify the sorting criteria and return the list of wearables sorted by that criteria. The sorting parameter will be a string that will accept the following values:

• newest
• oldest
• rarest
• less_rare
• name_a_z
• name_z_a

In cases where wearables or emotes dont have minted timestamps, they should be added at the bottom of the list.

Examples:

Sort any combination of wearables by rarest

/lambdas/users/:userId/wearables?includeDefinitions=true&collectionIds=decentraland,urn:decentraland:off-chain:base-avatars,urn:decentraland:matic:collections-thirdparty:cryptoavatars,urn:decentraland:matic:collections-thirdparty:kollectiff&sort=rarest

Filter any combination of emotes by name A-Z

/lambdas/users/:userId/wearables?includeDefinitions=true&collectionIds=decentraland,base&sort=name_a_z

Get wearable market details (NEW ENDPOINT)

The user will be able to click on a wearable and open a modal with some details as:

• Creator (optional): Name and image of the creator of the wearable.
• Collection (optional): Name and image of the collection where the wearable belongs to.
• Network: Network where the wearable is deployed.
• Contract address: Address of the contract where the wearable is deployed.
• Latest sales: List of the latest sales of the wearable. It would be a list of the following information:
  • From: Name and image of the seller.
  • To: Name and image of the buyer.
  • Type: Type of the transaction (Transfered or Minted).
  • When: Date of the transaction.
  • Price: Price of the transaction.
• Minting number and total scarcity (optional): Minted number + total amount of NFTs of this wearable.

To be able to request this information, the client will have to use the minted id instead of the generic id. This information is available when querying the wearables. In order to be able to get this info, we will need to create a new endpoint that will return the details of a specific wearable:

Example:

Get market details of a specific wearable

/lambdas/wearables/:wearableId/market-details

Get a random list of equipped wearables (NEW ENDPOINT)

The user will be able to click on a [RANDOMIZE] button and equip a random list of wearables on his avatar.

In order to be able to do it, we will need to create a new endpoint that will return a random list of wearable ids from the owned wearables of the user. The wearables must be from each category, avoiding conflicts with hides and replaces whenever possible.

Example:

Get random wearables of a specific user

/lambdas/users/:userId/random-wearables

Outfits

The outfit information should be stored in the catalyst because we need to support multi-platform.

~~On the other hand, we should avoid storing it in profiles since it is information that only each individual user needs and does not need to be shared with other users. Adding it to the profiles would add an additional data transfer for each request unnecessarily.~~

Update: The outfit information will be stored in the user profile.

Regarding to the outfit image previews, they can be generated in the client when saving the outfit. There is no need to store the images in the catalysts.

We may encounter the need of checking that the wearables in the outfit are valid for the current user. (What happens if the user does not have the wearable anymore?)

Equip/save wearables

We though about several alternatives to avoid performance issues by saving the profile every time you equip an item:

1. Equip all the items and only when the backpack is closed, make the request to save the profile.
2. The profile will be updated/saved in intervals of 20 seconds (to be defined) or whenever the user closes the backpack.
3. Save the profile when the user equips a wearable.

Saving the profile would mean that the nearby users will be able to see the changes as soon as possible.

Update: we will go for option 1, taking in consideration that we cant save the profile during the quitting of the application. It will only be saved when closing the backpack, just like the "save" button worked before.

Emotes pagination

We will need to refactor the current emotes catalog, the same way on how we did on wearables catalog, so its supports pagination. We will also reduce kernel responsibilities, moving all (or most) of the emote handling in the renderer. No need backend support. Open ticket: #4429

BackpackEditorV2

In Unity we are going to implement a new UI for the backpack v2 requirements, including all the components needed.

We will keep retro-compatibility with the old one, including the UI, service calls and dependencies.

Open questions

1. We need to define an approach on how to request/store the outfits information for every user (ref: outfits). Its going to be stored in the user profile.
2. We need to discuss and define an approach on when we should save the profile (ref: equip/save wearables). Option 1.
3. Is the preview image generation for outfits going to be a suitable approach? (ref: outfits). Yes, it will be done in the client.
4. In order to query market details for every wearable, we need to solve the specific minted wearableId (now the client uses the generic one). In case the user has many wearables of the same generic id, how are we going to display the information?
5. The randomizer will need a bounded scope because querying the whole wearables that the user may have would be too expensive. Is it ok that the randomized result is smaller?
6. Is it necessary that the nearby users see the avatar changes as soon as you equip a wearable in the backpack? Taking in consideration that we will go for option 1, the nearby users will only see the avatar changes when closing the backpack, very similar on how it works today.

[mikhail-dcl]

In order to be able to do it, we will need to add a new parameter to the existing Owned Wearables endpoint that will be used to specify the collection ids, either Decentraland, 'urn:decentraland:off-chain:base-avatars' or any third party collection, and return the list of wearables that match with that collections. The new parameter will be called collectionId and will be a string that will accept any value. We should be able to specify more than one collection id separated by commas.

Endpoints affected: Owned Wearables: /lambdas/users/:userId/wearables?includeDefinitions=true&collectionId=:collectionId1,:collectionId2...

It is not clear from the design why we need to include several collections. Also, it may blow up as the length of the URL may exceed the limit. In that case I would recommend to use a body instead

[sandrade-dcl]

It is not clear from the design why we need to include several collections.

According to the design of the new Backpack, we can load in the same grid and at the same time different collections of wearables. For example: Base wearables (swiching off the ONLY COLLECTIBLES toogle) + Owned wearables (selecting Decentraland from the COLLECTION dropdown) + one or more Third Party wearables (selecting one or more TPW collections from the COLLECTION dropdown). In a situation like this, we're going to need to be able to apply pagination/sorting/searching on all of these collections together, so we would need to be able to request them together.

Also, it may blow up as the length of the URL may exceed the limit. In that case I would recommend to use a body instead

Yes, it would be a good solution for this case where we would have to send a list of ids.

[mikhail-dcl]

Considering that we introduce more dimensions we should be very smart with pagination and caching. I see the following potential risks:

1. If we treat every combination of dimensions separately we may blow up the network bandwidth
2. We may retrieve data copies when some of the wearables are already cached (but under different keys/dimensions) that just increase network pressure for nothing
3. We may store inefficiently on the client side with all these copies which will lead to memory pressure instead
4. Off the top of my head it's not clear how to introduce such a smart caching system to prevent network bandwidth from exploding

I would consider cooperating with the "platform" team on this problem as they should have some sort of caching/grouping or other mechanisms that allow fetching wearables faster. Maybe we should revise endpoints to select excessive wearables upfront taking into consideration other dimensions.

Have you thought about it?

[aixaCode]

Few more comments from my side:

1. I see that there are few open questions in the ARD to be answered. I would suggest to write them down in separate section at the end of document to give more visibility for them

2. Regarding sorting, based on the description It's not clear for me if we need to use endpoint for that, or can it be done one the client side to avoid extra calls?

[lorux0]

Considering that we introduce more dimensions we should be very smart with pagination and caching. I see the following potential risks:

1. If we treat every combination of dimensions separately we may blow up the network bandwidth
2. We may retrieve data copies when some of the wearables are already cached (but under different keys/dimensions) that just increase network pressure for nothing
3. We may store inefficiently on the client side with all these copies which will lead to memory pressure instead
4. Off the top of my head it's not clear how to introduce such a smart caching system to prevent network bandwidth from exploding

I would consider cooperating with the "platform" team on this problem as they should have some sort of caching/grouping or other mechanisms that allow fetching wearables faster. Maybe we should revise endpoints to select excessive wearables upfront taking into consideration other dimensions.

Have you thought about it?

1. We are requesting wearables/emotes via pagination. In the current backpack we are requesting everything at once, so its a worst scenario than this proposal.
2. We should not apply a cache for these requests in the lambdas. We have many filter params and thus many different results for each request.
3. In the client, wearables & emotes should be always cached by id. I think we should improve the caching system by properly managing the usage references over the entries. One idea is to periodically add and subtract usage references over any cached wearable/emote. Anyway, the current context of the backpack is the worst case scenario of memory usage, since we are requesting all the wearables and emotes at once. I think this improvement could be out of scope of this proposal.
4. In the client, if we improve the caching system by clearing wearables/emotes not being used after some time, we should not have major memory problems. In the lambdas, dont use cache. I think we should not over-think a solution right now until we get some results.

[lorux0]

Few more comments from my side:

1. I see that there are few open questions in the ARD to be answered. I would suggest to write them down in separate section at the end of document to give more visibility for them
2. Regarding sorting, based on the description It's not clear for me if we need to use endpoint for that, or can it be done one the client side to avoid extra calls?

1. Will do!
2. We cannot do the sorting in the client, because everything is paginated. The client doesn't have all the information to do a correct sorting.