The chainfollower took multiple days to sync the assets. This wasn't practically since the time will likely increase in the future. Additionally it was needed to set the starting point of the chainfollower with two environment variables, which was just seen as a work around.
The problem were:
The chainfollower got stuck - that's why the custom starting point was needed
Saving the assets were very slow
This hindered users to use the background easily without days of syncing.
Proposed Solution
We are saving assets in a local cache and save them batch wise. Additionally I adjusted the insertAssets function to skip already existing assets. Through this solution we are able to remove the check hasAsset for each asset (which takes a few seconds for every asset)
Additionally catching errors from the Chainfollower and continuing the chainfollower.
This solution increases the speed on mainnet to 10% Assets per hour + removes the need of the custom starting point.
Context
The chainfollower took multiple days to sync the assets. This wasn't practically since the time will likely increase in the future. Additionally it was needed to set the starting point of the chainfollower with two environment variables, which was just seen as a work around. The problem were:
This hindered users to use the background easily without days of syncing.
Proposed Solution
We are saving assets in a local cache and save them batch wise. Additionally I adjusted the
insertAssetsfunction to skip already existing assets. Through this solution we are able to remove the checkhasAssetfor each asset (which takes a few seconds for every asset) Additionally catching errors from the Chainfollower and continuing the chainfollower.This solution increases the speed on mainnet to 10% Assets per hour + removes the need of the custom starting point.