How can we reduce loading time for VizCartographerJourney data?

[dhruv10]

To plot the journey of a user (cartographer + challenger) we need to fetch all of their points - listed points, challenging points, and others. We need to capture every point's name, geohash, deposit, createdAt, and status.

An official FOAM API (example) gives these details only for the listed points.

Now, to fetch these details for other points we are following a quiet long process (replicating working of https://foam.tools) which is described as follows:

1. Fetching a user's all points they have interacted with using blocklytics API (example)
2. Filtering out challenged points and storing it an array
3. Converting challenge_id into poll_id for each of them
4. Getting listingHash from poll_id using another FOAM's API (example) for every point in the array.
5. Then again calling another FOAM API (example) that gives us the required data for a cartographer using listinghash

This is a really long process and takes time even when we are parallelly making requests for unrelated data. Can we reduce this process somehow?

[calebsheridan]

@dhruv10 Will it help if Step 1 returns a value for listing_hash on challenged points (instead of null)?

[dhruv10]

@dhruv10 Will it help if Step 1 returns a value for listing_hash on challenged points (instead of null)?

Yes @calebsheridan this would be very helpful since it would reduce many steps. We would like to run a pilot whenever you're ready with the new API.

[dhruv10]

We fetch every point's name, geohash, deposit, createdAt, and status.

On Fri, Jan 31, 2020 at 1:21 AM Caleb Sheridan notifications@github.com wrote:

@dhruv10 https://github.com/dhruv10 what data do you fetch in Step 5?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/thevantageproject/foamviz/issues/92?email_source=notifications&email_token=AHYC5KXZ6PEBJPLFTDZVH7TRAMVV7A5CNFSM4KL65XRKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKMJK7Y#issuecomment-580425087, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHYC5KXHSTCN5X2IS3TW55TRAMVV7ANCNFSM4KL65XRA .

--

Dhruv Bhatnagar [image: https://]about.me/dhruvb https://about.me/dhruvb?promo=email_sig&utm_source=product&utm_medium=email_sig&utm_campaign=edit_panel&utm_content=plaintext

[calebsheridan]

Hi @dhruv10 we've updated the return object to include listing hash. Please check that it makes sense for your needs.

[dhruv10]

Hey, @calebsheridan thanks for the updated API. It's working as expected. We were able to reduce a little (but significant) loading time.

[dhruv10]

Merged in #103

[dhruv10]

The longest fetching time is for API in step 5 (4s to 6s)

5. Then again calling another FOAM API (example) that gives us the required data for a cartographer using listinghash

Try step-5-API here

Need to think of how can this be reduced somehow.

[dhruv10]

106