Code freezes when running `OpenseaScraper.rankings()`

[SKreutz]

If I try to run the script on Linux (computer in the same network) it stops right here without any errors like it's frozen. It works perfectly fine on Mac and it's the same code. It worked a week ago and nothing was changed.

✅ === OpenseaScraper.rankings() === === OpenseaScraper.rankings() === ...fetching 1 pages (= top 100 collections) ...opening url: https://opensea.io/rankings?sortBy=one_day_volume ...🚧 waiting for cloudflare to resolve ...exposing helper functions through script tag ...scrolling to bottom and fetching collections. -> freezes here <-

There are no errors. It seems like its stuck in an infinite loop

---

Edit: After a lot of investigation i finally found what causes the freezing:

If I change the url in the rankings.js from "https://opensea.io/rankings?sortBy=total_volume" to "https://opensea.io/rankings?sortBy=one_day_volume" the script freezes, it works perfectly with the total_volume though. Seven_day_volume does also cause freezing. It is weird because it also worked with one_day_volume a few days ago

[dcts]

Thanks for the report, appreciate it!

You can always turn on debug mode and then you can actually watch the browser and see the scraping process. To turn on:

// options
const options = {
  debug: true,
  logs: true,
  sort: true,
  browserInstance: undefined,
}

const nbrOfPages = 2;
const ranking = await OpenseaScraper.rankings(nbrOfPages, options);

I run rankings() on a linux but cannot reproduce the bug. It might have something to do with screensize (smaller or larger screen might show another DOM due to responsive webdesign).

Can you rerun the function with debug turned on and report what you see actually happens?

[SKreutz]

Thank you for answering I appreciate your help.

When debugging with total volume it opens the browser and waits for the collections to load. It then scrolls down to the bottom, closes the browser and returns the slugs.

When debugging with 24h volume on the same screen size it does the same but always stops scrolling at about position 50 - 57 (not consistent). After a few minutes it keeps scrolling a tiny but further to the next position and stops again. After another 3 - 5 Minutes it again scrolls one or "one and a half" positions further. It maybe could go through after about 2 hours or so.

Whats also kinda funny, if I give it a small "nudge" and scroll down myself a little bit it it sometimes finishes and returns all slugs correctly (also not consistent)

[dcts]

this is really weird behaviour, will look into the code if I can find something that might cause this. Will let you know if I find something!

[SKreutz]

this is really weird behaviour, will look into the code if I can find something that might cause this. Will let you know if I find something!

Thank you, let me know if you need further information

[dcts]

moved to #36

[dcts]

Scrape top100 with this simple script

const nextDataStr = document.getElementById("__NEXT_DATA__").innerText;
const nextData = JSON.parse(nextDataStr);
const top100 = nextData.props.relayCache[0][1].json.data.rankings.edges.map(obj => obj.node);