dcts / opensea-scraper

Scrapes nft floor prices and additional information from opensea. Used for https://nftfloorprice.info
MIT License
184 stars 73 forks source link

Small updates, see comments below #26

Closed robertistok closed 2 years ago

robertistok commented 2 years ago

Take this url for example.

You could see, that not all NFTs are listed and that breaks the function. This would fix it.

A workaround is to supply a URL with the buy_now filter enabled, like in this url.

robertistok commented 2 years ago

I added commit#e15fb42 to this PR as well.

I am building a listing bot, and would need to have the displayImageUrl returned in the /offers endpoint as well. I think this is a useful feature to have

robertistok commented 2 years ago

Added another commit regarding using the sort, see #2fc3143

dcts commented 2 years ago

hey @robertistok, great changes! Am reviewing...

Just one question, you provided two times the same link in your first comment, did you mean: url1 with no "buy_now" filter enabled: https://opensea.io/collection/boredapeyachtclub?search[sortAscending]=true&search[sortBy]=PRICE&search[stringTraits][0][name]=Background&search[stringTraits][0][values][0]=Purple&search[stringTraits][1][name]=Earring&search[stringTraits][1][values][0]=Silver%20Hoop&search[stringTraits][2][name]=Eyes&search[stringTraits][2][values][0]=Bloodshot

url2 with "buy_now" filter enabled: https://opensea.io/collection/boredapeyachtclub?search[sortAscending]=true&search[sortBy]=PRICE&search[stringTraits][0][name]=Background&search[stringTraits][0][values][0]=Purple&search[stringTraits][1][name]=Earring&search[stringTraits][1][values][0]=Silver%20Hoop&search[stringTraits][2][name]=Eyes&search[stringTraits][2][values][0]=Bloodshot&search[toggles][0]=BUY_NOW

I can confirm that url1 breaks the offersByUrl function, but it still is broken in your update, or am I missing something? 🤔

But the improvements are great, so will merge, but can you check if the url1 bug also persists in your update? Just to be sure its not some weird behavior on my machine.

To fix the bug I would suggest to simply make the "BUY_NOW" mandatory and append it to the URL if it is missing:

const mandatoryQueryParam = "search[toggles][0]=BUY_NOW";
if (!url.includes(mandatoryQueryParam)) {
  const joinChar = url.includes("?") ? "&" : "?";
  url += `${joinChar}${mandatoryQueryParam}`;
}
robertistok commented 2 years ago

Thanks for merging!

I think it makes sense to have my solution, as that removes the burden from the user. I can also use this function to just simply get a list of NFTs with a specific trait. Not all of them will be on the market, so it's good to take care of that case