bartervg / barter.vg

Track and hold discussion on Barter.vg bugs, enhancements, and other issues
https://barter.vg
MIT License
20 stars 4 forks source link

Is the Barter.vg offering any API for data like the tradables and wishlists of a game? #195

Closed AxJ11 closed 1 year ago

AxJ11 commented 3 years ago

As title, it would be great because we can use some!

antigravities commented 3 years ago

Yes, use https://barter.vg/i/[itemid]/json, i.e. https://barter.vg/i/19210/json

Fetch info for a Steam AppID without translating to a Barter ID using https://barter.vg/steam/app/[appid]/json, i.e. https://barter.vg/steam/app/439720/json

AxJ11 commented 3 years ago

Whoa!!!!! Awesome!!!! I will look into it!!! Thanks!

And if you don't mind, may I ask how can I fetch the JSON data in Javascript for my current user script in Tampermonkey? I use these codes down below for testing the "Virtua Tennis 4" but it shows "Uncaught (in promise) TypeError: Failed to fetch" in Chrome: fetch('https://barter.vg/i/2590/json').then(function(response) { return response.json(); }).then(function(myJson) { console.log(myJson); });

antigravities commented 3 years ago

Not sure what the issue is there, need more info. Make sure your Tampermonkey script is allowed to send HTTP requests across origins. Maybe #devbiz in the Discord guild is a better place to discuss this?

AxJ11 commented 3 years ago

Make sure your Tampermonkey script is allowed to send HTTP requests across origins.

How and where can I check/correct that?

Maybe #devbiz in the Discord guild is a better place to discuss this?

Yes, indeed. I'll head over there for further questions if I still can't figure it out myself. Than you!!

AxJ11 commented 3 years ago

I think you're right. My problem should be on the Tampermonkey for the permission to send HTTP requests across origins. I tested the same codes on the local HTML file and they work successfully.

AxJ11 commented 3 years ago

Oh geeezy, I'm really stuck at the "GM_xmlhttpRequest" thingy.

AxJ11 commented 3 years ago

I think I find a solution for that.

Luckz commented 3 years ago

Oh geeezy, I'm really stuck at the "GM_xmlhttpRequest" thingy.

That is indeed the design.

Depending on what your userscript does (you didn't say), it might be more appropriate to cache tradables/wishlist count for all games once every day or two — in case it's a private modification of SWI, for example. (That's different API endpoints then.)

AxJ11 commented 3 years ago

@Luckz Thanks for replying to this but this thread is for my own script to the Barter.vg. I want to fetch the games' data through the API to show the tradable/wishes counts on the pages of the "searching tradable/wishlist matches" and "offers/edit offer" all the time. I had finished the "bundle page" part though. It's a bit easier because it doesn't need the API to show those counts.

But I do need the SWI to add a green/blue background for owned/wishlisted games. That's the next improvement for my script on the bundle page.

AxJ11 commented 3 years ago

Hi, there! I successfully get the JSON data by the "GM_xmlhttpRequest" finally. However, I found some of the data are not updated, for example, the counts of the "tradable" and "wishlist." The numbers would likely be more or less by 1 to 10 compared to the actual page of the item. Is it normal? How can I get the same numbers on the page of the item? Is it possible I save the numbers on the page of the item and then pass them to the page I want?

Revadike commented 3 years ago

I'm not sure if it's a cache problem, but if it is, you can try adding something like "?_=" + Date.now() to the URL.

AxJ11 commented 3 years ago

Do you mean like this?

https://barter.vg/i/xxxx/json?+=Date.now()

Revadike commented 3 years ago

no, https://barter.vg/i/xxxx/json?_=1619962490148

AxJ11 commented 3 years ago

Thanks but It seems like not the cache problem. Too bad. 😥

Revadike commented 3 years ago

Thanks but It seems like not the cache problem. Too bad. 😥

Are you using Date.now() to calculate 1619962490148? Otherwise it may also be a serverside cache issue, which means you can't do anything about it.

AxJ11 commented 3 years ago

Yes, the codes look like this:

let item_href = xxx.getAttribute('href') + 'json?_=' + Date.now()

And the "console.log(itemhref)" do show like this: ` https://barter.vg/i/91361/json?=1619974003769 https://barter.vg/i/84156/json?_=1619974003770 https://barter.vg/i/77394/json?_=1619974003771 https://barter.vg/i/47307/json?_=1619974003772 https://barter.vg/i/36194/json?_=1619974003773 https://barter.vg/i/35303/json?_=1619974003774 https://barter.vg/i/109338/json?_=1619974003774 https://barter.vg/i/24943/json?_=1619974003775 .... `

But most of the numbers still don't match up. Maybe they design it that way. Too bad.

bartervg commented 3 years ago

I'm not sure if it's a cache problem, but if it is, you can try adding something like "?_=" + Date.now() to the URL.

I believe that Cloudflare will ignore this and anything to the right of the question mark. However, the header cache-control is set to private, max-age=0 so the page shouldn't be cached. The collection counts (tradable, wishlist, library) are cached every 30 minutes. Only signed-in users will see the live counts when viewing the profile pages.

These counts aren't that computationally intensive, but in the past, there were problems with search crawlers requesting every page in rapid succession. There didn't seem to be any benefit of bots getting the live vs. 30-minute-old counts, so I removed the live list from every profile page is not signed in to avoid a site slowdown during each bot crawl.

If it's important to have the live counts instead of the cached counts, it may be more efficient to develop a feed for that purpose.

AxJ11 commented 3 years ago

...it may be more efficient to develop a feed for that purpose.

What does "to develop a feed" mean? How and where should I start from?

bartervg commented 1 year ago

The numbers would likely be more or less by 1 to 10 compared to the actual page of the item.

There are two sets of counts: cached and live. I believe every page except the item profile page uses cached counts.

What does "to develop a feed" mean? How and where should I start from?

I would need to create a new feed or modify an existing one to add the live data. Perhaps there's a reason for needing the live data, but I'm going to close this under the assumption that the cached counts are good enough.