search

serpapi / public-roadmap

Public Roadmap for SerpApi, LLC (https://serpapi.com)
50 stars 4 forks source link

No more "link" on product means finding seller's store URL costs 60x as much for 60 product queries #1849

Closed kingmeers closed 1 month ago

kingmeers commented 1 month ago

We use SerpAPI to get product links from results and analyse certain factors, and now we have to use Product API to get the direct link, for each product by id, costing 60 credits for 60 products?

I use about 100 credits a day during testing, that would now become 6000 credits for my case, I really hope I'm mistaken and have missed something,

Google shopping API does not even provide the source url for the store, just the "source" name, but Walmart/Amazon etc. have hundreds if not thousands of different "seller names" on the same domain, so we can't even use that field to deduce similarly owned products, this is a huge deal, has anyone figured out any smart ways to circumvent this?

schaferyan commented 1 month ago

As noted on https://github.com/serpapi/public-roadmap/issues/1846 since Google doesn't provide a direct link to products in the US Google Shopping layout anymore, this is unfortunately not possible without either using a different gl, or accessing the Google Product Pages. In order to keep things transparent and simple, we unfortunately need you to use the Google Product API to access those pages via our API.

Again, we're very sorry about this and we're grateful for your patience.

schaferyan commented 1 month ago

@kingmeers I've re-opened #1845, which would resolve this. Note that there is still no guarantee, but there is an open PR to add a new parameter to the Google Shopping API, which would enable one additional call on our side per Google Shopping API search to get and return the link as before.

kingmeers commented 4 weeks ago

Any update?

hilmanski commented 3 weeks ago

We're still working towards a solution. Feel free to follow this thread: https://github.com/serpapi/public-roadmap/issues/1845. We'll let you know when there's any update. Thank you very much for your patience!

marm123 commented 3 weeks ago

Following up here with a note from our Senior Engineer @tanys123:

We have made a new release that adds link to Google Shopping API. You have to enable it by passing direct_link=true. Feel free to try it out and feedback to us. image

Tagging for visibility @kingmeers

Fee free to reach out with any questions or concerns. We'll be happy to assist!

kingmeers commented 3 weeks ago

Fee free to reach out with any questions or concerns. We'll be happy to assist!

That's awesome! Does it cost any extra credits per call? Also, is this essentially the original response exactly, just now requiring the direct_link=true passed?

NateSkiles commented 3 weeks ago

@kingmeers

Does it cost any extra credits per call?

No, we are committed to maintaining the cost of a single search to one search credit. Adding direct_link=true will not cost any additional credits.

Also, is this essentially the original response exactly, just now requiring the direct_link=true passed?

The direct link returned is the same as the original response, but we have added some additional links Google includes in the response.

sonika-serpapi commented 3 weeks ago

@kingmeers While it doesn't cost any extra credits, I want to also mention that the API calls with the direct_link parameter might take longer than without. This is because of the different approaches we have to take to extract the link and links values. This, unfortunately, puts a heavier load on our backend, which might result in a decreased performance in terms of response times. I'm very sorry in advance if that's the case, and I hope you understand.

kingmeers commented 3 weeks ago

The direct link returned is the same as the original response, but we have added some additional links Google includes in the response.

This is great, is this because google added back the product links or because of a workaround built by your team? Basically, are you technically calling the product api every time i call an endpoint with direct_link=true ?

sonika-serpapi commented 3 weeks ago

This is great, is this because google added back the product links or because of a workaround built by your team? Basically, are you technically calling the product api every time i call an endpoint with direct_link=true ?

@kingmeers We are recovering these links using another one of Google's endpoints, which requires an additional request from us. Google has not restored the the product links on their end for the google shopping page.