search

serpapi / public-roadmap

Public Roadmap for SerpApi, LLC (https://serpapi.com)
57 stars 6 forks source link

[Google Shopping API] All filters options don't appear #2043

Open sonika-serpapi opened 1 month ago

sonika-serpapi commented 1 month ago

A customer reported that only the filter options which are shown on the google shopping page. More options are shown by clicking on "See more", which we don't currently return in the response.

Currently shown:

image (4)

image (5)

When expanded: [Not currently shown in the response]

image (6)

Public links: Playground

Internal links: Front | Search Inspect 1

schaferyan commented 1 month ago

Thank you @sonika-serpapi. I think this is a feature request rather than a bug. We only scrape what appears directly on the page by default. If it requires user action (such as clicking "See more") to expand something when viewing the page in the browser, we may or may not be able to support it with SerpApi.

sonika-serpapi commented 1 month ago

Sounds good, thank you @schaferyan!

Ovi commented 1 month ago

The Google Shopping API indeed makes additional request to retrieve more items, as @schaferyan predicted.

https://www.google.com/async/wpr_lhs https://www.google.com/async/wpr_lhs?vet=12ahUKEwjdisq_gJiJAxVgk1YBHWAnHEYQ7coMegQIARAD..i&ei=VV4SZ52MOOCm2roP4M7wsAQ&opi=95576897&udm=28&gl=us&hl=en&uule=w%20CAIQICINVW5pdGVkIFN0YXRlcw&yv=3&cs=0&async=query:patio%20furniture%20at%20Shop.com,_basejs:%2Fxjs%2F_%2Fjs%2Fk%3Dxjs.s.en_US.oO8IWFQoyDs.2018.O%2Fam%3DAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAIoIBAAAAAAABQAAQAAAAAAAAAAAAAAAAAAIAAAQAICQEACAAAAACABQCACBAQAAAIAAABAQAAASKAfv85AAAAAAAAAAAAAAQAEwAAAAAA4AIAgAA0BAAAAAgAAMAAAABAAAABAAAABQAAAAIAAAAAAAAAAAAAAAAAACCAANAPAAAAAAAAAAAAAAAAAAAAABqgAIAAfgAAAAAAAQAAAEEAAOCADEAAAAAAAAAA7gOA5wHDIYUFAAAAAAAAAAAAAACAACQI5oD0FwSAAAAAAAAAAAAAAAAAAAAgJWjicgMACQ%2Fdg%3D0%2Fbr%3D1%2Frs%3DACT90oHrP7jB5TjWJpgIgScgKGEpFpconA,_basecss:%2Fxjs%2F_%2Fss%2Fk%3Dxjs.s.aKzrRXdHj4M.L.B1.O%2Fam%3DQOoQIAQAAAQAEdAAAgAAAAAAAAAAAAAAAAAAAAAkAAAABAAAAAAAIAAAIgAAZRIAAJAQAARgAwACAAAfAMABAYABAAAAAAAACQAAAAAAgAAAHQIAAAAAAAAgAAAQQQAAQCEAAAAwEAAAAAMAAAAAIAQIAEAAAyAEAAqQgAA0BKAfhQAAAMAEAABCQBjBMABBBQADOAUAAAAAIAEDAAAQAIQAAACAgAIAIEAAAAiBADAAACQiAAAQABCgCAAEAABgAAAABBAIAICYCeCADEAAAAAAAAAAIAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFAAAAAAAAAAAAAAAAAAAAAAABA%2Fbr%3D1%2Frs%3DACT90oGDyXCgaHuosAR0dl0NaogRpP48-g,_basecomb:%2Fxjs%2F_%2Fjs%2Fk%3Dxjs.s.en_US.oO8IWFQoyDs.2018.O%2Fck%3Dxjs.s.aKzrRXdHj4M.L.B1.O%2Fam%3DQOoQIAQAAAQAEdAAAgAAAAAAAAAAAAAAAAAAAAAkAAAABAAAAAAAIAAIooBAZRIAAJBQAARgAwACAAAfAMABAYABAIAAAQAICQEACAAAgACAHQKACBAQAAAoAAARQQAAQSOAfv85EAAAAAMAAAAAIAQIE0AAAyAE4AqQgAA0BKAfhQgAAMAEAABCQBjBMABBBQADOAcAAAAAIAEDAAAQAIQAACCAgNIPIEAAAAiBADAAACQiAAAQABqgCIAEfgBgAAAABRAIAMGYCeCADEAAAAAAAAAA7gOA5wHDIYUFAAAAAAAAAAAAAACAACQI5oD0FwSAAAAAAAAAAAAAAAAAAAAgJWjicgMACQ%2Fd%3D1%2Fed%3D1%2Fdg%3D0%2Fbr%3D1%2Fujg%3D1%2Frs%3DACT90oF9HnZsb72XYK5sW9VsWLFMszCoAw,_fmt:prog,_id:_VV4SZ52MOOCm2roP4M7wsAQ_28
sonika-serpapi commented 1 month ago

The Google Shopping API indeed makes additional request to retrieve more items, as @schaferyan predicted.

https://www.google.com/async/wpr_lhs

Thank you for looking into this @Ovi. Are we able to implement this at all? Or will this require an additional parameter or an additional API call or something similar?

Ovi commented 1 month ago

Thank you for looking into this @Ovi. Are we able to implement this at all? Or will this require an additional parameter or an additional API call or something similar?

As far as I know, we currently do not support this. @aciddjus @zyc9012 @tanys123 can you please answer Sonika's query?

Similar issues: https://github.com/serpapi/public-roadmap/issues/1520, https://github.com/serpapi/public-roadmap/issues/675

tanys123 commented 1 month ago

Yes, as @Ovi pointed out, it is likely we have to create new engine for it. Let's monitor for some time if more customers request this.