serp-spider / core

:spider: The PHP SERP Spider - A search engine scraper
https://serp-spider.github.io/
Other
89 stars 44 forks source link

Featured Snippets being included in Classical ranking #32

Open dan-at-unkubed opened 6 years ago

dan-at-unkubed commented 6 years ago

I'm pretty impressed with what you have here. However, I've noticed that the new "Featured Snippets" for all search results are being included as the Rank #1 serp (via the provided getOnPagePosition method). There should instead be a custom method for determining whether a Classical result is actually a Featured Snippet (I'm currently seeing this as a Rank #0 result across the web, and the featured-snippet result is usually repeated within the top ten serp results). I just came across your library today, so I haven't really gotten into it very much, but I'll likely be trying to implement a custom parsing callback function to try and identify those featured snippets in the meantime. Hopefully this feature enhancement will rank with you. :) Thanks.

gsouf commented 6 years ago

Hi @dan-at-unkubed

Thanks for repporting, I will try to be looking into your case asap. Stay tunned

gsouf commented 6 years ago

@dan-at-unkubed

Gave a deeper look to your issue. When you say "featured snippet" are you mentioning the result that answers a question like on the following SERP? https://www.google.co.uk/search?q=how+to+take+a+screenshot

dan-at-unkubed commented 6 years ago

Yup. Elements on search pages now typically show up in this order:

1) Ads 2) Featured snippet (answers a question related to the searched-for keywords) 3) Related questions with a click-to-show "featured snippet" associated with the searches for those questions 4) Typical SERPs

The "featured snippet" is a relatively new feature and is typically a repeat of one of the top 5 SERPs (but not always) for the searched-for keyword. So they can't really count as being the top-ranking SERP, and they also have a completely different location within the html structure of the page.

On Sat, Nov 25, 2017 at 6:25 AM, Soufiane Ghzal notifications@github.com wrote:

@dan-at-unkubed https://github.com/dan-at-unkubed

Gave a deeper look to your issue. When you say "featured snippet" are you mentioning the result that answer a question like on the following SERP? https://www.google.co.uk/search?q=how+to+take+a+screenshot

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/serp-spider/core/issues/32#issuecomment-346940493, or mute the thread https://github.com/notifications/unsubscribe-auth/AJ5MajzC6C11jKrQ0TMd_U_b_ndKLqIGks5s6BVlgaJpZM4Qmrpk .

gsouf commented 6 years ago

@dan-at-unkubed I see your issue. However I think that some people are interested in parsing those results while other like you would prefer to ignore them.

The parser is modular and I think a good way to deal with it would be to allow developer to selectively choose which features to enable or not in the parser. In your case you would disable those featured snippets.

Would that solution work for you?

dan-at-unkubed commented 6 years ago

So, I found .is('classical') within the documentation, and was able to determine the classical ranking for each result by their position within the getNaturalResults() response, and that worked great for me. It might be enough for you to just add a small note to the documentation somewhere that getOnPagePosition() doesn't return a "SERP ranking" (ie.: (NOT a SERP ranking)), and is instead just a position on the page of various SERP result types.

Looking at the name of the method on this side of things, it makes sense that this would be the case, but I guess I must have read it with the wrong context in mind. Was looking for a function that returned SERP rankings of the fetched results, and that's what I saw. My bad.

I agree with you that it's best to keep the library completely agnostic with respect to the type of SERP result being displayed. So, a simple documentation update will work just fine to resolve this issue for me.

Appreciate your help on this.

gsouf commented 6 years ago

I open it back to remember to add it to the doc. Thanks for the explainations