serp-spider / search-engine-google

:spider: Google client for SERPS
https://serp-spider.github.io
Other
168 stars 61 forks source link

DOM Change? #77

Open thebennos opened 7 years ago

thebennos commented 7 years ago

INVALID DOM! 4.10.2017 11:41:23Fatal error: Uncaught Serps\SearchEngine\Google\Exception\InvalidDOMException: Unable to check javascript status. Google DOM has possibly changed and an update may be required. in /application/code/vendor/serps/search-engine-google/src/Page/GoogleSerp.php:116 4.10.2017 11:41:23Stack trace: 4.10.2017 11:41:23#0 /application/code/vendor/serps/search-engine-google/src/Page/GoogleSerp.php(60): Serps\SearchEngine\Google\Page\GoogleSerp->javascriptIsEvaluated() 4.10.2017 11:41:23#1 /application/code/rankings/ranking_checker_v2_intern.php(314): Serps\SearchEngine\Google\Page\GoogleSerp->getAdwordsResults() 4.10.2017 11:41:23#2 [internal function]: {closure}(Object(PhpAmqpLib\Message\AMQPMessage)) 4.10.2017 11:41:23#3 /application/code/vendor/php-amqplib/php-amqplib/PhpAmqpLib/Channel/AMQPChannel.php(983): call_user_func(Object(Closure), Object(PhpAmqpLib\Message\AMQPMessage)) 4.10.2017 11:41:23#4 [internal function]: PhpAmqpLib\Channel\AMQPChannel->basic_deliver(Object(PhpAmqpLib\Wire\AMQPReader), Object(PhpAmqpLib\Message\AMQPMessage)) 4.10.2017 11:41:23#5 /application/code/vendor/php-amqplib/php-amqplib/PhpAmqpLib/Channel/AbstractChannel.p in /application/code/vendor/serps/search-engine-google/src/Page/GoogleSerp.php on line 116 4.10.2017 11:41:23PHP Fatal error: Uncaught Serps\SearchEngine\Google\Exception\InvalidDOMException: Unable to check javascript status. Google DOM has possibly changed and an update may be required. in /application/code/vendor/serps/search-engine-google/src/Page/GoogleSerp.php:116 4.10.2017 11:41:23Stack trace:

thebennos commented 7 years ago

solved

the request was not a keyword it was a cache: www.domain.com query.

gsouf commented 7 years ago

Hi @thebennos Was the issue due to your application?

thebennos commented 7 years ago

yeah, mostly. the HTML response is a little different, if you query with something like cache: or inurl:

The DOM parser then throws the expection. Think it is looking for the adwords section that does not appear with above query.

gsouf commented 7 years ago

Not sure, because the error tells that the general structure of the page is not the one the parser knows. That's not related to adword. Would need to take a closer look

Thanks for the details

judgdredd commented 6 years ago

I've have this issue also, I only just started using serp-spider and received this error InvalidDOMException in GoogleSerp.php line 47: Raw dom is not supported, please provide an evaluated version of the dom Google DOM has possibly changed and an update may be required.

My code is directly from the examples. $browser = new Browser(new CurlClient()); $googleUrl = new GoogleUrl(); $googleUrl->setSearchTerm('simpsons'); $googleClient = new GoogleClient($browser); $googleData = $googleClient->query($googleUrl); $results = $googleData->getNaturalResults();

The exception seems to only happen when using getNaturalResults() which is the only one i need to use unfortunately.

Edit: it also happens with getAdwordsResults()

Edit 2: Fixed... I just needed to add a user-agent and it no longer throws the error.

gsouf commented 6 years ago

@judgdredd you need to set a valid user agent for google to consider your browser valid. See: https://serp-spider.github.io/documentation/search-engine/google/