search

metaperl / signalstart-analysis

Scrape www.SignalStart.com and provide data for deeper analysis of providers.
MIT License
1 stars 2 forks source link

element click intercepted during next page logic #7

Open metaperl opened 1 week ago

metaperl commented 1 week ago 
------------------------------- next page logic --------------------------------------
[69856:74684:0623/104813.615:ERROR:device_event_log_impl.cc(196)] [10:48:13.614] USB: usb_service_win.cc:105 SetupDiGetDeviceProperty({{A45C254E-DF1C-4EFD-8020-67D146A850E0}, 6}) failed: Element not found. (0x490)
Created TensorFlow Lite XNNPACK delegate for CPU.
DEBUG:selenium.webdriver.remote.remote_connection:POST http://localhost:59644/session/2295136597097b1a0a9ea11b54bcc752/element {'using': 'css selector', 'value': '.fa-angle-right'}
2024-06-23 10:48:21 [selenium.webdriver.remote.remote_connection] DEBUG: POST http://localhost:59644/session/2295136597097b1a0a9ea11b54bcc752/element {'using': 'css selector', 'value': '.fa-angle-right'}
DEBUG:urllib3.connectionpool:http://localhost:59644 "POST /session/2295136597097b1a0a9ea11b54bcc752/element HTTP/11" 200 0
2024-06-23 10:48:21 [urllib3.connectionpool] DEBUG: http://localhost:59644 "POST /session/2295136597097b1a0a9ea11b54bcc752/element HTTP/11" 200 0
DEBUG:selenium.webdriver.remote.remote_connection:Remote response: status=200 | data={"value":{"element-6066-11e4-a52e-4f735466cecf":"f.6B1BF3E882CED03496228B6CC432E5ED.d.2362C2CA6693689000449B7C6D99FDA8.e.292"}} | headers=HTTPHeaderDict({'Content-Length': '127', 'Content-Type': 'application/json; charset=utf-8', 'cache-control': 'no-cache'})
2024-06-23 10:48:21 [selenium.webdriver.remote.remote_connection] DEBUG: Remote response: status=200 | data={"value":{"element-6066-11e4-a52e-4f735466cecf":"f.6B1BF3E882CED03496228B6CC432E5ED.d.2362C2CA6693689000449B7C6D99FDA8.e.292"}} | headers=HTTPHeaderDict({'Content-Length': '127', 'Content-Type': 'application/json; charset=utf-8', 'cache-control': 'no-cache'})
DEBUG:selenium.webdriver.remote.remote_connection:Finished Request
2024-06-23 10:48:21 [selenium.webdriver.remote.remote_connection] DEBUG: Finished Request
 *** NEXT IS -NOT- NONE: there is another page to navigate to
DEBUG:selenium.webdriver.remote.remote_connection:POST http://localhost:59644/session/2295136597097b1a0a9ea11b54bcc752/element/f.6B1BF3E882CED03496228B6CC432E5ED.d.2362C2CA6693689000449B7C6D99FDA8.e.292/click {}
2024-06-23 10:48:21 [selenium.webdriver.remote.remote_connection] DEBUG: POST http://localhost:59644/session/2295136597097b1a0a9ea11b54bcc752/element/f.6B1BF3E882CED03496228B6CC432E5ED.d.2362C2CA6693689000449B7C6D99FDA8.e.292/click {}
DEBUG:urllib3.connectionpool:http://localhost:59644 "POST /session/2295136597097b1a0a9ea11b54bcc752/element/f.6B1BF3E882CED03496228B6CC432E5ED.d.2362C2CA6693689000449B7C6D99FDA8.e.292/click HTTP/11" 400 0
2024-06-23 10:48:22 [urllib3.connectionpool] DEBUG: http://localhost:59644 "POST /session/2295136597097b1a0a9ea11b54bcc752/element/f.6B1BF3E882CED03496228B6CC432E5ED.d.2362C2CA6693689000449B7C6D99FDA8.e.292/click HTTP/11" 400 0
DEBUG:selenium.webdriver.remote.remote_connection:Remote response: status=400 | data={"value":{"error":"element click intercepted","message":"element click intercepted: Element \u003Ci class=\"fa fa-angle-right\">\u003C/i> is not clickable at point (1122, 1378). Other element would receive the click: \u003Ciframe src=\"about:blank\" frameborder=\"0\" scrolling=\"no\" width=\"130px\" height=\"44px\" style=\"outline:none !important; visibility:visible !important; resize:none !important; box-shadow:none !important; overflow:visible !important; background:none !important; opacity:1 !important; filter:alpha(opacity=100) !important; -ms-filter:progid:DXImageTransform.Microsoft.Alpha(Opacity 1}) !important; -mz-opacity:1 !important; -khtml-opacity:1 !important; top:auto !important; right:10px !important; bottom:0px !important; left:auto !important; position:fixed !important; border:0 !important; min-height:44px !important; min-width:130px !important; max-height:44px !important; max-width:130px !important; padding:0 !important; margin:0 !important; -moz-transition-property:none !important; -webkit-transition-property:none !important; -o-transition-property:none !important; transition-property:none !important; transform:none !important; -webkit-transform:none !important; -ms-transform:none !important; width:130px !important; height:44px !important; display:block !important; z-index:1000001 !important; background-color:transparent !important; cursor:none !important; float:none !important; border-radius:unset !important; pointer-events:auto !important; clip:auto !important; color-scheme:light !important;\" id=\"eoir34jr17281719154092384\" class=\"\" title=\"chat widget\">\u003C/iframe>\n  (Session info: chrome=126.0.6478.63)","stacktrace":"\tGetHandleVerifier [0x00300ED3+27667]\n\t(No symbol) [0x00296BD4]\n\t(No symbol) [0x00191AFF]\n\t(No symbol) [0x001D8BB2]\n\t(No symbol) [0x001D6F3D]\n\t(No symbol) [0x001D4DCB]\n\t(No symbol) [0x001D4377]\n\t(No symbol) [0x001C911F]\n\t(No symbol) [0x001F392C]\n\t(No symbol) [0x001C8BA5]\n\t(No symbol) [0x001F3BC4]\n\t(No symbol) [0x0020CAC7]\n\t(No symbol) [0x001F36C6]\n\t(No symbol) [0x001C74D1]\n\t(No symbol) [0x001C804D]\n\tGetHandleVerifier [0x005B3D43+2857603]\n\tGetHandleVerifier [0x00607EAD+3202029]\n\tGetHandleVerifier [0x003846D4+566292]\n\tGetHandleVerifier [0x0038B8CC+595468]\n\t(No symbol) [0x0029F9C4]\n\t(No symbol) [0x0029C6D8]\n\t(No symbol) [0x0029C877]\n\t(No symbol) [0x0028E76E]\n\tBaseThreadInitThunk [0x77497BA9+25]\n\tRtlInitializeExceptionChain [0x77B4C10B+107]\n\tRtlClearBits [0x77B4C08F+191]\n\t(No symbol) [0x00000000]\n"}} | headers=HTTPHeaderDict({'Content-Length': '2535', 'Content-Type': 'application/json; charset=utf-8', 'cache-control': 'no-cache'})
2024-06-23 10:48:22 [selenium.webdriver.remote.remote_connection] DEBUG: Remote response: status=400 | data={"value":{"error":"element click intercepted","message":"element click intercepted: Element \u003Ci class=\"fa fa-angle-right\">\u003C/i> is not clickable at point (1122, 1378). Other element would receive the click: \u003Ciframe src=\"about:blank\" frameborder=\"0\" scrolling=\"no\" width=\"130px\" height=\"44px\" style=\"outline:none !important; visibility:visible !important; resize:none !important; box-shadow:none !important; overflow:visible !important; background:none !important; opacity:1 !important; filter:alpha(opacity=100) !important; -ms-filter:progid:DXImageTransform.Microsoft.Alpha(Opacity 1}) !important; -mz-opacity:1 !important; -khtml-opacity:1 !important; top:auto !important; right:10px !important; bottom:0px !important; left:auto !important; position:fixed !important; border:0 !important; min-height:44px !important; min-width:130px !important; max-height:44px !important; max-width:130px !important; padding:0 !important; margin:0 !important; -moz-transition-property:none !important; -webkit-transition-property:none !important; -o-transition-property:none !important; transition-property:none !important; transform:none !important; -webkit-transform:none !important; -ms-transform:none !important; width:130px !important; height:44px !important; display:block !important; z-index:1000001 !important; background-color:transparent !important; cursor:none !important; float:none !important; border-radius:unset !important; pointer-events:auto !important; clip:auto !important; color-scheme:light !important;\" id=\"eoir34jr17281719154092384\" class=\"\" title=\"chat widget\">\u003C/iframe>\n  (Session info: chrome=126.0.6478.63)","stacktrace":"\tGetHandleVerifier [0x00300ED3+27667]\n\t(No symbol) [0x00296BD4]\n\t(No symbol) [0x00191AFF]\n\t(No symbol) [0x001D8BB2]\n\t(No symbol) [0x001D6F3D]\n\t(No symbol) [0x001D4DCB]\n\t(No symbol) [0x001D4377]\n\t(No symbol) [0x001C911F]\n\t(No symbol) [0x001F392C]\n\t(No symbol) [0x001C8BA5]\n\t(No symbol) [0x001F3BC4]\n\t(No symbol) [0x0020CAC7]\n\t(No symbol) [0x001F36C6]\n\t(No symbol) [0x001C74D1]\n\t(No symbol) [0x001C804D]\n\tGetHandleVerifier [0x005B3D43+2857603]\n\tGetHandleVerifier [0x00607EAD+3202029]\n\tGetHandleVerifier [0x003846D4+566292]\n\tGetHandleVerifier [0x0038B8CC+595468]\n\t(No symbol) [0x0029F9C4]\n\t(No symbol) [0x0029C6D8]\n\t(No symbol) [0x0029C877]\n\t(No symbol) [0x0028E76E]\n\tBaseThreadInitThunk [0x77497BA9+25]\n\tRtlInitializeExceptionChain [0x77B4C10B+107]\n\tRtlClearBits [0x77B4C08F+191]\n\t(No symbol) [0x00000000]\n"}} | headers=HTTPHeaderDict({'Content-Length': '2535', 'Content-Type': 'application/json; charset=utf-8', 'cache-control': 'no-cache'})
DEBUG:selenium.webdriver.remote.remote_connection:Finished Request
2024-06-23 10:48:22 [selenium.webdriver.remote.remote_connection] DEBUG: Finished Request
ERROR:scrapy.core.scraper:Spider error processing <GET https://www.signalstart.com/search-signals> (referer: None)
Traceback (most recent call last):
  File "C:\code\signalstart-analysis\venv\Lib\site-packages\scrapy\utils\defer.py", line 279, in iter_errback
    yield next(it)
          ^^^^^^^^
  File "C:\code\signalstart-analysis\venv\Lib\site-packages\scrapy\utils\python.py", line 350, in __next__
    return next(self.data)
           ^^^^^^^^^^^^^^^
  File "C:\code\signalstart-analysis\venv\Lib\site-packages\scrapy\utils\python.py", line 350, in __next__
    return next(self.data)
           ^^^^^^^^^^^^^^^
  File "C:\code\signalstart-analysis\venv\Lib\site-packages\scrapy\core\spidermw.py", line 106, in process_sync
    for r in iterable:
  File "C:\code\signalstart-analysis\venv\Lib\site-packages\scrapy\spidermiddlewares\referer.py", line 352, in <genexpr>
    return (self._set_referer(r, response) for r in result or ())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\signalstart-analysis\venv\Lib\site-packages\scrapy\core\spidermw.py", line 106, in process_sync
    for r in iterable:
  File "C:\code\signalstart-analysis\venv\Lib\site-packages\scrapy\spidermiddlewares\urllength.py", line 27, in <genexpr>
    return (r for r in result or () if self._filter(r, spider))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\signalstart-analysis\venv\Lib\site-packages\scrapy\core\spidermw.py", line 106, in process_sync
    for r in iterable:
  File "C:\code\signalstart-analysis\venv\Lib\site-packages\scrapy\spidermiddlewares\depth.py", line 31, in <genexpr>
    return (r for r in result or () if self._filter(r, response, spider))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\signalstart-analysis\venv\Lib\site-packages\scrapy\core\spidermw.py", line 106, in process_sync
    for r in iterable:
  File "C:\code\signalstart-analysis\signalstart\signalstart\spiders\signalstart_spider.py", line 184, in parse
    next.click()
  File "C:\code\signalstart-analysis\venv\Lib\site-packages\selenium\webdriver\remote\webelement.py", line 94, in click
    self._execute(Command.CLICK_ELEMENT)
  File "C:\code\signalstart-analysis\venv\Lib\site-packages\selenium\webdriver\remote\webelement.py", line 395, in _execute
    return self._parent.execute(command, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\signalstart-analysis\venv\Lib\site-packages\selenium\webdriver\remote\webdriver.py", line 354, in execute
    self.error_handler.check_response(response)
  File "C:\code\signalstart-analysis\venv\Lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 229, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementClickInterceptedException: Message: element click intercepted: Element <i class="fa fa-angle-right"></i> is not clickable at point (1122, 1378). Other element would receive the click: <iframe src="about:blank" frameborder="0" scrolling="no" width="130px" height="44px" style="outline:none !important; visibility:visible !important; resize:none !important; box-shadow:none !important; overflow:visible !important; background:none !important; opacity:1 !important; filter:alpha(opacity=100) !important; -ms-filter:progid:DXImageTransform.Microsoft.Alpha(Opacity 1}) !important; -mz-opacity:1 !important; -khtml-opacity:1 !important; top:auto !important; right:10px !important; bottom:0px !important; left:auto !important; position:fixed !important; border:0 !important; min-height:44px !important; min-width:130px !important; max-height:44px !important; max-width:130px !important; padding:0 !important; margin:0 !important; -moz-transition-property:none !important; -webkit-transition-property:none !important; -o-transition-property:none !important; transition-property:none !important; transform:none !important; -webkit-transform:none !important; -ms-transform:none !important; width:130px !important; height:44px !important; display:block !important; z-index:1000001 !important; background-color:transparent !important; cursor:none !important; float:none !important; border-radius:unset !important; pointer-events:auto !important; clip:auto !important; color-scheme:light !important;" id="eoir34jr17281719154092384" class="" title="chat widget"></iframe>
  (Session info: chrome=126.0.6478.63)
Stacktrace:
        GetHandleVerifier [0x00300ED3+27667]
        (No symbol) [0x00296BD4]
        (No symbol) [0x00191AFF]
        (No symbol) [0x001D8BB2]
        (No symbol) [0x001D6F3D]
        (No symbol) [0x001D4DCB]
        (No symbol) [0x001D4377]
        (No symbol) [0x001C911F]
        (No symbol) [0x001F392C]
        (No symbol) [0x001C8BA5]
        (No symbol) [0x001F3BC4]
        (No symbol) [0x0020CAC7]
        (No symbol) [0x001F36C6]
        (No symbol) [0x001C74D1]
        (No symbol) [0x001C804D]
        GetHandleVerifier [0x005B3D43+2857603]
        GetHandleVerifier [0x00607EAD+3202029]
        GetHandleVerifier [0x003846D4+566292]
        GetHandleVerifier [0x0038B8CC+595468]
        (No symbol) [0x0029F9C4]
        (No symbol) [0x0029C6D8]
        (No symbol) [0x0029C877]
        (No symbol) [0x0028E76E]
        BaseThreadInitThunk [0x77497BA9+25]
        RtlInitializeExceptionChain [0x77B4C10B+107]
        RtlClearBits [0x77B4C08F+191]
        (No symbol) [0x00000000]

2024-06-23 10:48:22 [scrapy.core.scraper] ERROR: Spider error processing <GET https://www.signalstart.com/search-signals> (referer: None)
Traceback (most recent call last):
  File "C:\code\signalstart-analysis\venv\Lib\site-packages\scrapy\utils\defer.py", line 279, in iter_errback
    yield next(it)
          ^^^^^^^^
  File "C:\code\signalstart-analysis\venv\Lib\site-packages\scrapy\utils\python.py", line 350, in __next__
    return next(self.data)
           ^^^^^^^^^^^^^^^
  File "C:\code\signalstart-analysis\venv\Lib\site-packages\scrapy\utils\python.py", line 350, in __next__
    return next(self.data)
           ^^^^^^^^^^^^^^^
  File "C:\code\signalstart-analysis\venv\Lib\site-packages\scrapy\core\spidermw.py", line 106, in process_sync
    for r in iterable:
  File "C:\code\signalstart-analysis\venv\Lib\site-packages\scrapy\spidermiddlewares\referer.py", line 352, in <genexpr>
    return (self._set_referer(r, response) for r in result or ())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\signalstart-analysis\venv\Lib\site-packages\scrapy\core\spidermw.py", line 106, in process_sync
    for r in iterable:
  File "C:\code\signalstart-analysis\venv\Lib\site-packages\scrapy\spidermiddlewares\urllength.py", line 27, in <genexpr>
    return (r for r in result or () if self._filter(r, spider))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\signalstart-analysis\venv\Lib\site-packages\scrapy\core\spidermw.py", line 106, in process_sync
    for r in iterable:
  File "C:\code\signalstart-analysis\venv\Lib\site-packages\scrapy\spidermiddlewares\depth.py", line 31, in <genexpr>
    return (r for r in result or () if self._filter(r, response, spider))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\signalstart-analysis\venv\Lib\site-packages\scrapy\core\spidermw.py", line 106, in process_sync
    for r in iterable:
  File "C:\code\signalstart-analysis\signalstart\signalstart\spiders\signalstart_spider.py", line 184, in parse
    next.click()
  File "C:\code\signalstart-analysis\venv\Lib\site-packages\selenium\webdriver\remote\webelement.py", line 94, in click
    self._execute(Command.CLICK_ELEMENT)
  File "C:\code\signalstart-analysis\venv\Lib\site-packages\selenium\webdriver\remote\webelement.py", line 395, in _execute
    return self._parent.execute(command, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\signalstart-analysis\venv\Lib\site-packages\selenium\webdriver\remote\webdriver.py", line 354, in execute
    self.error_handler.check_response(response)
  File "C:\code\signalstart-analysis\venv\Lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 229, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementClickInterceptedException: Message: element click intercepted: Element <i class="fa fa-angle-right"></i> is not clickable at point (1122, 1378). Other element would receive the click: <iframe src="about:blank" frameborder="0" scrolling="no" width="130px" height="44px" style="outline:none !important; visibility:visible !important; resize:none !important; box-shadow:none !important; overflow:visible !important; background:none !important; opacity:1 !important; filter:alpha(opacity=100) !important; -ms-filter:progid:DXImageTransform.Microsoft.Alpha(Opacity 1}) !important; -mz-opacity:1 !important; -khtml-opacity:1 !important; top:auto !important; right:10px !important; bottom:0px !important; left:auto !important; position:fixed !important; border:0 !important; min-height:44px !important; min-width:130px !important; max-height:44px !important; max-width:130px !important; padding:0 !important; margin:0 !important; -moz-transition-property:none !important; -webkit-transition-property:none !important; -o-transition-property:none !important; transition-property:none !important; transform:none !important; -webkit-transform:none !important; -ms-transform:none !important; width:130px !important; height:44px !important; display:block !important; z-index:1000001 !important; background-color:transparent !important; cursor:none !important; float:none !important; border-radius:unset !important; pointer-events:auto !important; clip:auto !important; color-scheme:light !important;" id="eoir34jr17281719154092384" class="" title="chat widget"></iframe>
  (Session info: chrome=126.0.6478.63)
Stacktrace:
        GetHandleVerifier [0x00300ED3+27667]
        (No symbol) [0x00296BD4]
        (No symbol) [0x00191AFF]
        (No symbol) [0x001D8BB2]
        (No symbol) [0x001D6F3D]
        (No symbol) [0x001D4DCB]
        (No symbol) [0x001D4377]
        (No symbol) [0x001C911F]
        (No symbol) [0x001F392C]
        (No symbol) [0x001C8BA5]
        (No symbol) [0x001F3BC4]
        (No symbol) [0x0020CAC7]
        (No symbol) [0x001F36C6]
        (No symbol) [0x001C74D1]
        (No symbol) [0x001C804D]
        GetHandleVerifier [0x005B3D43+2857603]
        GetHandleVerifier [0x00607EAD+3202029]
        GetHandleVerifier [0x003846D4+566292]
        GetHandleVerifier [0x0038B8CC+595468]
        (No symbol) [0x0029F9C4]
        (No symbol) [0x0029C6D8]
        (No symbol) [0x0029C877]
        (No symbol) [0x0028E76E]
        BaseThreadInitThunk [0x77497BA9+25]
        RtlInitializeExceptionChain [0x77B4C10B+107]
        RtlClearBits [0x77B4C08F+191]
        (No symbol) [0x00000000]

DEBUG:scrapy.core.engine:Crawled (200) <GET https://www.signalstart.com/analysis/scalex/271455> (referer: https://www.signalstart.com/search-signals)
2024-06-23 10:48:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.signalstart.com/analysis/scalex/271455> (referer: https://www.signalstart.com/search-signals)
2024-06-23 10:48:22.419 | DEBUG    | signalstart.spiders.signalstart_spider:parse_details:100 - --------------------------------------- Details parse
DJCool1 commented 1 week ago

Seems once it reaches the last page it doesn't end cleanly, it starts spewing errors.