search

shaikhsajid1111 / facebook_page_scraper

Scrapes facebook's pages front end with no limitations & provides a feature to turn data into structured JSON or CSV
https://pypi.org/project/facebook-page-scraper/
MIT License
236 stars 66 forks source link

Failure when calling scrap_to_json() #58

Closed LUKE1860 closed 1 year ago

LUKE1860 commented 1 year ago

Error:

[WDM] - Current google-chrome version is 112.0.5615
[WDM] - Get LATEST driver version for 112.0.5615
[WDM] - Driver [***] found in cache

DevTools listening on ws://127.0.0.1:56478/devtools/browser/e6c8b2dc-2403-490c-9eeb-36b531efcdea
127.0.0.1:56489: Traceback (most recent call last):
  File "C:\Python310\lib\site-packages\seleniumwire\thirdparty\mitmproxy\server\server.py", line 113, in handle
    root_layer()
  File "C:\Python310\lib\site-packages\seleniumwire\thirdparty\mitmproxy\server\modes\http_proxy.py", line 9, in __call__
    layer()
  File "C:\Python310\lib\site-packages\seleniumwire\thirdparty\mitmproxy\server\protocol\tls.py", line 285, in __call__
    layer()
  File "C:\Python310\lib\site-packages\seleniumwire\thirdparty\mitmproxy\server\protocol\http1.py", line 100, in __call__
    layer()
  File "C:\Python310\lib\site-packages\seleniumwire\thirdparty\mitmproxy\server\protocol\http.py", line 205, in __call__
    if not self._process_flow(flow):
  File "C:\Python310\lib\site-packages\seleniumwire\thirdparty\mitmproxy\server\protocol\http.py", line 304, in _process_flow
    return self.handle_regular_connect(f)
  File "C:\Python310\lib\site-packages\seleniumwire\thirdparty\mitmproxy\server\protocol\http.py", line 223, in handle_regular_connect
    layer()
  File "C:\Python310\lib\site-packages\seleniumwire\thirdparty\mitmproxy\server\protocol\tls.py", line 278, in __call__
    self._establish_tls_with_client_and_server()
  File "C:\Python310\lib\site-packages\seleniumwire\thirdparty\mitmproxy\server\protocol\tls.py", line 358, in _establish_tls_with_client_and_server
    self._establish_tls_with_server()
  File "C:\Python310\lib\site-packages\seleniumwire\thirdparty\mitmproxy\server\protocol\tls.py", line 445, in _establish_tls_with_server
    self.server_conn.establish_tls(
  File "C:\Python310\lib\site-packages\seleniumwire\thirdparty\mitmproxy\connections.py", line 290, in establish_tls
    self.convert_to_tls(cert=client_cert, sni=sni, **kwargs)
  File "C:\Python310\lib\site-packages\seleniumwire\thirdparty\mitmproxy\net\tcp.py", line 382, in convert_to_tls
    context = tls.create_client_context(
  File "C:\Python310\lib\site-packages\seleniumwire\thirdparty\mitmproxy\net\tls.py", line 276, in create_client_context
    context = _create_ssl_context(
  File "C:\Python310\lib\site-packages\seleniumwire\thirdparty\mitmproxy\net\tls.py", line 163, in _create_ssl_context
    context = SSL.Context(method)
  File "C:\Python310\lib\site-packages\OpenSSL\SSL.py", line 674, in __init__
    res = _lib.SSL_CTX_set_ecdh_auto(context, 1)
AttributeError: module 'lib' has no attribute 'SSL_CTX_set_ecdh_auto'

[0422/154849.820:ERROR:ssl_client_socket_impl.cc(992)] handshake failed; returned -1, SSL error code 1, net_error -100
127.0.0.1:56492: Traceback (most recent call last):
  File "C:\Python310\lib\site-packages\seleniumwire\thirdparty\mitmproxy\server\server.py", line 113, in handle
    root_layer()
  File "C:\Python310\lib\site-packages\seleniumwire\thirdparty\mitmproxy\server\modes\http_proxy.py", line 9, in __call__
    layer()
  File "C:\Python310\lib\site-packages\seleniumwire\thirdparty\mitmproxy\server\protocol\tls.py", line 285, in __call__
    layer()
  File "C:\Python310\lib\site-packages\seleniumwire\thirdparty\mitmproxy\server\protocol\http1.py", line 100, in __call__
    layer()
  File "C:\Python310\lib\site-packages\seleniumwire\thirdparty\mitmproxy\server\protocol\http.py", line 205, in __call__
    if not self._process_flow(flow):
  File "C:\Python310\lib\site-packages\seleniumwire\thirdparty\mitmproxy\server\protocol\http.py", line 304, in _process_flow
    return self.handle_regular_connect(f)
  File "C:\Python310\lib\site-packages\seleniumwire\thirdparty\mitmproxy\server\protocol\http.py", line 223, in handle_regular_connect
    layer()
  File "C:\Python310\lib\site-packages\seleniumwire\thirdparty\mitmproxy\server\protocol\tls.py", line 278, in __call__
    self._establish_tls_with_client_and_server()
  File "C:\Python310\lib\site-packages\seleniumwire\thirdparty\mitmproxy\server\protocol\tls.py", line 358, in _establish_tls_with_client_and_server
    self._establish_tls_with_server()
  File "C:\Python310\lib\site-packages\seleniumwire\thirdparty\mitmproxy\server\protocol\tls.py", line 445, in _establish_tls_with_server
    self.server_conn.establish_tls(
  File "C:\Python310\lib\site-packages\seleniumwire\thirdparty\mitmproxy\connections.py", line 290, in establish_tls
    self.convert_to_tls(cert=client_cert, sni=sni, **kwargs)
  File "C:\Python310\lib\site-packages\seleniumwire\thirdparty\mitmproxy\net\tcp.py", line 382, in convert_to_tls
    context = tls.create_client_context(
  File "C:\Python310\lib\site-packages\seleniumwire\thirdparty\mitmproxy\net\tls.py", line 276, in create_client_context
    context = _create_ssl_context(
  File "C:\Python310\lib\site-packages\seleniumwire\thirdparty\mitmproxy\net\tls.py", line 163, in _create_ssl_context
    context = SSL.Context(method)
  File "C:\Python310\lib\site-packages\OpenSSL\SSL.py", line 674, in __init__
    res = _lib.SSL_CTX_set_ecdh_auto(context, 1)
AttributeError: module 'lib' has no attribute 'SSL_CTX_set_ecdh_auto'

[0422/154849.935:ERROR:ssl_client_socket_impl.cc(992)] handshake failed; returned -1, SSL error code 1, net_error -100
2023-04-22 15:48:50,068 - facebook_page_scraper.scraper - ERROR - Error at scrap_to_csv : Message: unknown error: net::ERR_CONNECTION_CLOSED
  (Session info: headless chrome=112.0.5615.138)
Stacktrace:
Backtrace:
        GetHandleVerifier [0x00B8DCE3+50899]
        (No symbol) [0x00B1E111]
        (No symbol) [0x00A25588]
        (No symbol) [0x00A21D87]
        (No symbol) [0x00A18B45]
        (No symbol) [0x00A19B1A]
        (No symbol) [0x00A18E20]
        (No symbol) [0x00A18275]
        (No symbol) [0x00A1820C]
        (No symbol) [0x00A16F06]
        (No symbol) [0x00A17668]
        (No symbol) [0x00A26D22]
        (No symbol) [0x00A7E631]
        (No symbol) [0x00A6B8FC]
        (No symbol) [0x00A7E01C]
        (No symbol) [0x00A6B6F6]
        (No symbol) [0x00A47708]
        (No symbol) [0x00A4886D]
        GetHandleVerifier [0x00DF3EAE+2566302]
        GetHandleVerifier [0x00E292B1+2784417]
        GetHandleVerifier [0x00E2327C+2759788]
        GetHandleVerifier [0x00C25740+672048]
        (No symbol) [0x00B28872]
        (No symbol) [0x00B241C8]
        (No symbol) [0x00B242AB]
        (No symbol) [0x00B171B7]
        BaseThreadInitThunk [0x75627D49+25]
        RtlInitializeExceptionChain [0x7781B74B+107]
        RtlClearBits [0x7781B6CF+191]
Traceback (most recent call last):
  File "C:\Python310\lib\site-packages\facebook_page_scraper\scraper.py", line 151, in scrap_to_csv
    data = self.scrap_to_json()  # get the data in JSON format from the same class method
  File "C:\Python310\lib\site-packages\facebook_page_scraper\scraper.py", line 83, in scrap_to_json
    self.__driver.get(self.URL)
  File "C:\Python310\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 436, in get
    self.execute(Command.GET, {'url': url})
  File "C:\Python310\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 424, in execute
    self.error_handler.check_response(response)
  File "C:\Python310\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: net::ERR_CONNECTION_CLOSED
  (Session info: headless chrome=112.0.5615.138)
Stacktrace:
Backtrace:
        GetHandleVerifier [0x00B8DCE3+50899]
        (No symbol) [0x00B1E111]
        (No symbol) [0x00A25588]
        (No symbol) [0x00A21D87]
        (No symbol) [0x00A18B45]
        (No symbol) [0x00A19B1A]
        (No symbol) [0x00A18E20]
        (No symbol) [0x00A18275]
        (No symbol) [0x00A1820C]
        (No symbol) [0x00A16F06]
        (No symbol) [0x00A17668]
        (No symbol) [0x00A26D22]
        (No symbol) [0x00A7E631]
        (No symbol) [0x00A6B8FC]
        (No symbol) [0x00A7E01C]
        (No symbol) [0x00A6B6F6]
        (No symbol) [0x00A47708]
        (No symbol) [0x00A4886D]
        GetHandleVerifier [0x00DF3EAE+2566302]
        GetHandleVerifier [0x00E292B1+2784417]
        GetHandleVerifier [0x00E2327C+2759788]
        GetHandleVerifier [0x00C25740+672048]
        (No symbol) [0x00B28872]
        (No symbol) [0x00B241C8]
        (No symbol) [0x00B242AB]
        (No symbol) [0x00B171B7]
        BaseThreadInitThunk [0x75627D49+25]
        RtlInitializeExceptionChain [0x7781B74B+107]
        RtlClearBits [0x7781B6CF+191]
    Code:
    ```
    from facebook_page_scraper import Facebook_scraper

page_name = "metaai" posts_count = 10 browser = "chrome" proxy = "IP:PORT" timeout = 10 headless = True meta_ai = Facebook_scraper(page_name, posts_count, browser, proxy=proxy, timeout=timeout, headless=headless) json_data = meta_ai.scrap_to_json() print(json_data)

Packages: pyOpenSSL 21.0.0 selenium 4.1.0 cryptography 38.0.4

LUKE1860 commented 1 year ago

I fixed it.