zytedata / zyte-smartproxy-selenium

A wrapper over Selenium Wire to provide Zyte Smart Proxy Manager specific functionalities.
3 stars 4 forks source link

No such file or directory issue #8

Open 2-fly-4-ai opened 1 month ago

2-fly-4-ai commented 1 month ago

I'm running into this issue when trying run zyte_smartproxy_selenium

  File "/usr/local/lib/python3.8/site-packages/seleniumwire/handler.py", line 67, in request
    self.proxy.storage.save_request(request)
  File "/usr/local/lib/python3.8/site-packages/seleniumwire/storage.py", line 90, in save_request
    os.mkdir(request_dir)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/.seleniumwire/storage-478d5014-9490-474c-a9e2-14c1b99ce6d3/request-8f006b77-bb6c-4d70-bbd0-b084fba7ba8c```

Here are my current settings:

import os from dotenv import load_dotenv from selenium import webdriver from selenium.webdriver.chrome.options import Options from webdriver_manager.chrome import ChromeDriverManager from selenium.webdriver.common.by import By from flask import Flask, request, jsonify from zyte_smartproxy_selenium import webdriver as zyte_webdriver import time import json import base64 from io import BytesIO

load_dotenv()

app = Flask(name)

SPM_APIKEY = os.getenv('SPM_APIKEY')

def get_driver(): chrome_options = Options() chrome_options.add_argument('--no-sandbox') chrome_options.add_argument('--headless') chrome_options.add_argument('--disable-dev-shm-usage')

# Initialize the Chrome driver with Zyte SmartProxy options
driver = zyte_webdriver.Chrome(
    options=chrome_options,
    spm_options={
        'spm_apikey': SPM_APIKEY,
        'headers': {
            'X-Crawlera-No-Bancheck': '1',
            'X-Crawlera-Profile': 'desktop',
            'X-Crawlera-Cookies': 'disable',
        }
    }
)


   Here is a thread I found with a similar issue: https://github.com/wkeeling/selenium-wire/issues/402

   I"m currently running on the Google cloud run platform.
storymode7 commented 1 month ago

Hi @2-fly-4-ai ,

Since you are using cloud run, can you try in memory storage for selenium requests & body?

https://github.com/wkeeling/selenium-wire/?tab=readme-ov-file#in-memory-storage