Problem retrieving headers

[mmoore99]

google-chrome --version Google Chrome 96.0.4664.93

wget https://chromedriver.storage.googleapis.com/96.0.4664.45/chromedriver_linux64.zip

I am attempting to retrieve the headers, but get the following error:

root@fiddlemon-prod: /home/ubuntu/southwest-headers
> env/bin/python southwest-headers.py
Traceback (most recent call last):
  File "southwest-headers.py", line 29, in <module>
    driver = webdriver.Chrome(os.getcwd() + "/chromedriver", options=chrome_options)
  File "/home/ubuntu/southwest-headers/env/lib/python3.8/site-packages/seleniumwire/webdriver.py", line 195, in __init__    super().__init__(*args, **kwargs)
  File "/home/ubuntu/southwest-headers/env/lib/python3.8/site-packages/selenium/webdriver/chrome/webdriver.py", line 70, in __init__
    super(WebDriver, self).__init__(DesiredCapabilities.CHROME['browserName'], "goog",
  File "/home/ubuntu/southwest-headers/env/lib/python3.8/site-packages/selenium/webdriver/chromium/webdriver.py", line 93, in __init__
    RemoteWebDriver.__init__(
  File "/home/ubuntu/southwest-headers/env/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 268, in __init__
    self.start_session(capabilities, browser_profile)
  File "/home/ubuntu/southwest-headers/env/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 359, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/home/ubuntu/southwest-headers/env/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 424, in execute
    self.error_handler.check_response(response)
  File "/home/ubuntu/southwest-headers/env/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally.
  (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Stacktrace:
#0 0x55c91137dee3 <unknown>
#1 0x55c910e4b608 <unknown>
#2 0x55c910e6e8ef <unknown>
#3 0x55c910e6a46f <unknown>
#4 0x55c910ea4f45 <unknown>
#5 0x55c910e9f163 <unknown>
#6 0x55c910e74bfc <unknown>
#7 0x55c910e75c05 <unknown>
#8 0x55c9113afbaa <unknown>
#9 0x55c9113c5651 <unknown>
#10 0x55c9113b0b05 <unknown>
#11 0x55c9113c6a68 <unknown>
#12 0x55c9113a505f <unknown>
#13 0x55c9113e1818 <unknown>
#14 0x55c9113e1998 <unknown>
#15 0x55c9113fceed <unknown>
#16 0x7f83ad2d6609 <unknown>

Any suggetions would be appreciated.

[mmoore99]

I was running as root when I executed the command in the first post. I changed to a user account and am now getting the following error:

mike@fiddlemon-prod:/home/ubuntu/southwest-headers$ env/bin/python southwest-headers.py
Traceback (most recent call last):
  File "southwest-headers.py", line 29, in <module>
    driver = webdriver.Chrome(os.getcwd() + "/chromedriver", options=chrome_options)
  File "/home/ubuntu/southwest-headers/env/lib/python3.8/site-packages/seleniumwire/webdriver.py", line 189, in __init__
    config = self._setup_backend(seleniumwire_options)
  File "/home/ubuntu/southwest-headers/env/lib/python3.8/site-packages/seleniumwire/webdriver.py", line 36, in _setup_backend
    self.backend = backend.create(
  File "/home/ubuntu/southwest-headers/env/lib/python3.8/site-packages/seleniumwire/backend.py", line 24, in create
    backend = MitmProxy(addr, port, options)
  File "/home/ubuntu/southwest-headers/env/lib/python3.8/site-packages/seleniumwire/server.py", line 27, in __init__
    self.storage = storage.create(**self._get_storage_args())
  File "/home/ubuntu/southwest-headers/env/lib/python3.8/site-packages/seleniumwire/storage.py", line 39, in create
    return RequestStorage(base_dir=kwargs.get('base_dir'))
  File "/home/ubuntu/southwest-headers/env/lib/python3.8/site-packages/seleniumwire/storage.py", line 70, in __init__
    os.makedirs(self.session_dir, exist_ok=True)
  File "/usr/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/tmp/.seleniumwire/storage-7815e557-5d89-4cbc-bbf6-93b21a29f898'

[byalextran]

let's try this ...

make sure you're in the southwest-headers folder and then run:

git pull origin develop
git checkout develop
env/bin/python southwest-headers.py

two things to test:

1. if it solves the last error.
2. if it also works now as the root user.

[mmoore99]

Thanks for responding so quickly and above all, thanks for your efforts to produce this solution to the Southwest checkin situation.

Regarding the testing of the fixes:

It now works fine when running under root.

When running under a user account the following error is generated:

ubuntu@fiddlemon-prod:~/southwest-headers$ cd /home/ubuntu/southwest-headers/ && env/bin/python southwest-headers.py
Traceback (most recent call last):
  File "southwest-headers.py", line 63, in <module>
    with open(output_file, "w") as json_file:
PermissionError: [Errno 13] Permission denied: 'southwest_headers.json'

[byalextran]

ahh, my guess is that you created the southwest-headers directory as the root user and when you logged in as a regular user, you tried running the script from that same folder created by root?

if that's the case, try logging in as the regular user and re-downloading/installing southwest-headers into it's own folder in the regular user's home directory. see if that gets things working as a regular user (which is how i'd probably recommend running this script as opposed to a root user).

[mmoore99]

ahh, my guess is that you created the southwest-headers directory as the root user and when you logged in as a regular user, you tried running the script from that same folder created by root?

if that's the case, try logging in as the regular user and re-downloading/installing southwest-headers into it's own folder in the regular user's home directory. see if that gets things working as a regular user (which is how i'd probably recommend running this script as opposed to a root user).

Ok, that works. Thanks very much for your help.