[BUG] Jobs fail with AttributeError

[Markus00000]

Describe the bug

Since version 3.18.1 jobs fail with an AttributeError. I will also post this as a comment to the Arch packager.

Traceback:

$ webchanges --test 1 -v
2024-02-28 10:57:08,190 cli[132501915649856] INFO: webchanges: 3.18.1 Copyright 2020- Mike Borsetti
2024-02-28 10:57:08,190 cli[132501915649856] INFO: CPython: 3.11.7 ('main', 'Jan 29 2024 16:03:57') GCC 13.2.1 20230801
2024-02-28 10:57:08,191 cli[132501915649856] INFO: System: Linux-6.7.6-arch1-1-x86_64-with-glibc2.39
Created default config file at /home/m/.config/webchanges/config.yaml
> Edit it with webchanges --edit-config
2024-02-28 10:57:08,493 storage[132501915649856] INFO: Loaded configuration from /home/m/.config/webchanges/config.yaml
2024-02-28 10:57:08,493 storage[132501915649856] INFO: Using sqlite3 3.45.1 database at /home/m/.cache/webchanges/cache.db
2024-02-28 10:57:08,514 storage[132501915649856] INFO: Loaded 69 jobs from [PosixPath('/home/m/.config/webchanges/jobs.yaml')]
2024-02-28 10:57:08,514 handler[132501915649856] INFO: Job 1: https://github.com/mborsetti/webchanges/issues started processing (UrlJob)
2024-02-28 10:57:08,514 jobs[132501915649856] INFO: Job 1: Sending GET request to https://github.com/mborsetti/webchanges/issues
2024-02-28 10:57:08,514 jobs[132501915649856] INFO: Job 1: Using the HTTPX HTTP client library (HTTP/2 support is not available since h2 is not installed)
Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/webchanges/handler.py", line 185, in process
    data, self.new_etag = self.job.retrieve(self, headless)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/webchanges/jobs.py", line 956, in retrieve
    data, etag = self._retrieve_httpx(headers=headers, timeout=timeout)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/webchanges/jobs.py", line 667, in _retrieve_httpx
    client = httpx.Client(
             ^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'Client'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/webchanges/handler.py", line 200, in process
    self.traceback = self.job.format_error(e, traceback.format_exc())
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/webchanges/jobs.py", line 989, in format_error
    exception, (httpx.HTTPError, httpx.InvalidURL, httpx.CookieConflict, httpx.StreamError)
                ^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'HTTPError'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/webchanges", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/lib/python3.11/site-packages/webchanges/cli.py", line 391, in main
    urlwatch_command.run()
  File "/usr/lib/python3.11/site-packages/webchanges/command.py", line 1078, in run
    self.handle_actions()
  File "/usr/lib/python3.11/site-packages/webchanges/command.py", line 1011, in handle_actions
    self.test_job(self.urlwatch_config.test_job)
  File "/usr/lib/python3.11/site-packages/webchanges/command.py", line 417, in test_job
    job_state.process(headless=not self.urlwatch_config.no_headless)
  File "/usr/lib/python3.11/site-packages/webchanges/handler.py", line 211, in process
    self.traceback = self.job.format_error(e, traceback.format_exc())
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/webchanges/jobs.py", line 989, in format_error
    exception, (httpx.HTTPError, httpx.InvalidURL, httpx.CookieConflict, httpx.StreamError)
                ^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'HTTPError'

To Reproduce

Example job:

name: "Test"
url: "https://github.com/mborsetti/webchanges/issues"

Version info

2024-02-28 10:58:17,365 cli[126227908015936] INFO: webchanges: 3.18.1 Copyright 2020- Mike Borsetti 2024-02-28 10:58:17,365 cli[126227908015936] INFO: CPython: 3.11.7 ('main', 'Jan 29 2024 16:03:57') GCC 13.2.1 20230801 2024-02-28 10:58:17,366 cli[126227908015936] INFO: System: Linux-6.7.6-arch1-1-x86_64-with-glibc2.39

[mborsetti]

For some reason the httpx HTTP Client library is not installed in your system, even though is listed as a requirement.

pip show webchanges
[...]
Requires: colorama, cssselect, h2, html2text, **httpx**, lxml, markdown2, msgpack, platformdirs, pyyaml, tzdata

(emphasis mine)

Since it's a requirement there's no testing on what happens when the package is missing, and you discovered a bug in the fallback to requests. While I will be fixing this, it sounds that there's something wrong with the Arch packager (sorry, not familiar with it).

[mborsetti]

3.19 has been released.