Goodreads Sync not compatible with Python 3.12

[StegSchreck]

calibre, version 7.10.0
ERROR: Unhandled exception: <b>TypeError</b>:HTTPSConnection.__init__() got an unexpected keyword argument 'key_file'

calibre 7.10  embedded-python: False
Linux-6.8.9-3-MANJARO-x86_64-with-glibc2.39 Linux ('64bit', 'ELF')
('Linux', '6.8.9-3-MANJARO', '#1 SMP PREEMPT_DYNAMIC Thu May  9 18:25:19 UTC 2024')
Python 3.12.3
Interface language: None
EXE path: /usr/bin/python3
Successfully initialized third party plugins: Action Chains (1, 20, 4) && Amazon.com Multiple Countries (1, 0, 0) && Barnes & Noble (1, 5, 5) && Count Pages (1, 13, 6) && DNB_DE (3, 2, 4) && EpubCheck (0, 2, 5) && Extract ISBN (1, 6, 2) && Extract People Other Metadata (1, 0, 0) && Goodreads (1, 8, 1) && Goodreads Sync (1, 16, 5) && Kindle Collections (1, 7, 29) && Kindle hi-res covers (0, 5, 1) && Modify ePub (1, 8, 3)
Traceback (most recent call last):
  File "calibre_plugins.goodreads_sync.dialogs", line 420, in _accept_clicked
    self.goodreads_shelf_books = self.grhttp.get_goodreads_books_on_shelves(self.user_name, self.selected_shelves)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "calibre_plugins.goodreads_sync.core", line 440, in get_goodreads_books_on_shelves
    (response, content) = self._oauth_request_get(oauth_client, url)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "calibre_plugins.goodreads_sync.core", line 129, in _oauth_request_get
    response, content = oauth_client.request(url, 'GET', headers=headers)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "calibre_plugins.goodreads_sync.oauth2.__init__", line 689, in request
    return httplib2.Http.request(self, uri, method=method, body=body,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "calibre_plugins.goodreads_sync.httplib2.__init__", line 1561, in request
    conn = self.connections[conn_key] = connection_type(
                                        ^^^^^^^^^^^^^^^^
  File "calibre_plugins.goodreads_sync.httplib2.__init__", line 999, in __init__
    httplib.HTTPSConnection.__init__(self, host, port=port,
TypeError: HTTPSConnection.__init__() got an unexpected keyword argument 'key_file'

In Python 3.6, the two parameters key_file and cert_file of http.client.HTTPSConnection were deprecated in favour of an ssl.SSLContext parameter. Now in Python 3.12 the deprecated parameters were removed completely. This PR refactors the usage of key_file and cert_file so that is compliant with Python 3.12 while being backwards compatible to older Python versions.

I've provided a fix in #70

[kiwidude68]

Hi - your fix didn't actually immediately work (because we don't pass through a cert_file for the needs of this plugin) but it was a simple enough change to fix that. Thanks for the contribution!

[StegSchreck]

Hey @kiwidude68 Thanks for your fast reaction to this topics. I was updating the plugin today in my calibre, but unfortunately there is still an error, this time in initializing the HTTPSConnectionWithTimeout:

calibre, version 7.10.0
ERROR: Unhandled exception: <b>AttributeError</b>:'HTTPSConnectionWithTimeout' object has no attribute 'key_file'

calibre 7.10  embedded-python: False
Linux-6.9.0-1-MANJARO-x86_64-with-glibc2.39 Linux ('64bit', 'ELF')
('Linux', '6.9.0-1-MANJARO', '#1 SMP PREEMPT_DYNAMIC Mon May 13 07:01:28 UTC 2024')
Python 3.12.3
Interface language: None
EXE path: /usr/bin/python3
Successfully initialized third party plugins: Action Chains (1, 20, 4) && Amazon.com Multiple Countries (1, 0, 0) && Barnes & Noble (1, 5, 5) && Count Pages (1, 13, 6) && DNB_DE (3, 2, 4) && EpubCheck (0, 2, 5) && Extract ISBN (1, 6, 2) && Extract People Other Metadata (1, 0, 0) && Goodreads (1, 8, 1) && Goodreads Sync (1, 16, 6) && Kindle Collections (1, 7, 29) && Kindle hi-res covers (0, 5, 1) && Modify ePub (1, 8, 3)
Traceback (most recent call last):
  File "calibre_plugins.goodreads_sync.dialogs", line 420, in _accept_clicked
    self.goodreads_shelf_books = self.grhttp.get_goodreads_books_on_shelves(self.user_name, self.selected_shelves)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "calibre_plugins.goodreads_sync.core", line 440, in get_goodreads_books_on_shelves
    (response, content) = self._oauth_request_get(oauth_client, url)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "calibre_plugins.goodreads_sync.core", line 129, in _oauth_request_get
    response, content = oauth_client.request(url, 'GET', headers=headers)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "calibre_plugins.goodreads_sync.oauth2.__init__", line 689, in request
    return httplib2.Http.request(self, uri, method=method, body=body,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "calibre_plugins.goodreads_sync.httplib2.__init__", line 1688, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "calibre_plugins.goodreads_sync.httplib2.__init__", line 1428, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "calibre_plugins.goodreads_sync.httplib2.__init__", line 1348, in _conn_request
    conn.connect()
  File "calibre_plugins.goodreads_sync.httplib2.__init__", line 1098, in connect
    sock, self.key_file, self.cert_file,
          ^^^^^^^^^^^^^
AttributeError: 'HTTPSConnectionWithTimeout' object has no attribute 'key_file'

As this is a self-defined class in this project and it clearly still has the attribute key_file, I am a bit confused on why this happens know. I was thinking if it would make sense to change the signature in HTTPSConnectionWithTimeout to use context as well, and create the SSLContext inside of Http.request() where it is being initialized via the dict. Wdyt?

[kiwidude68]

@StegShreck - Please remember I cannot properly test your changes since I am on Windows and it is Python 3.10. Your first set of changes had other problems which meant it would not run at all. I have merged in your latest commit - is this now a workable version of the plugin or do more changes yet have to be made? As it is only you who can test your changes at this point?

I won't put out the next release build until you have confirmed it now works as needed.

[StegSchreck]

As I mentioned in my first message in the ticket #70, there are only windows build scripts available in this repo, so I cannot test this locally on my Linux system. I can only go by what the error message in calibre shows me after updating to the newest release - which is also frustrating for me.

[kiwidude68]

Plugins are just a zip file - you dont need a build script! Just replace the file in the plugin zip with the one you have modified, it is that simple.

[StegSchreck]

I didn't know that and would have wished to have this info earlier so I would have a chance to test locally. The Contributing.md only mentions the need for running the windows .cmd files for creating the plugin zip. I would suggest that the documentation gets an extensions, because this windows stuff scared me off initially and I am sure others as well.

Additionally, I would strongly recommend: upgrade your environment to Python 12. For this you can also use something like pyenv, even on windows.

[kiwidude68]

Python on Windows is supplied as part of the calibre binaries, you can't independently use your own version. It won't be bumping in version until the next major calibre version upgrade.

You didn't answer the actual question I need to know - have you now tested it? Does it work?

[StegSchreck]

I will try to find some time before my vacation to test it locally.

[bernieke]

With https://github.com/kiwidude68/calibre_plugins/pull/74 it is now working.