Error on GR oAuth

[Sparticuz]

LazyLibrarian version number (at the bottom of config page) 1.5.0

Operating system used (windows, mac, linux, NAS type) Arch

Interface in use (default, bookstrap) bootstrap

Which api (Goodreads, GoogleBooks, both) GR

Source of your LazyLibrarian installation (git, zip file, 3rd party package) AUR

Trying to get Goodreads Sync running, I've created my API key and Secret, filled those out and saved the settings. Then when I click Request oAuth1, the following is returned in Devtools.

500 Internal Server Error
The server encountered an unexpected condition which prevented it from fulfilling the request.

Traceback (most recent call last):
  File "/usr/lib/lazylibrarian/cherrypy/_cprequest.py", line 670, in respond
    response.body = self.handler()
  File "/usr/lib/lazylibrarian/cherrypy/lib/encoding.py", line 217, in __call__
    self.body = self.oldhandler(*args, **kwargs)
  File "/usr/lib/lazylibrarian/cherrypy/_cpdispatch.py", line 61, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "/usr/lib/lazylibrarian/lazylibrarian/webServe.py", line 3132, in grauthStep1
    return GA.goodreads_oauth1()
  File "/usr/lib/lazylibrarian/lazylibrarian/grsync.py", line 71, in goodreads_oauth1
    authorize_link = '%s?oauth_token=%s' % (authorize_url, request_token['oauth_token'])
KeyError: 'oauth_token'
Powered by CherryPy 3.6.0

[philborman]

Interesting. seems we get a "200 ok" response, but without an oauth_token in the result. Hard to fix here as I can't reproiduce the problem. Are you able to do a little editing of the python code?
If not I will upload a new version of grsync.py so we can see what you are getting back as a result. If you are ok with editing, in grsync.py line 72 says request_token = dict(parse_qsl(content)) just below that if you put logger.debug(content) we should get to know what the problem is.

[Sparticuz]

Hah, this is very odd! It logged correctly! 22-Mar-2018 09:00:23 - DEBUG :: CP Server Thread-3 : grsync.py:goodreads_oauth1:70 : b'oauth_token=***********&oauth_token_secret=*******'

EDIT: still got the error and it still didn't work.

[philborman]

Okay... Need to dig deeper into why we don't think it's there. It's either the parse_qsl line, or the next one that's at fault. The next line does some decoding for py2 that py3 doesn't need. Are you on python2 or python3 ? (is this line being run or not) What do you get if you change your extra debug line from logger.debug(content) to logger.debug(request_token) (is the parse_qsl returning what we expect)

[Sparticuz]

I'm on 3. first line is content, second is request_token

22-Mar-2018 09:12:08 - DEBUG   :: CP Server Thread-7 : grsync.py:goodreads_oauth1:70 : b'oauth_token=***********&oauth_token_secret=************'
22-Mar-2018 09:12:08 - DEBUG   :: CP Server Thread-7 : grsync.py:goodreads_oauth1:71 : {b'oauth_token': b'***********', b'oauth_token_secret': b'****************'}

[Sparticuz]

It's the authorize_link line that's failing, I inserted a logger.debug(authorize_link) after it, and it never ran.

[philborman]

Your first post shows the authorize_link line fails because it can't find 'oauth_token' in the request_token dictionary. The question is why we can't find it in the dictionary, as it seems to be there. Can you add another line just above the "authorize_link = " line printing out request_token again (that's the request dictionary after the unicode conversion)

[Sparticuz]

logger.debug(request_token[b'oauth_token']) seems to work, but then it adds b'***' to my token. It looks like some kind of encoding problem. (hint found here: https://stackoverflow.com/questions/24816114/keyerror-when-key-exists#comment38523452_24816114)

[philborman]

The "parse_qsl" function returns a dictionary of bytes. The key and value are both bytes. Python3 needs them converting to unicode, which is the purpose of the

        if not PY2:
            request_token = {key.decode("utf-8"): request_token[key].decode("utf-8") for key in request_token}

It looks like either we don't think you're on python3 (so the decode doesn't run) or the decode fails. Can you try adding some debugging lines to see which it is, something like

        if not PY2:
            logger.debug("decoding dictionary")
            request_token = {key.decode("utf-8"): request_token[key].decode("utf-8") for key in request_token}
        logger.debug(request_token)

[Sparticuz]

So I don't even have those lines. I'm running 1.5.0, not master, which looks to be the problem.

[philborman]

Ah, ok. You could add those lines in to fix the problem, or upgrade to current version. Is the lazylibrarian internal upgrade mechanism disabled in the AUR package?

[Sparticuz]

The AUR package is only updated on version and point releases.

[philborman]

Ok, I would suggest just adding those lines yourself for now. The next major release of lazylibrarian is probably a week from now, needs more testing as there are large changes.

[philborman]

Just pushed v1.5.1 interim release with all the current fixes, but held back the major upgrade a little longer. Should fix your issue though. Is the AUR package built automatically?

[Sparticuz]

No, but I can mark it as out of date, and @fryfrog is usually pretty quick to update his packages.

[fryfrog]

I can get it now-ish. :)

[philborman]

@fryfrog Hi,

Just a heads-up, on Saturday (10 Nov) we will be moving the repo to

https://gitlab.com/LazyLibrarian/LazyLibrarian.git

The github/dobytang one will be mothballed as dobytang is no longer around and we cannot get full access rights to it. We would have liked the LazyLibrarian group at github as that's inactive but the owner hasn't responded to requests to release it, so we're moving to gitlab.

The new location also has a brand new much needed documentation section, and we hope to include automated builds of deb rpm and snap packages.

I am notifying you as the maintainer of the aur package. If this is no longer correct please let me know. Many thanks for your support of lazylibrarian

Phil.