403 Forbidden Error

[warthog3]

Describe the problem Seeing this new error when trying to start odmpy

`odmpy libby odmpy Interactive Client for Libby

An unexpected error has occurred Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/odmpy/libby.py", line 357, in make_request res.raise_for_status() File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/requests/models.py", line 1021, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://sentry-read.svc.overdrive.com/chip/sync

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/odmpy/odm.py", line 846, in run synced_state = libby_client.sync() ^^^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/odmpy/libby.py", line 457, in sync res: Dict = self.make_request("chip/sync") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/odmpy/libby.py", line 366, in make_request ErrorHandler.process(http_err) File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/odmpy/libby_errors.py", line 103, in process raise ClientError( odmpy.libby_errors.ClientError: <odmpy.libby_errors.ClientError; http_status=403, msg='403 Client Error: Forbidden for url: https://sentry-read.svc.overdrive.com/chip/sync', error_response='{"result":"missing_chip"}''> Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/odmpy/libby.py", line 357, in make_request res.raise_for_status() File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/requests/models.py", line 1021, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://sentry-read.svc.overdrive.com/chip/sync

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.12/bin/odmpy", line 8, in sys.exit(main()) ^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/odmpy/main.py", line 27, in main run() File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/odmpy/odm.py", line 846, in run synced_state = libby_client.sync() ^^^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/odmpy/libby.py", line 457, in sync res: Dict = self.make_request("chip/sync") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/odmpy/libby.py", line 366, in make_request ErrorHandler.process(http_err) File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/odmpy/libby_errors.py", line 103, in process raise ClientError( odmpy.libby_errors.ClientError: <odmpy.libby_errors.ClientError; http_status=403, msg='403 Client Error: Forbidden for url: https://sentry-read.svc.overdrive.com/chip/sync', error_response='{"result":"missing_chip"}''>`

To Reproduce odmpy libby

Version/Environment odmpy 0.8.1 [Python 3.12.3-darwin]

[wtanksleyjr]

This is a new problem, but there's a simple fix:

Run odmpy libby --reset to clear the Libby registration, and then use the Libby webapp or the phone app to generate a code, and run odmpy libby and enter that code. It'll be back up again.

On Wed, Apr 24, 2024 at 6:57 PM Chris Klann @.***> wrote:

Also getting this error using odmpy 0.8.1 [Python 3.9.2-linux]

— Reply to this email directly, view it on GitHub https://github.com/ping/odmpy/issues/69#issuecomment-2076185923, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJ7H6JSKKFBR7NYPIESFO3Y7BPJNAVCNFSM6AAAAABGX6LV5SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANZWGE4DKOJSGM . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[leomem]

The reset did not work for me. After reset and reentering the code, downloading got the following error. python3.9/site-packages/odmpy/processing/odm.py", line 135, in process_odm raise ValueError("Unable to find Metadata in ODM") ValueError: Unable to find Metadata in ODM

The ODM file's content shows the error.

`<!DOCTYPE html>

{
  "result": "missing_chip",
  "policies": {
    "privacy": "https://company.cdn.overdrive.com/policies/privacy-policy",
    "terms": "https://company.cdn.overdrive.com/policies/terms-and-conditions.htm",
    "jobs": "https://company.overdrive.com/careers"
  }
}

~ `

[wtanksleyjr]

Just to be clear, are you saying you see this when trying to download a book? So you run odmpy libby and see your checked out books in the list with a choice of what to do, but when you select one to download it errors out as above?

-Wm

On Thu, Apr 25, 2024 at 6:36 AM leomem @.***> wrote:

The reset did not work for you. After reset and reentering the code, downloading got the following error. python3.9/site-packages/odmpy/processing/odm.py", line 135, in process_odm raise ValueError("Unable to find Metadata in ODM") ValueError: Unable to find Metadata in ODM

The ODM file's content shows the error.

`

{ "result": "missing_chip", "policies": { "privacy": "https://company.cdn.overdrive.com/policies/privacy-policy", "terms": "https://company.cdn.overdrive.com/policies/terms-and-conditions.htm", "jobs": "https://company.overdrive.com/careers" } } ~ ` — Reply to this email directly, view it on GitHub , or unsubscribe . You are receiving this because you commented.Message ID: ***@***.***>

[leomem]

That's correct. I run odmpy libby and see my checked out books in the list. When I tried to download any of them, I got the error. The generated .odm file contained the above error. I have tried forceful reinstall too.

[wtanksleyjr]

Whew! Sorry about that. I'll have to step back and hope one of the experts can look at this, @ping maybe.

[jahlborn]

see #70

[wtanksleyjr]

OK, I have to confirm this - I'm now getting the same result. I kind of doubt they're attacking odmpy specifically, since they could do that just by sending GitHub a takedown notice; but for now it's the same effect.

[wtanksleyjr]

If you're in a situation where you're about to have a book expire while waiting for this, please note that the existing overdrive app for Windows should still work (if you're in a position to run it, of course). You'll have to download it from their site (paste this link into your browser press enter and then approve the security warning, don't try to click it or it'll just close the tab due to being an MSI file): http://app.overdrive.com/downloads/ODMediaConsoleSetup.msi; this is their official version so it's as safe as it was back when it was released. Go to your library's Overdrive site (usually something like lapl.overdrive.com). Sign in with your library card and last 4 digits of your phone number (or last 5 digits, or whatever). From there you should be able to get a download file, and that app can open it and use it to get the usual mp3 files. Not as nice as what we've become used to, but that's life!

If you need more metadata, you can download some nice JSON from an URL of the form https://thunder.api.overdrive.com/v2/media/9722081 (I picked a random libby identifier there). It'll work even if you don't have the book checked out, so long as you have the book ID. Use odmpy libby --exportloans loans.json to find the book ID of the books you currently have checked out.

[leomem]

The overdrive windows program works. Thanks a lot. Not sure how to use the JSON metadata yet.

[cerinawithasea]

This is The error I am getting on both my Mac and windows laptop reset is not fixing it The error will come after something similar to

Opening audiobook "When Trying to Return Home"... Downloaded odm to "when-trying-to-return-home-9350409.odm" An unexpected error has occurred Then the error

Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/odmpy/odm.py", line 1210, in run process_odm( File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/odmpy/processing/odm.py", line 135, in process_odm raise ValueError("Unable to find Metadata in ODM") ValueError: Unable to find Metadata in ODM

I have tried different books

The reset did not work for me. After reset and reentering the code, downloading got the following error. python3.9/site-packages/odmpy/processing/odm.py", line 135, in process_odm raise ValueError("Unable to find Metadata in ODM") ValueError: Unable to find Metadata in ODM

The ODM file's content shows the error.

`

{ "result": "missing_chip", "policies": { "privacy": "https://company.cdn.overdrive.com/policies/privacy-policy", "terms": "https://company.cdn.overdrive.com/policies/terms-and-conditions.htm", "jobs": "https://company.overdrive.com/careers" } } ~ `

[mitrian7]

Go to your library's Overdrive site (usually something like lapl.overdrive.com). Sign in with your library card and last 4 digits of your phone number (or last 5 digits, or whatever). From there you should be able to get a download file, and that app can open it and use it to get the usual mp3 files. Not as nice as what we've become used to, but that's life!

So, this is cool, thank you. But also worth noting, you don't even need the desktop app. Just download the odm file from the overdrive site as you described, then you can use that odm file with odmpy directly. For example, if you want the benefits of merging with chapter metadata that odmpy does so nicely, you can run something like this:

odmpy dl "title.odm" -m --mergeformat m4b --mergecodec aac

... where title.odm is what you downloaded from the overdrive site.

[moviebrain]

Is this something that can even be fixed? The browser extensions stopped working about 48 hours prior to this stopping, so it seems like one by one these tools are being targeted specifically. Honestly I’d love if this just happens to be a back end change that needs a tweak but I’d fully understand any and all contributors not wanting to engage in a cat and mouse endless game of “fix the authentication”

If it can’t be fixed I’ll greatly miss the ability to play library audiobooks on the unsupported kids-boxy- speaker-things we have, it’s genuinely reignited my their love of stories. Thanks @ping for everything you did, in advance.

[mitrian7]

Is this something that can even be fixed?

While I do hope there's a solution, I think even if it can't be fixed, the workaround (see the post above) is mostly painless, it just adds the extra step of downloading the .odm manually from OverDrive. You can still get all the power and benefits of odmpy here.

[jahlborn]

Is this something that can even be fixed?

While I do hope there's a solution, I think even if it can't be fixed, the workaround (see the post above) is mostly painless, it just adds the extra step of downloading the .odm manually from OverDrive. You can still get all the power and benefits of odmpy here.

i certainly hope there is a solution as well. the workaround is manual, and the power of this library is that i can automate the process.

maybe this library could be patched to grab the odm files from the overdrive website...

[bckelly1]

I've been taking a look at this and to me it looks like they've changed how the JWT token is structured. The old JWT is very different than the one the web UI is using. Not sure if it's a red herring but it may be that the python client needs to use a different method to fetch the JWT.

Not an expert, and I'll look deeper when I get some time.

[jahlborn]

in theory, this should be fixable. if the libby app and the libby webpage can load the data, then it should be possible to determine what api calls they are making and reproduce them. granted, they can use a ton of obfuscation to make that difficult...

[Prajna-Karuna]

From there you should be able to get a download file

I have looked everywhere and can't find a way to download the odm. Neither from the overdrive.com site nor from the installed msi file (which doesn't allow me to login, so it is just an empty overdrive GUI) Can you be more specific on how to download the odm?

[wtanksleyjr]

Prajnah, would you mind identifying which library you are using, and I'll find the right site for you?

For example: I'm using Los Angeles Public Library, so to get ODMs I go to lapl.overdrive.com (notice the abbreviation for the library's name, it's not always that simple but sometimes is), log in, click "My Account", choose "Loans", and on any specific audiobook title, click "Have Overdrive?", and you'll see a button appear, click it and you'll download the ODM.

BTW you can open that in the Windows Overdrive app, but if you're using odmpy already it turns out you can just use odmpy to do all the downloading using the same file. See above for instructions.

-Wm

[moviebrain]

Prajnah, would you mind identifying which library you are using, and I'll find the right site for you? For example: I'm using Los Angeles Public Library, so to get ODMs I go to lapl.overdrive.com (notice the abbreviation for the library's name, it's not always that simple but sometimes is), log in, click "My Account", choose "Loans", and on any specific audiobook title, click "Have Overdrive?", and you'll see a button appear, click it and you'll download the ODM. BTW you can open that in the Windows Overdrive app, but if you're using odmpy already it turns out you can just use odmpy to do all the downloading using the same file. See above for instructions. … -Wm

My local branch overdrive site hid the download link for the odm because I was using a hardware/software combo that the Overdrive for (win/mac) didn’t support. I had to change the user-agent string in Firefox about:config to an appropriate OS version and then the “download” button appeared again.

[bckelly1]

in theory, this should be fixable. if the libby app and the libby webpage can load the data, then it should be possible to determine what api calls they are making and reproduce them. granted, they can use a ton of obfuscation to make that difficult...

I just did this and I noticed a couple things. The JWT in use actually changes at least 3 times. Also, the JWT is really weird. It's not a standard JWT format that I recognize. They are much longer and they have literally ... in the middle. It doesn't parse as a valid JWT.

Anyway, I did actually get it to work for books, but audiobooks has a problem that I have to debug (still the missing chip). You can splice the contents/results of two different calls and merge them into the libby.json replacing the chip, identity and __libby_sync_code fields.

[ziferius]

Prajnah, would you mind identifying which library you are using, and I'll find the right site for you? For example: I'm using Los Angeles Public Library, so to get ODMs I go to lapl.overdrive.com (notice the abbreviation for the library's name, it's not always that simple but sometimes is), log in, click "My Account", choose "Loans", and on any specific audiobook title, click "Have Overdrive?", and you'll see a button appear, click it and you'll download the ODM. BTW you can open that in the Windows Overdrive app, but if you're using odmpy already it turns out you can just use odmpy to do all the downloading using the same file. See above for instructions. … -Wm

My local branch overdrive site hid the download link for the odm because I was using a hardware/software combo that the Overdrive for (win/mac) didn’t support. I had to change the user-agent string in Firefox about:config to an appropriate OS version and then the “download” button appeared again.

You can always go to https://www.overdrive.com..... in the top right there is a 'Find Library' link. Click that, and key in your zip and it'll find your library.

[jahlborn]

looks like some of the extensions have been fixed. just used the latest version of the firefox libby downloader extension and it worked great.

[wtanksleyjr]

Whatever's going on is kinda subtle - I've seen non-working apps with no fixes begin working again, and then stop working. It's possible there are some servers still running old code and if you get directed to them old apps work. OTOH, I don't know for sure. Maybe the breaking change is more subtle than it seems at first.

On Sun, Apr 28, 2024 at 1:39 PM James Ahlborn @.***> wrote:

looks like some of the extensions have been fixed. just used the latest version of the firefox libby downloader extension and it worked great.

— Reply to this email directly, view it on GitHub https://github.com/ping/odmpy/issues/69#issuecomment-2081646895, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJ7H6LOD3AYUZXHAJPGSETY7VM6JAVCNFSM6AAAAABGX6LV5SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBRGY2DMOBZGU . You are receiving this because you commented.Message ID: @.***>

[jahlborn]

I just did this and I noticed a couple things. The JWT in use actually changes at least 3 times. Also, the JWT is really weird. It's not a standard JWT format that I recognize. They are much longer and they have literally ... in the middle. It doesn't parse as a valid JWT.

@bckelly1 i did a little digging online and i saw some stuff mentioning "opaque" tokens for oauth2. for instance, auth0 has an opaque token which has two periods in the middle of it, which is not a valid jwt. not sure if that actually helps you in anyway, but just thought i'd throw it out there.

https://www.reddit.com/r/aws/comments/v16z4b/if_a_jwt_contains_two_consecutive_period/ https://community.auth0.com/t/why-my-token-has-2-dots-in-it/106186

[bckelly1]

I think there is a deeper change that will be required, though it looks easy maybe? I stumbled across this while looking for answers and it seems to mention similar topics. However, I don't see how the code actually fixed the problem so I'll have to dig a little deeper when I get some time.

[jahlborn]

some progress appears to have been made in #71

[jahlborn]

@bckelly1 have you been able to figure anything else out?

[bckelly1]

@bckelly1 have you been able to figure anything else out?

Not much, I haven't put a lot of effort into it. I did find that this workaround in the firefox plugin is working, but it is highly manual. In theory, this could work with odmpy but I have yet to make it work for me. They embedded the spine info into an image, but I must not understand the instructions because it hasn't worked for me.

[wtanksleyjr]

Tonight my odmpy libby just fell out of sync, and there doesn't seem to be a way to get the 8-digit-code to make it come back into sync. So from here on out, it's odm downloads only.

[j-antunes]

@wtanksleyjr - I tried it today and I was able to get the 8 digit code, by clicking on "Are you connecting sonos speakers"

[DaveAlton]

@wtanksleyjr - I tried it today and I was able to get the 8 digit code, by clicking on "Are you connecting sonos speakers"

Did this allow you to auto download your audiobooks?

[coloradoruss]

The libby audiobook download is broken. The Firefox and Chrome plugins are broken as well. The only thing that works is dl - I hope someone can get the libby thing working at some point, but at least we have something. You have to get the odm file from your library.overdrive.com, have overdrive..., download mp3... and run dl on the odm file. The command I use is odmpy dl -c -m -j -k -mergeformat m4b

Russ

Confidentiality Notice: This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this for the addressee, you must not use, copy, disclose, or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message.Non-Binding and Expiration Disclaimer: Content of emails, to include any included links and attachments thereto, sent or received shall not create a binding contract or be construed to be conveyance or acceptance of any offer, contract or any other agreement, whatsoever, nor shall the same be binding on any party until a written contract is signed by all parties. In the event a bona fide, signed offer is conveyed via email, the same shall expire, if not accepted and fully executed by all parties, upon the latter of its explicitly stated expiration date or, if such a date is not specified, twenty-four hours from its initial transmission by the offeror and shall thereafter not be binding on any party.

Wire Fraud Notice: Never trust wiring instructions sent via email. Cyber criminals are hacking email accounts and sending emails with fake wiring instructions. These emails are convincing and sophisticated. Always independently confirm wiring instructions in person or via a telephone call to a trusted and verified phone number. Never wire money without double-checking that the wiring instructions are correct.Affiliated Business Disclosure: I have an ownership interest in both Pink Realty, Inc and Pink Realty Rentals and work closely with both companies and their agents and employees, my real estate license and my brokerage activity is with Pink Realty Rentals.

On Sun, Aug 11, 2024 at 10:30 AM Dave of the North @.***> wrote:

@wtanksleyjr https://github.com/wtanksleyjr - I tried it today and I was able to get the 8 digit code, by clicking on "Are you connecting sonos speakers"

Did this allow you to auto download your audiobooks?

— Reply to this email directly, view it on GitHub https://github.com/ping/odmpy/issues/69#issuecomment-2282813100, or unsubscribe https://github.com/notifications/unsubscribe-auth/ATRQTMVPG3WW6ZWD2PAS67DZQ6GTPAVCNFSM6AAAAABGX6LV5SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOBSHAYTGMJQGA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[codyc1515]

@wtanksleyjr - I tried it today and I was able to get the 8 digit code, by clicking on "Are you connecting sonos speakers"

Same here. Looks like the way forward is using Passkeys.

[DaveAlton]

Same here. Looks like the way forward is using Passkeys.

For the record, I believe this ticket is to record that there's an issue auto-downloading audiobooks.

This solves an issue, but not the one originally ticketed.

[wtanksleyjr]

Yes, the "odmpy dl" subcommand still works (magnificently!), whew, but once I use the Sonos choice, so does the "odmpy libby --exportloans" command, which allows me to figure out which ODM file goes with which Libby metadata, generate detailed audiobookshelf data, warn if I haven't downloaded an ODM file for a book that's about to expire, and so on.

[j-antunes]

@wtanksleyjr - How are you getting the odm file?

[coloradoruss]

You have to get the odm file from your library.overdrive.com, have overdrive..., download mp3...

I think you can go to overdrive.com and search for your library's url. For example, TN is https://reads.overdrive.com/

Message ID: @.***>

[ziferius]

Definitely go to https://www.overdrive.com/libraries. There are folks that are passing around spreadsheets of URLs for libraries and it's not needed.

On Mon, Aug 12, 2024 at 11:54 AM coloradoruss @.***> wrote:

You have to get the odm file from your library.overdrive.com, have overdrive..., download mp3...

I think you can go to overdrive.com and search for your library's url. For example, TN is https://reads.overdrive.com/

Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/ping/odmpy/issues/69#issuecomment-2284496322, or unsubscribe https://github.com/notifications/unsubscribe-auth/AD3S2P4BJCWLCVCYPJXBWD3ZRDSCXAVCNFSM6AAAAABGX6LV5SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOBUGQ4TMMZSGI . You are receiving this because you commented.Message ID: @.***>

[jahlborn]

intersting, looks like someone made some progress here, i wonder if it is helpful to this project: https://github.com/bookbonobo/libby-download-extension/pull/240

[wtanksleyjr]

Libby sign-in is failing with a new error, blocking Sonos and Android choices now. It also seems to destroy existing signins, the error I see is a certificate authentication failure, Libby seems to be sending an HTTPS certificate that doesn't match the machine it's coming from. This is reported in #78 .

[wtanksleyjr]

Regarding #78 someone has posted a fix at https://github.com/jordkh/odmpy , this does not fix the major problem here but it does allow Libby login once again - you still have to manually download the odm file from your library's Loans page on overdrive.com, but you can again use Libby's list of checked out books.

[wtanksleyjr]

The command I just ran to install this was python3 -m pip install git+https://git@github.com/jordkh/odmpy --upgrade --force-reinstall

[jahlborn]

@wtanksleyjr can you clarify specifically what should be working now? being able to download files with odmpy given an odm file has been working the whole time.

[wtanksleyjr]

Logging in to Libby works again (that is, syncing your odmpy libby account with some other Libby machine), which means you can now get a list of checked-out books. I use that to get detailed metadata from Libby on each ODM book, as well as knowing when I checked out a book but forgot to download its ODM.

-Wm

On Sun, Oct 6, 2024 at 7:49 PM James Ahlborn @.***> wrote:

@wtanksleyjr https://github.com/wtanksleyjr can you clarify specifically what should be working now? being able to download files with odmpy given an odm file has been working the whole time.

— Reply to this email directly, view it on GitHub https://github.com/ping/odmpy/issues/69#issuecomment-2395784082, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJ7H6IBORGXKOTK4M3U6OLZ2HZFNAVCNFSM6AAAAABGX6LV5SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOJVG44DIMBYGI . You are receiving this because you were mentioned.Message ID: @.***>

[wtanksleyjr]

There are people working on making this work within browser extensions.

For example: https://github.com/bookbonobo/libby-download-extension/pull/240

I don't think we've actually lost yet - the problem is that nobody here has the expertise to work on the problem.

[Ritergeek]

As of today, Overdrive has disabled the option to Download in Overdrive for Windows format on the Overdrive site for my library and apparently all subscribing libraries. This apparently means odm files. Does this totally tank the Libby Downloader potential?

[wtanksleyjr]

It doesn't change anything about the Libby downloader - specifically, we haven't figured out how to get its login token. Once the login token is fetched (which currently can only be done in a web browser), the files downloaded are the same MP3 files we always received before.

However, for now odmpy can only do one thing: show you what files you have checked out (odmpy libby). Everything else is completely disabled.

[jahlborn]

https://github.com/ping/odmpy/issues/78#issuecomment-2486773994