search

nilaoda / N_m3u8DL-RE

Cross-Platform, modern and powerful stream downloader for MPD/M3U8/ISM. English/简体中文/繁體中文.
MIT License
3.87k stars 407 forks source link

Cannot download MPD contents, 403 Error Forbidden, likely not adding header "Range: bytes=xxxx-yyyy" #292

Open dendrite210 opened 9 months ago

dendrite210 commented 9 months ago

Output:

23:55:59.958 INFO : N_m3u8DL-RE (Beta version) 20230628
23:56:00.014 INFO : Loading URL: https://d1jobzhhm62zby.cloudfront.net/[uniqueid]/bk_tant_024144/19618221/cenc/g1/bk_tant_024144_22,32,64_v8.master.mpd?ss_sec=20&iss_sec=10&isc=1&use_token_based_signing=true
23:56:00.283 INFO : Content Matched: Dynamic Adaptive Streaming over HTTP
23:56:00.285 INFO : Parsing streams...
23:56:00.295 WARN : Writing meta json
23:56:00.297 INFO : Extracted, there are 2 streams, with 0 basic streams, 2 audio streams, 0 subtitle streams
23:56:00.299 INFO : Aud *CENC 1 | 65 Kbps | mp4a.40.2 | und | 2CH | 2 Segments | ~09h41m25s
23:56:00.300 INFO : Aud *CENC 0 | 32 Kbps | mp4a.40.2 | und | 1CH | 2 Segments | ~09h41m25s
23:56:03.545 INFO : Parsing streams...
23:56:03.547 INFO : Selected streams:
23:56:03.548 INFO : Aud *CENC 1 | 65 Kbps | mp4a.40.2 | und | 2CH | 2 Segments | ~09h41m25s
23:56:03.550 WARN : Writing meta json
23:56:03.551 INFO : Save Name: bk_tant_024144_22,32,64_v8.master
23:56:03.555 INFO : Start downloading...Aud 1 | 65 Kbps | mp4a.40.2 | und | 2CH
23:56:03.556 WARN : When CENC encryption is detected, binary merging is automatically enabled
23:56:03.563 WARN : Type: cenc
23:56:03.565 WARN : PSSH(WV): EhACtF7PJCdZonzFmCqemE4iEhDOp2oc2DGk7Vk21q9lsjWXGgdBdWRpYmxlIjdjaWQ6DQpBclJlenlRbldhSjh4WmdxbnBoT0lnPT0senFkcUhOZ3hwTzFaTnRhdlpiSTFsdz09
23:56:03.567 WARN : KID: 02b45ecf242759a27cc5982a9e984e22
23:56:03.581 WARN : Reading media info...
23:56:03.757 INFO : [0x1]: Audio, aac (mp4a), 62 kb/s
23:56:07.282 WARN : Response status code does not indicate success: 403 (Forbidden).
23:56:07.287 ERROR: Segment count check not pass, total: 3, downloaded: 2.
23:56:07.292 ERROR: Failed

The Following mpd loads, however, no files are downloaded because the server returns 403. I suspect it is because the "Range" Header is missing.

curl 'https://d1jobzhhm62zby.cloudfront.net/[unique-id]/bk_tant_024144/19618221/cenc/g1/bk_tant_024144_22,32,64_v8.master.mpd?ss_sec=20&iss_sec=10&isc=1&use_token_based_signing=true' \
  -H 'Accept: */*' \
  -H 'Accept-Language: en-US,en;q=0.9' \
  -H 'Connection: keep-alive' \
  -H 'Origin: https://www.audible.ca' \
  -H 'Referer: https://www.audible.ca/' \
  -H 'Sec-Fetch-Dest: empty' \
  -H 'Sec-Fetch-Mode: cors' \
  -H 'Sec-Fetch-Site: cross-site' \
  -H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36 Edg/117.0.2045.60' \
  -H 'sec-ch-ua: "Microsoft Edge";v="117", "Not;A=Brand";v="8", "Chromium";v="117"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-ch-ua-platform: "Windows"' \
  --compressed

From the MPD file... SegmentList seems to contain the pertinent information

<SegmentList timescale='22050' duration='215040'>
<Initialization sourceURL='bk_tant_024144_22_64.mp4?ss_sec=20&amp;iss_sec=10&amp;isc=1&amp;use_token_based_signing=true' range='0-1038'/>
<SegmentURL media='bk_tant_024144_22_64.mp4?ss_sec=20&amp;iss_sec=10&amp;isc=1&amp;use_token_based_signing=true' mediaRange='1039-83704'/>
</SegmentList>

-- and SegmentBase

<SegmentBase timescale='22050' indexRange='1039-22538' indexRangeExact='true' presentationTimeOffset='215040'>
<Initialization range='0-1038'/>
</SegmentBase>

MPD contents:

<?xml version='1.0' encoding='utf-8'?>
<MPD minBufferTime='PT10S' type='static' mediaPresentationDuration='PT9H41M25.054S' profiles='urn:mpeg:dash:profile:isoff-main:2011'
xmlns='urn:mpeg:dash:schema:mpd:2011'
xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xsi:schemaLocation='urn:mpeg:DASH:schema:MPD:2011 http://standards.iso.org/ittf/PubliclyAvailableStandards/MPEG-DASH_schema_files/DASH-MPD.xsd'>
<Period id='0' duration='PT9.752S'>
<AdaptationSet id='0' contentType='audio' lang='und' segmentAlignment='true' bitstreamSwitching='true'>
<Representation id='0' mimeType='audio/mp4' codecs='mp4a.40.2' bandwidth='32768' audioSamplingRate='22050'>
<AudioChannelConfiguration schemeIdUri='urn:mpeg:dash:23003:3:audio_channel_configuration:2011' value='1'/>
<ContentProtection xmlns:cenc='urn:mpeg:cenc:2013' cenc:default_KID='cea76a1c-d831-a4ed-5936-d6af65b23597' schemeIdUri='urn:mpeg:dash:mp4protection:2011' value='cenc'/>
<ContentProtection xmlns:cenc='urn:mpeg:cenc:2013' schemeIdUri='urn:uuid:edef8ba9-79d6-4ace-a3c8-27dcd51d21ed'>
<cenc:pssh>
AAAAhnBzc2gAAAAA7e+LqXnWSs6jyCfc1R0h7QAAAGYSEAK0Xs8kJ1mifMWYKp6YTiISEM6nahzYMaTtWTbWr2WyNZcaB0F1ZGlibGUiN2NpZDoNCkFyUmV6eVFuV2FKOHhaZ3FucGhPSWc9PSx6cWRxSE5neHBPMVpOdGF2WmJJMWx3PT0=
</cenc:pssh>
</ContentProtection>
<ContentProtection xmlns:cenc='urn:mpeg:cenc:2013' schemeIdUri='urn:uuid:9a04f079-9840-4286-ab92-e65be0885f95'>
<cenc:pssh>
AAAARHBzc2gBAAAAmgTweZhAQoarkuZb4IhflQAAAAICtF7PJCdZonzFmCqemE4izqdqHNgxpO1ZNtavZbI1lwAAAAA=
</cenc:pssh>
</ContentProtection>
<SegmentList timescale='22050' duration='215040'>
<Initialization sourceURL='bk_tant_024144_22_32.mp4?ss_sec=20&amp;iss_sec=10&amp;isc=1&amp;use_token_based_signing=true' range='0-1038'/>
<SegmentURL media='bk_tant_024144_22_32.mp4?ss_sec=20&amp;iss_sec=10&amp;isc=1&amp;use_token_based_signing=true' mediaRange='1039-44643'/>
</SegmentList>
</Representation>
<Representation id='1' mimeType='audio/mp4' codecs='mp4a.40.2' bandwidth='65536' audioSamplingRate='22050'>
<AudioChannelConfiguration schemeIdUri='urn:mpeg:dash:23003:3:audio_channel_configuration:2011' value='2'/>
<ContentProtection xmlns:cenc='urn:mpeg:cenc:2013' cenc:default_KID='02b45ecf-2427-59a2-7cc5-982a9e984e22' schemeIdUri='urn:mpeg:dash:mp4protection:2011' value='cenc'/>
<ContentProtection xmlns:cenc='urn:mpeg:cenc:2013' schemeIdUri='urn:uuid:edef8ba9-79d6-4ace-a3c8-27dcd51d21ed'>
<cenc:pssh>
AAAAhnBzc2gAAAAA7e+LqXnWSs6jyCfc1R0h7QAAAGYSEAK0Xs8kJ1mifMWYKp6YTiISEM6nahzYMaTtWTbWr2WyNZcaB0F1ZGlibGUiN2NpZDoNCkFyUmV6eVFuV2FKOHhaZ3FucGhPSWc9PSx6cWRxSE5neHBPMVpOdGF2WmJJMWx3PT0=
</cenc:pssh>
</ContentProtection>
<ContentProtection xmlns:cenc='urn:mpeg:cenc:2013' schemeIdUri='urn:uuid:9a04f079-9840-4286-ab92-e65be0885f95'>
<cenc:pssh>
AAAARHBzc2gBAAAAmgTweZhAQoarkuZb4IhflQAAAAICtF7PJCdZonzFmCqemE4izqdqHNgxpO1ZNtavZbI1lwAAAAA=
</cenc:pssh>
</ContentProtection>
<SegmentList timescale='22050' duration='215040'>
<Initialization sourceURL='bk_tant_024144_22_64.mp4?ss_sec=20&amp;iss_sec=10&amp;isc=1&amp;use_token_based_signing=true' range='0-1038'/>
<SegmentURL media='bk_tant_024144_22_64.mp4?ss_sec=20&amp;iss_sec=10&amp;isc=1&amp;use_token_based_signing=true' mediaRange='1039-83704'/>
</SegmentList>
</Representation>
</AdaptationSet>
</Period>
<Period id='1' duration='PT9H41M15.302S'>
<AdaptationSet id='0' contentType='audio' lang='und' subsegmentAlignment='true' bitstreamSwitching='true'>
<SupplementalProperty schemeIdUri='urn:mpeg:dash:period-continuity:2015' value='0'/>
<Representation id='0' mimeType='audio/mp4' codecs='mp4a.40.2' bandwidth='32768' audioSamplingRate='22050'>
<AudioChannelConfiguration schemeIdUri='urn:mpeg:dash:23003:3:audio_channel_configuration:2011' value='1'/>
<ContentProtection xmlns:cenc='urn:mpeg:cenc:2013' cenc:default_KID='cea76a1c-d831-a4ed-5936-d6af65b23597' schemeIdUri='urn:mpeg:dash:mp4protection:2011' value='cenc'/>
<ContentProtection xmlns:cenc='urn:mpeg:cenc:2013' schemeIdUri='urn:uuid:edef8ba9-79d6-4ace-a3c8-27dcd51d21ed'>
<cenc:pssh>
AAAAhnBzc2gAAAAA7e+LqXnWSs6jyCfc1R0h7QAAAGYSEAK0Xs8kJ1mifMWYKp6YTiISEM6nahzYMaTtWTbWr2WyNZcaB0F1ZGlibGUiN2NpZDoNCkFyUmV6eVFuV2FKOHhaZ3FucGhPSWc9PSx6cWRxSE5neHBPMVpOdGF2WmJJMWx3PT0=
</cenc:pssh>
</ContentProtection>
<ContentProtection xmlns:cenc='urn:mpeg:cenc:2013' schemeIdUri='urn:uuid:9a04f079-9840-4286-ab92-e65be0885f95'>
<cenc:pssh>
AAAARHBzc2gBAAAAmgTweZhAQoarkuZb4IhflQAAAAICtF7PJCdZonzFmCqemE4izqdqHNgxpO1ZNtavZbI1lwAAAAA=
</cenc:pssh>
</ContentProtection>
<BaseURL>../../../../base/bk_tant_024144/19618221/cenc/g1/bk_tant_024144_22_32.mp4?ss_sec=20&amp;iss_sec=10&amp;isc=1&amp;use_token_based_signing=true</BaseURL>
<SegmentBase timescale='22050' indexRange='1039-22538' indexRangeExact='true' presentationTimeOffset='215040'>
<Initialization range='0-1038'/>
</SegmentBase>
</Representation>
<Representation id='1' mimeType='audio/mp4' codecs='mp4a.40.2' bandwidth='65536' audioSamplingRate='22050'>
<AudioChannelConfiguration schemeIdUri='urn:mpeg:dash:23003:3:audio_channel_configuration:2011' value='2'/>
<ContentProtection xmlns:cenc='urn:mpeg:cenc:2013' cenc:default_KID='02b45ecf-2427-59a2-7cc5-982a9e984e22' schemeIdUri='urn:mpeg:dash:mp4protection:2011' value='cenc'/>
<ContentProtection xmlns:cenc='urn:mpeg:cenc:2013' schemeIdUri='urn:uuid:edef8ba9-79d6-4ace-a3c8-27dcd51d21ed'>
<cenc:pssh>
AAAAhnBzc2gAAAAA7e+LqXnWSs6jyCfc1R0h7QAAAGYSEAK0Xs8kJ1mifMWYKp6YTiISEM6nahzYMaTtWTbWr2WyNZcaB0F1ZGlibGUiN2NpZDoNCkFyUmV6eVFuV2FKOHhaZ3FucGhPSWc9PSx6cWRxSE5neHBPMVpOdGF2WmJJMWx3PT0=
</cenc:pssh>
</ContentProtection>
<ContentProtection xmlns:cenc='urn:mpeg:cenc:2013' schemeIdUri='urn:uuid:9a04f079-9840-4286-ab92-e65be0885f95'>
<cenc:pssh>
AAAARHBzc2gBAAAAmgTweZhAQoarkuZb4IhflQAAAAICtF7PJCdZonzFmCqemE4izqdqHNgxpO1ZNtavZbI1lwAAAAA=
</cenc:pssh>
</ContentProtection>
<BaseURL>../../../../base/bk_tant_024144/19618221/cenc/g1/bk_tant_024144_22_64.mp4?ss_sec=20&amp;iss_sec=10&amp;isc=1&amp;use_token_based_signing=true</BaseURL>
<SegmentBase timescale='22050' indexRange='1039-22538' indexRangeExact='true' presentationTimeOffset='215040'>
<Initialization range='0-1038'/>
</SegmentBase>
</Representation>
</AdaptationSet>
</Period>
</MPD>

Chunk 1 -H 'Range: bytes=1039-22538' from bk_tant_024144_22_32.mp4

curl 'https://d1jobzhhm62zby.cloudfront.net/[unique-id]/base/bk_tant_024144/19618221/cenc/g1/bk_tant_024144_22_32.mp4?ss_sec=20&iss_sec=10&isc=1&use_token_based_signing=true' \
  -H 'Accept: */*' \
  -H 'Accept-Language: en-US,en;q=0.9' \
  -H 'Connection: keep-alive' \
  -H 'Origin: https://www.audible.ca' \
  -H 'Range: bytes=1039-22538' \
  -H 'Referer: https://www.audible.ca/' \
  -H 'Sec-Fetch-Dest: empty' \
  -H 'Sec-Fetch-Mode: cors' \
  -H 'Sec-Fetch-Site: cross-site' \
  -H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36 Edg/117.0.2045.60' \
  -H 'sec-ch-ua: "Microsoft Edge";v="117", "Not;A=Brand";v="8", "Chromium";v="117"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-ch-ua-platform: "Windows"' \
  --compressed

Chunk 2 -H 'Range: bytes=1039-22538' from bk_tant_024144_22_64.mp4

curl 'https://d1jobzhhm62zby.cloudfront.net/[unique-id]/base/bk_tant_024144/19618221/cenc/g1/bk_tant_024144_22_64.mp4?ss_sec=20&iss_sec=10&isc=1&use_token_based_signing=true' \
...
  -H 'Range: bytes=1039-22538' \
...

Chunk 3 -H 'Range: bytes=0-1038' bk_tant_024144_22_64.mp4

curl 'https://d1jobzhhm62zby.cloudfront.net/[unique-id]/base/bk_tant_024144/19618221/cenc/g1/bk_tant_024144_22_64.mp4?ss_sec=20&iss_sec=10&isc=1&use_token_based_signing=true' \
...
  -H 'Range: bytes=0-1038' \
...

Chunk 4 -H 'Range: bytes=1039-83704' bk_tant_024144_22_64.mp4

curl 'https://d1jobzhhm62zby.cloudfront.net/[unique-id]/base/bk_tant_024144/19618221/cenc/g1/bk_tant_024144_22_64.mp4?ss_sec=20&iss_sec=10&isc=1&use_token_based_signing=true' \
...
  -H 'Range: bytes=1039-83704' \
...

Chunk 5 -H 'Range: bytes=0-1038' bk_tant_024144_22_64.mp4

curl 'https://d1jobzhhm62zby.cloudfront.net/[unique-id]/base/bk_tant_024144/19618221/cenc/g1/bk_tant_024144_22_64.mp4?ss_sec=20&iss_sec=10&isc=1&use_token_based_signing=true' \
...
  -H 'Range: bytes=0-1038' \
...

Chunk 6 -H 'Range: bytes=22539-187553' 

curl 'https://d1jobzhhm62zby.cloudfront.net/[unique-id]/base/bk_tant_024144/19618221/cenc/g1/bk_tant_024144_22_64.mp4?ss_sec=20&iss_sec=10&isc=1&use_token_based_signing=true' \
...
  -H 'Range: bytes=22539-187553' \
  ...

Chunk 7 -H 'Range: bytes=187554-352263' \

curl 'https://d1jobzhhm62zby.cloudfront.net/[unique-id]/base/bk_tant_024144/19618221/cenc/g1/bk_tant_024144_22_64.mp4?ss_sec=20&iss_sec=10&isc=1&use_token_based_signing=true' \
...
  -H 'Range: bytes=187554-352263' \
...

etc... bytes=352264-516669 bytes=516670-681471 bytes=681472-846046 bytes=846047-1010676

nilaoda commented 8 months ago

The Range request header refers to requesting a partial content of a resource file. The 403 status code is usually unrelated to this request header.

dendrite210 commented 8 months ago

What appears to be true for this server is that unless the range header is specified, the server returns 403.

Basically, I am unable to get N_m3u8DL-RE to download any file from the mpd.

On Sat., Oct. 14, 2023, 9:54 a.m. nilaoda, @.***> wrote:

The Range request header refers to requesting a partial content of a resource file. The 403 status code is usually unrelated to this request header.

— Reply to this email directly, view it on GitHub https://github.com/nilaoda/N_m3u8DL-RE/issues/292#issuecomment-1762994544, or unsubscribe https://github.com/notifications/unsubscribe-auth/AETQEUUCWSPE2CIX2Q7X323X7KYSNANCNFSM6AAAAAA5ZYPHM4 . You are receiving this because you authored the thread.Message ID: @.***>

hyww commented 1 month ago

@dendrite210 I've come up with a workaround.

I would like to know if the server in your case responses with Content-Range header in bytes <range-start>-<range-end>/<size> format?