search

yaronzz / Tidal-Media-Downloader

Download 'TIDAL' Music On Windows/Linux/MacOs (PYTHON/C#)
http://doc.yaronzz.com/post/tidal_dl_installation/
Apache License 2.0
3.38k stars 470 forks source link

[FEATURE] Implement application/dash+xml stream type #834

Closed nobodyexpectsahistorian closed 2 years ago

nobodyexpectsahistorian commented 2 years ago

Which tool tidal-dl/tidal-gui

Which feature Implement downloading the dash+xml streams for flac/losless and lossy files(?).

Description It seems most working api-keys nowadays only provide streaming via dash+xml. Tidal-dl should support that, since implementing this seems not to be too complicated. There is already an implementation out there https://github.com/Dniel97/orpheusdl-tidal . As a proof of concept I have copied the relevant code from orpheusdl-tidal into tidal-dl and it seems to work. See https://github.com/nobodyexpectsahistorian/Tidal-Media-Downloader

sramsay100 commented 2 years ago

+1 Yes please, This would be great.

DDKnoll commented 2 years ago

I was also working on this and have a simple implementation: It all works right now: https://github.com/DDKnoll/Tidal-Media-Downloader

Although, it looks like your version actually parses the segments from the manifest and downloads them directly. Mine just had ffmpeg do that work.

nobodyexpectsahistorian commented 2 years ago

I was also working on this and have a simple implementation: It all works right now: https://github.com/DDKnoll/Tidal-Media-Downloader

Although, it looks like your version actually parses the segments from the manifest and downloads them directly. Mine just had ffmpeg do that work.

Nice! That's right, my version actually parses the mpd-information and uses tidal-dl's download mechanism to get the segments and put them together. Your solution however is probably a bit more elegant since ffmpeg needs to be called anyways to fix the container file.

PatrykMis commented 2 years ago

@yaronzz: @DDKnoll's solution can be merged - it just works!

nobodyexpectsahistorian commented 2 years ago

@yaronzz: @DDKnoll's solution can be merged - it just works!

Sadly for me, it doesn't work. I do think it should, but for me ffmpeg always errors out. System: Debian 10.

Initially it says: Filename extension of 'xxx' is not a common multimedia extension, blocked for security reasons. If you wish to override this adjust allowed_extensions, you can set it to 'ALL' to allow all

After setting that override parameter, it still doesn't work, now saying: Failed to open an initialization section in playlist 0 [dash] Error when loading first fragment, playlist 0

EDIT: This doesn't seem to be the real cause of problems here. When changing to the directory with the temporary .mpd file and calling ffmpeg from there it works flawlessly. It rather seems that ffmpeg doesn't like (absolute) paths. I have no clue why this is and/or if this is just a special problem on my system regarding some characters in my directories that ffmpeg doesn't like …

EDIT2: One needs to set the download quality to MASTER for DDKnoll's fix to work. This will obviously not change the downloaded file if tidal only offers normal flac anyways. This way it works on macOS 10.15, on Debian 10 ffmpeg still errors out (at least for me).

p2mgrup commented 2 years ago

Do you know how to fix a problem with CountryCode parameter which is missing in that updated version @DDKnoll or @nobodyexpectsahistorian

nobodyexpectsahistorian commented 2 years ago

Do you know how to fix a problem with CountryCode parameter which is missing in that updated version @DDKnoll or @nobodyexpectsahistorian

I can't reproduce the bug on my system. However, as I wrote here, your are possibly using a no longer working api key.

baobabresearch commented 2 years ago

I was also working on this and have a simple implementation: It all works right now: https://github.com/DDKnoll/Tidal-Media-Downloader

Although, it looks like your version actually parses the segments from the manifest and downloads them directly. Mine just had ffmpeg do that work.

How I can install your code? Can you update .exe?

nobodyexpectsahistorian commented 2 years ago

How I can install your code? Can you update .exe?

If your are on GNU/Linux or a similar unix-like system I could give you some directions, however with Windows systems I have no idea. I have also no idea how to update the executable. Maybe @DDKnoll could help.

tmatzxzone commented 2 years ago

If your are on GNU/Linux or a similar unix-like system I could give you some directions,

Teach me edit: nvm just got it work now

iamdco commented 2 years ago

I was also working on this and have a simple implementation: It all works right now: https://github.com/DDKnoll/Tidal-Media-Downloader

Although, it looks like your version actually parses the segments from the manifest and downloads them directly. Mine just had ffmpeg do that work.

i use error [http @ 0x56362f817de0] Protocol 'http' not on whitelist 'file,crypto'!

vi tidal_dl/util.py process = ffmpeg.input(mpdPath, hide_banner=None, y=None).output(path, acodec='copy', loglevel='error').run(); to process = ffmpeg.input(mpdPath, hide_banner=None, y=None, protocol_whitelist="tcp,http,file,crypto").output(path, acodec='copy', loglevel='error').run();

error2 dash @ 0x555ad70159c0] Error when loading first fragment, playlist 0

pqrenan commented 2 years ago

I was also working on this and have a simple implementation: It all works right now: https://github.com/DDKnoll/Tidal-Media-Downloader Although, it looks like your version actually parses the segments from the manifest and downloads them directly. Mine just had ffmpeg do that work.

How I can install your code? Can you update .exe?

I need help too :(

nobodyexpectsahistorian commented 2 years ago

How I can install your code? Can you update .exe?

I need help too :(

If you are on GNU/Linux or have a unix-like system (like macOS) you could do the following: (Please note: Your mileage may vary, depending on the distro etc.)

A few notes:

I hope this helps for everyone who wants to use tidal-dl immediately without waiting for an official update, however I can not guarantee for anything ;)

SageSyS commented 2 years ago

@nobodyexpectsahistorian I've tried both OrpheusDL-Tidal and @DDKnoll 's Pull request and the difference is that reported bitrates are higher on OrpheusDL (as well as the file size by the whole megabyte) but when I compare audio files (spectrums) - they're 100% the same, any explanation?

nobodyexpectsahistorian commented 2 years ago

@nobodyexpectsahistorian I've tried both OrpheusDL-Tidal and @DDKnoll 's Pull request and the difference is that reported bitrates are higher on OrpheusDL (as well as the file size by the whole megabyte) but when I compare audio files (spectrums) - they're 100% the same, any explanation?

Good question! Short answer: I have no clue …

I can absolutely see what your are talking about. We can produce three "different" files: 1. Downloaded by orpheusdl: biggest in file size, highest bitrate 2. Downloaded by tidal-dl with orpheus-code to parse the mpd-information and 3: Downloaded with tidal-dl and DDKnoll's code using ffmpeg for essentially everything. The last two files are nearly identical, although DDKnoll's files are usually slightly (~100Bytes) smaller then files produced by method 2. The bitrate is generally identical for 2 and 3.

I tend to think that the differences in file size and bitrate originate from the ffmpeg-conversion and/or the way tags and cover art is added since the source files after downloading and concatenation are bit-identical for method 1 and 2 but the end result is quite different. For method 3 there is no original source file since ffmpeg not only downloads it but does the conversion in the same step, so no luck comparing there. Whether or not these differences are of any concern regarding audio quality is beyond my understanding. I have also not enough experience with ffmpeg or audio/video-files-formats in general to understand what happens there "behind the curtain" and why the results can differ.

One other note: These files downloaded with any of the "new" methods all differ from the files that were previously directly available from tidal, although files downloaded via methods 2/3 generally only differ at ~100 bytes in file size and are (nearly) identical in bitrate.

I am not sure, if this helps …

haileypresto commented 2 years ago

which option should I set so that I can use my non-Hi-Fi Plus account IMG_20220204_112614

nobodyexpectsahistorian commented 2 years ago

which option should I set so that I can use my non-Hi-Fi Plus account IMG_20220204_112614

I don't have a non-Hi-Fi-Plus account, so I am not sure, but for my tests, I always answered "no" to the first question (mobile session), since it seems, this is only necessary for atmos files. After that it worked just the way tidal-dl works.

haileypresto commented 2 years ago

qué opción debo configurar para poder usar mi cuenta que no es Hi-Fi PlusIMG_20220204_112614

No tengo una cuenta que no sea Hi-Fi-Plus, así que no estoy seguro, pero para mis pruebas, siempre respondí "no" a la primera pregunta (sesión móvil), ya que parece que esto solo es necesario para archivos atmos. Después de eso, funcionó de la misma manera que lo hace tidal-dl.

which option should I set so that I can use my non-Hi-Fi Plus account IMG_20220204_112614

I don't have a non-Hi-Fi-Plus account, so I am not sure, but for my tests, I always answered "no" to the first question (mobile session), since it seems, this is only necessary for atmos files. After that it worked just the way tidal-dl works.

i got the same result. It seems that if I don't have a Hi-Fi Plus account it won't work. IMG_20220204_114852

nobodyexpectsahistorian commented 2 years ago

i got the same result. It seems that if I don't have a Hi-Fi Plus account it won't work.

Could you just answer "yes" to create a new session and try from there on? If your are using OrpheusDl for the first time, you may actually need to create this new session regardless of your account type. Sorry if I messed up there. However for more help regarding OrpheusDl you should probably ask on their pages …

nobodyexpectsahistorian commented 2 years ago

I was also working on this and have a simple implementation: It all works right now: https://github.com/DDKnoll/Tidal-Media-Downloader Although, it looks like your version actually parses the segments from the manifest and downloads them directly. Mine just had ffmpeg do that work.

i use error [http @ 0x56362f817de0] Protocol 'http' not on whitelist 'file,crypto'!

vi tidal_dl/util.py process = ffmpeg.input(mpdPath, hide_banner=None, y=None).output(path, acodec='copy', loglevel='error').run(); to process = ffmpeg.input(mpdPath, hide_banner=None, y=None, protocol_whitelist="tcp,http,file,crypto").output(path, acodec='copy', loglevel='error').run();

error2 dash @ 0x555ad70159c0] Error when loading first fragment, playlist 0

For your information: I encountered similar errors as I wrote here. I couldn't fix them either, however I came to believe that probably the ffmpeg version plays a significant role and that you probably need to have at least ffmpeg 4.3.x or newer for this to work …

haileypresto commented 2 years ago

Thanks for the help. whenever I hit Y I get the same message that I don't have a Hi-Fi Plus account. I will follow your recommendation. Have a nice day

iamdco commented 2 years ago

To:nobodyexpectsahistorian my ffmpeg ver is low 3.xx add protocol_whitelist="tcp,http,file,crypto" can download music

Great analysis, thanks. about size. I think it might have something to do with ID3v2 tags. Album cover images may vary in size. can use https://kid3.kde.org/ compare

SageSyS commented 2 years ago

Thanks for the help. whenever I hit Y I get the same message that I don't have a Hi-Fi Plus account. I will follow your recommendation. Have a nice day

Copy differences from this config

{
    "global": {
        "general": {
            "download_path": "./downloads/",
            "download_quality": "hifi",
            "search_limit": 10
         },
        "artist_downloading": {
            "return_credited_albums": true,
            "separate_tracks_skip_downloaded": true
        },
        "formatting": {
            "album_format": "{name}{explicit}",
            "playlist_format": "{name}{explicit}",
            "track_filename_format": "{track_number}. {name}",
            "single_full_path_format": "{name}",
            "enable_zfill": true
        },
        "codecs": {
            "proprietary_codecs": false,
            "spatial_codecs": true
        },
        "module_defaults": {
            "lyrics": "default",
            "covers": "default",
            "credits": "default"
        },
        "lyrics": {
            "embed_lyrics": true,
            "save_synced_lyrics": true
        },
        "covers": {
            "main_compression": "high",
            "main_resolution": 1400,
            "save_external": false,
            "external_format": "png",
            "external_compression": "low",
            "external_resolution": 3000,
            "save_animated_cover": true
        },
        "advanced": {
            "advanced_login_system": true,
            "codec_conversions": {
                "alac": "flac"
            },
            "conversion_flags": {
                "flac": {
                    "compression_level": "5"
                }
            },
            "conversion_keep_original": false,
            "cover_variance_threshold": 8,
            "debug_mode": false,
            "disable_subscription_checks": true,
            "enable_undesirable_conversions": false,
            "ignore_existing_files": false,
            "ignore_different_artists": true
        }
    },
    //I deleted extensions on purpose, YOU DON'T DELETE THEM
}
haileypresto commented 2 years ago

Gracias por la ayuda. cada vez que presiono Y recibo el mismo mensaje de que no tengo una cuenta Hi-Fi Plus. Seguiré tu recomendación. Que tenga un lindo día

Copie las diferencias de esta configuración

{
     "global" : {
         "general" : {
             "download_path" : " ./downloads/ " ,
             "download_quality" : " hifi " ,
             "search_limit" : 10
         },
        "artist_downloading" : {
             "return_credited_albums" : verdadero ,
             "separate_tracks_skip_downloaded" : verdadero
        },
        "formato" : {
             "formato_álbum" : " {nombre}{explícito} " ,
             " formato_lista_reproducción" : " {nombre}{explícito} " ,
             "formato_nombre_archivo_pista" : " {número_pista}. {nombre} " ,
             "formato_ruta_completa_única" : " {nombre} " ,
             "enable_zfill" : verdadero
        },
        "códecs" : {
             " códecs_propietarios" : falso ,
             "códecs_espaciales" : verdadero
        },
        "module_defaults" : {
             "letra" : " predeterminado " ,
             "covers" : " predeterminado " ,
             "créditos" : " predeterminado "
        },
        "letra" : {
             "embed_lyrics" : verdadero ,
             "save_synced_lyrics" : verdadero
        },
        "covers" : {
             "main_compression" : " high " ,
             "main_solution" : 1400 ,
             "save_external" : false ,
             "external_format" : " png " ,
             "external_compression" : " low " ,
             "external_solution" : 3000 ,
             "save_animated_cover" : cierto
        },
        "avanzado" : {
             "advanced_login_system" : verdadero ,
             "codec_conversions" : {
                 "alac" : " flac "
            },
            "conversion_flags" : {
                 "flac" : {
                     "compression_level" : " 5 "
                }
            },
            "conversion_keep_original" : falso ,
             "cover_variance_threshold" : 8 ,
             "debug_mode" : falso ,
             "disable_subscription_checks" : verdadero ,
             "enable_undeirable_conversions" : falso ,
             "ignorar_archivos_existentes" : falso ,
             "ignorar_diferentes_artistas" : verdadero
        }
    },
    //Eliminé extensiones a propósito, NO LAS ELIMINES 
}

thank you. works perfectly

UnholyAngel4YU commented 2 years ago

I have an error 2 when downloading a file https://github.com/yaronzz/Tidal-Media-Downloader/issues/834#issuecomment-1029464775

pheizaut commented 2 years ago

Got it working on windows using nobodyexpectsahistorian's guide and his version just had to switch source env/bin/activate to env\scripts\activate. It gives error: [ERR] Download failed! Songname ([WinError 2] The system cannot find the file specified) but songs still get downloaded as m4a and untagged. FFMPEG error I guess. Welp still got the audio.

win guide

  1. install python
  2. download/clone the repo
  3. open cmd in Tidal-Media-Downloader
  4. python -m venv env and env\scripts\activate
  5. cd tidaldl-py
  6. pip3 install -r requirements.txt
  7. python setup.py develop
  8. tidal-dl
ScubaAlex-WPG commented 2 years ago

So I was able to compile it. I am currently running it from \env\scripts. Added ffmpeg.exe in the same directory. I am able to download FLAC, but not MQA. Any ideas? image

evertinker commented 2 years ago

followed this and worked for me on win10 (but i don't get the error you mention).

also, more specifically it's .\env\Scripts\activate

ScubaAlex-WPG commented 2 years ago

compiled again, same thing.... Is there any way I can run some sort of debug mode?

evertinker commented 2 years ago

sorry I was trying to reply (github noob) to the post previous to yours. I used the current yaronzz release to download master quality tracks and nobodyexpectsahistorian's for non-master. I get the same error as you ScubaAlex-WPG.

SageSyS commented 2 years ago

So I was able to compile it. I am currently running it from \env\scripts. Added ffmpeg.exe in the same directory. I am able to download FLAC, but not MQA. Any ideas? image

Why'd you even download MQA? It's lossy and it sucks.

pqrenan commented 2 years ago

Got it working on windows using nobodyexpectsahistorian's guide and his version just had to switch source env/bin/activate to env\scripts\activate. It gives error: [ERR] Download failed! Songname ([WinError 2] The system cannot find the file specified) but songs still get downloaded as m4a and untagged. FFMPEG error I guess. Welp still got the audio.

win guide

  1. install python
  2. download/clone the repo
  3. open cmd in Tidal-Media-Downloader
  4. python -m venv env and env\scripts\activate
  5. cd tidaldl-py
  6. pip3 install -r requirements.txt
  7. python setup.py develop
  8. tidal-dl

Download ffmpeg.exe and put into your Python's Scripts directory (C:/Users/{yourusername}/appdata/local/programs/python/python310/scripts). Its working for me now.

ScubaAlex-WPG commented 2 years ago

@SageSyS , is there a way I can force the files to be downloaded as flacs? If I am trying to download a master version, I will get it in MQA.

queen4me commented 2 years ago

Is it true that the "FLAC-Version" of albums available in MQA is recoded MQA-crap and not the "real thing" FLAC?

PatrykMis commented 2 years ago

@queen4me It's not a rule of thumb. The best thing is to check files of interest via Audiochecker or command line aucdtect

PatrykMis commented 2 years ago

@SageSyS FLAC has different encoding levels from 1 up to 8: 1 means fastest and 8 means slowest. There is no difference in quality. Also metadata (tags, album art) makes difference in size. The best way is to decompress (flac -d <filename>) and then compare them; you probably get the same files, even comparing their checksums by MD5 or even SHA3-512 hashes. Alternatively, perfectly align them in wave editor and invert one of them - you'll see differences if there are any.

queen4me commented 2 years ago

@queen4me It's not a rule of thumb. The best thing is to check files of interest via Audiochecker or command line aucdtect

But how exactly can I make shure that the FLAC is reencoded (not compressed) CD/HiFi-Quality and not reencoded MQA? With Audiochecker you mean the tool Lossless Audio Checker? Thanks a lot for helping me out.

PatrykMis commented 2 years ago

@queen4me http://tausoft.org/wiki/True_Audio_Checker_Overview https://losslessaudiochecker.com/ Check those 2 links and you find the answer to your questions. If someone encode track to any lossless format (mp3, ogg etc.) and transcode it to flac or wave, this software tell it you for sure.

github-actions[bot] commented 2 years ago

Stale issue message