Marekkon5 / onetagger

Music tagger for Windows, MacOS and Linux with Beatport, Discogs, Musicbrainz, Spotify, Traxsource and many other platforms support.
https://onetagger.github.io/
GNU General Public License v3.0
629 stars 34 forks source link

Traxsource Autotag causes 502 Bad Gateway - The server returned an invalid or incomplete response. #345

Open drsgfdds opened 8 months ago

drsgfdds commented 8 months ago

Using Autotag to pull genres from Traxsource results in 1 or 2 successfully tagged songs with everything else failing. After attempting to tag from Traxsource, the entire website traxsource.com is entirely unavailable from my computer, attempts to visit in web browser or from another computer result in a 502 Bad Gateway - The server returned an invalid or incomplete response.

Please advise. Relevant logs below. Thank you.

Starting OneTagger v1.7.0 Commit: ba4d5e03 OS: windows

2024-03-01 09:49:12 [INFO] onetagger_ui: Starting web server on: http://127.0.0.1:36913 2024-03-01 09:49:23 [WARN] onetagger_ui: Error loading album art: Missing extension File: 2024-03-01 10:03:11 [WARN] tao::platform_impl::platform::event_loop::runner: NewEvents emitted without explicit RedrawEventsCleared 2024-03-01 10:03:11 [WARN] tao::platform_impl::platform::event_loop::runner: RedrawEventsCleared emitted without explicit MainEventsCleared 2024-03-01 10:03:59 [WARN] onetagger_ui: Error loading album art: Missing extension File: 2024-03-01 10:04:24 [ERROR] onetagger_ui::socket: Websocket: EOF while parsing a value at line 1 column 0, Data: 2024-03-01 10:04:24 [WARN] onetagger_ui::socket: Failed sending to socket: WebSocket protocol error: Sending after closing is not allowed 2024-03-01 10:08:34 [WARN] tao::platform_impl::platform::event_loop::runner: NewEvents emitted without explicit RedrawEventsCleared 2024-03-01 10:08:34 [WARN] tao::platform_impl::platform::event_loop::runner: RedrawEventsCleared emitted without explicit MainEventsCleared 2024-03-01 10:13:01 [WARN] tao::platform_impl::platform::event_loop::runner: NewEvents emitted without explicit RedrawEventsCleared 2024-03-01 10:13:01 [WARN] tao::platform_impl::platform::event_loop::runner: RedrawEventsCleared emitted without explicit MainEventsCleared 2024-03-01 10:13:36 [INFO] onetagger_ui::socket: AutoTagger config: TaggerConfig { platforms: ["traxsource"], path: Some("D:\Music\Techno\Kelela\(2023) Kelela - Closure (Flexulant x BAMBII Remix feat. Rahrah Gabor + brazy)"), tags: [Style, Mood, MetaTags, Genre, URL], separators: TagSeparators { id3: "; ", vorbis: None, mp4: ", " }, id3v24: true, overwrite: false, overwrite_tags: [Genre, Style], threads: 24, strictness: 0.7, merge_genres: true, album_art_file: false, camelot: false, parse_filename: true, filename_template: Some("%track% - %artist% - %title%"), short_title: false, match_duration: true, max_duration_difference: 5, match_by_id: true, multiple_matches: Default, post_command: None, styles_options: MergeToGenres, styles_custom_tag: Some(FrameName { id3: "STYLE", vorbis: "STYLE", mp4: "STYLE" }), track_number_leading_zeroes: 0, enable_shazam: false, force_shazam: true, skip_tagged: false, include_subfolders: true, only_year: false, title_regex: Some(""), move_success: false, move_success_path: None, move_failed: false, move_failed_path: None, write_lrc: false, enhanced_lrc: false, capitalize_genres: true, id3_comm_lang: None, remove_all_covers: false, multiplatform: false, fetch_all_results: false, custom: PlatformTaggerConfig({}), spotify: None } 2024-03-01 10:13:36 [INFO] onetagger_autotag: Starting tagger with: 8 files! 2024-03-01 10:13:36 [INFO] onetagger_autotag: Starting tagging: 8 files, 24 threads! 2024-03-01 10:13:36 [INFO] onetagger_autotag: Tagging: "D:\Music\Techno\Anden\(2021) Anden - Rewind\01 - Anden - Rewind.mp3" 2024-03-01 10:13:36 [INFO] onetagger_autotag: Tagging: "D:\Music\Techno\Above & Beyond Group Therapy\(2024) Above & Beyond Group Therapy - Group Therapy 560\05 - SØNIN - Nightlife (ABGT560).mp3" 2024-03-01 10:13:36 [INFO] onetagger_autotag: Tagging: "D:\Music\Techno\VANO 3000\(2021) VANO 3000 - Running Away (Time)\03 - VANO 3000 - Running Away (Vocal Remix).mp3" 2024-03-01 10:13:36 [INFO] onetagger_autotag: Tagging: "D:\Music\Techno\Machinedrum\(2023) Machinedrum - 4#TRAX\01 - Machinedrum - CAPONE.mp3" 2024-03-01 10:13:36 [INFO] onetagger_autotag: Tagging: "D:\Music\Techno\Béatrice Dupont\(2023) Béatrice Dupont - Paradise\01 - Béatrice Dupont - Paradise.mp3" 2024-03-01 10:13:36 [INFO] onetagger_autotag: Tagging: "D:\Music\Techno\Above & Beyond Group Therapy\(2023) Above & Beyond Group Therapy - Group Therapy 559\11 - Leena Punks - No Way Out (ABGT559).mp3" 2024-03-01 10:13:36 [INFO] onetagger_autotag: Starting traxsource 2024-03-01 10:13:36 [INFO] onetagger_autotag: Tagging: "D:\Music\Techno\Django Django\(2021) Django Django - Glowing in the Dark\11 - Django Django - Glowing in the Dark.mp3" 2024-03-01 10:13:36 [INFO] onetagger_autotag: Tagging: "D:\Music\Techno\MOTi\(2023) MOTi - Can\'t Sleep (with PLANT)\01 - MOTi - Can\'t Sleep (with PLANT).mp3" 2024-03-01 10:13:36 [ERROR] onetagger_autotag: Matching error: No results! ("D:\Music\Techno\VANO 3000\(2021) VANO 3000 - Running Away (Time)\03 - VANO 3000 - Running Away (Vocal Remix).mp3") 2024-03-01 10:13:36 [INFO] onetagger_autotag: [traxsource] State: Error, Accuracy: None, Path: "D:\Music\Techno\VANO 3000\(2021) VANO 3000 - Running Away (Time)\03 - VANO 3000 - Running Away (Vocal Remix).mp3" 2024-03-01 10:13:36 [ERROR] onetagger_autotag: Matching error: No results! ("D:\Music\Techno\Above & Beyond Group Therapy\(2024) Above & Beyond Group Therapy - Group Therapy 560\05 - SØNIN - Nightlife (ABGT560).mp3") 2024-03-01 10:13:36 [INFO] onetagger_autotag: [traxsource] State: Error, Accuracy: None, Path: "D:\Music\Techno\Above & Beyond Group Therapy\(2024) Above & Beyond Group Therapy - Group Therapy 560\05 - SØNIN - Nightlife (ABGT560).mp3" 2024-03-01 10:13:36 [ERROR] onetagger_autotag: Matching error: No results! ("D:\Music\Techno\Above & Beyond Group Therapy\(2023) Above & Beyond Group Therapy - Group Therapy 559\11 - Leena Punks - No Way Out (ABGT559).mp3") 2024-03-01 10:13:36 [INFO] onetagger_autotag: [traxsource] State: Error, Accuracy: None, Path: "D:\Music\Techno\Above & Beyond Group Therapy\(2023) Above & Beyond Group Therapy - Group Therapy 559\11 - Leena Punks - No Way Out (ABGT559).mp3" 2024-03-01 10:13:36 [ERROR] onetagger_autotag: Matching error: No results! ("D:\Music\Techno\Béatrice Dupont\(2023) Béatrice Dupont - Paradise\01 - Béatrice Dupont - Paradise.mp3") 2024-03-01 10:13:36 [INFO] onetagger_autotag: [traxsource] State: Error, Accuracy: None, Path: "D:\Music\Techno\Béatrice Dupont\(2023) Béatrice Dupont - Paradise\01 - Béatrice Dupont - Paradise.mp3" 2024-03-01 10:13:36 [ERROR] onetagger_autotag: Matching error: No results! ("D:\Music\Techno\MOTi\(2023) MOTi - Can\'t Sleep (with PLANT)\01 - MOTi - Can\'t Sleep (with PLANT).mp3") 2024-03-01 10:13:36 [INFO] onetagger_autotag: [traxsource] State: Error, Accuracy: None, Path: "D:\Music\Techno\MOTi\(2023) MOTi - Can\'t Sleep (with PLANT)\01 - MOTi - Can\'t Sleep (with PLANT).mp3" 2024-03-01 10:13:36 [ERROR] onetagger_autotag: Matching error: No results! ("D:\Music\Techno\Django Django\(2021) Django Django - Glowing in the Dark\11 - Django Django - Glowing in the Dark.mp3") 2024-03-01 10:13:36 [INFO] onetagger_autotag: [traxsource] State: Error, Accuracy: None, Path: "D:\Music\Techno\Django Django\(2021) Django Django - Glowing in the Dark\11 - Django Django - Glowing in the Dark.mp3" 2024-03-01 10:13:36 [WARN] onetagger_autotag: Failed extending track: error sending request for url (https://www.traxsource.com/track/11584981/capone): connection closed before message completed 2024-03-01 10:13:36 [WARN] onetagger_autotag: Failed extending track: error sending request for url (https://www.traxsource.com/track/8384225/rewind): connection closed before message completed 2024-03-01 10:13:38 [INFO] onetagger_autotag: [traxsource] State: Ok, Accuracy: Some(1.0), Path: "D:\Music\Techno\Machinedrum\(2023) Machinedrum - 4#TRAX\01 - Machinedrum - CAPONE.mp3" 2024-03-01 10:13:38 [INFO] onetagger_autotag: [traxsource] State: Ok, Accuracy: Some(1.0), Path: "D:\Music\Techno\Anden\(2021) Anden - Rewind\01 - Anden - Rewind.mp3" 2024-03-01 10:13:38 [INFO] onetagger_autotag: Written failed songs to: C:\Users\steve\AppData\Roaming\OneTagger\OneTagger\config\runs\failed-1709306018721.m3u, successful to: C:\Users\steve\AppData\Roaming\OneTagger\OneTagger\config\runs\success-1709306018721.m3u 2024-03-01 10:13:38 [INFO] onetagger_ui::socket: Tagging finished, took: 2 seconds. 2024-03-01 10:15:35 [WARN] tao::platform_impl::platform::event_loop::runner: NewEvents emitted without explicit RedrawEventsCleared 2024-03-01 10:15:35 [WARN] tao::platform_impl::platform::event_loop::runner: RedrawEventsCleared emitted without explicit MainEventsCleared 2024-03-01 10:17:03 [WARN] tao::platform_impl::platform::event_loop::runner: NewEvents emitted without explicit RedrawEventsCleared 2024-03-01 10:17:03 [WARN] tao::platform_impl::platform::event_loop::runner: RedrawEventsCleared emitted without explicit MainEventsCleared

Marekkon5 commented 8 months ago

Hello, looks like a server side issue. Perhaps they finally implemented some sort of rate limiting or anti-onetagger measures.

rolly-dev commented 8 months ago

Hello, looks like a server side issue. Perhaps they finally implemented some sort of rate limiting or anti-onetagger measures.

May i suggest a guy on reddit who knows EXACTLY what to do about it? :)

mkgilmore commented 8 months ago

Hello, looks like a server side issue. Perhaps they finally implemented some sort of rate limiting or anti-onetagger measures.

May i suggest a guy on reddit who knows EXACTLY what to do about it? :)

I'd certainly be interested in hearing what to do about it! It's causing me some proper issues not being able to tag from Traxsource!

Also looks like the IP ban may be permanent? (Or certainly longer than a week for me currently, the website is still giving me the 502 Bad Gateway error - but I can get around that using a VPN).

rolly-dev commented 8 months ago

Hello, looks like a server side issue. Perhaps they finally implemented some sort of rate limiting or anti-onetagger measures.

May i suggest a guy on reddit who knows EXACTLY what to do about it? :)

I'd certainly be interested in hearing what to do about it! It's causing me some proper issues not being able to tag from Traxsource!

Also looks like the IP ban may be permanent? (Or certainly longer than a week for me currently, the website is still giving me the 502 Bad Gateway error - but I can get around that using a VPN).

It's not permanent, max 24 hours only as i know for sure. Obviously ip ban will be immediately applied again if you try to use OneTagger and app will start to hit Traxsource with bunch of simultaneous requests of certain type. Also as i know Traxsource is aware about problem and really interested in solving this issue, but there's no way to contact directly with Marekkon5 or at least they didn't find that way using public resources. So i saying that on their behalf and waiting for Marekkon5 :)

Marekkon5 commented 8 months ago

Hello, looks like a server side issue. Perhaps they finally implemented some sort of rate limiting or anti-onetagger measures.

May i suggest a guy on reddit who knows EXACTLY what to do about it? :)

I'd certainly be interested in hearing what to do about it! It's causing me some proper issues not being able to tag from Traxsource! Also looks like the IP ban may be permanent? (Or certainly longer than a week for me currently, the website is still giving me the 502 Bad Gateway error - but I can get around that using a VPN).

It's not permanent, max 24 hours only as i know for sure. Obviously ip ban will be immediately applied again if you try to use OneTagger and app will start to hit Traxsource with bunch of simultaneous requests of certain type. Also as i know Traxsource is aware about problem and really interested in solving this issue, but there's no way to contact directly with Marekkon5 or at least they didn't find that way using public resources. So i saying that on their behalf and waiting for Marekkon5 :)

what?

Marekkon5 commented 8 months ago

@Marekkon5 Please can you communiate with u/Murky_Squash3639 on reddit using your account - that's Traxsource guy, reddit is just as interim way of communication, so both of you won't expose your emails here.

I did not receive any Reddit DMs whatsoever.

Marekkon5 commented 8 months ago

@Marekkon5 Please can you communiate with u/Murky_Squash3639 on reddit using your account - that's Traxsource guy, reddit is just as interim way of communication, so both of you won't expose your emails here.

I did not receive any Reddit DMs whatsoever.

As i know he can't send you DM there, but he's available for chat/messages.

My clients don't support chat, so I don't get notified, will check when I am able to.

tr3v0rb commented 4 months ago

I know the Known Problems section it states Traxsource is intentionally slowed down, seems like by not enough. Each time I run it, it causes a bad gateway issue. Can get round the "ban" by using a VPN and then scraping 1 track at a time, but would it be possible for Onetagger to go slower for Traxsource so I don't have to do this? Cheers! Trev

Marekkon5 commented 4 months ago

@tr3v0rb use the latest Actions build, Traxsource should be slowed down there to match the limits.

Nickyg001 commented 2 months ago

I have the latest version 1.7 of OneTagger (I discovered and installed it yesterday for the first time) and I didn't know about the ban from Traxsource! So the problem hasn't been solved yet?

Marekkon5 commented 2 months ago

You should use latest GitHub actions commit for proper Traxsource rate limiting, sorry for the issues.

Spotify is also known for giving high rate limits, other platforms we haven't encountered issues (yet...).

With those two (Spotify, Traxsource) we recommend using latest Actions build, waiting current "ban" out and limiting threads to 1.

Nickyg001 commented 2 months ago

You should use latest GitHub actions commit for proper Traxsource rate limiting, sorry for the issues.

Spotify is also known for giving high rate limits, other platforms we haven't encountered issues (yet...).

With those two (Spotify, Traxsource) we recommend using latest Actions build, waiting current "ban" out and limiting threads to 1.

Could you help me with this? Where can I find the latest build of Actions?

I see an entry > Action but I don't understand how, what to download or how to install.. is it always a *.exe/zip file?

Why is this change (essential, which you apparently solved) not integrated into the software itself?

rolly-dev commented 2 months ago

I have the latest version 1.7 of OneTagger (I discovered and installed it yesterday for the first time) and I didn't know about the ban from Traxsource! So the problem hasn't been solved yet?

* Does the ban last 24 hours?

* Does it always last 24 hours or can the ban be increased or for life?

* Does it only happen for massive requests for TAGs?
  I scanned 80 files and it happened...

* If I use it with a few files at a time can the BAN happen?

* beatport Don't you have these IP ban 502 problems?

* Are there other similar problems known with other sites e.g. Drezer, Spotify...?

Does the ban last 24 hours? - Yes, it is. Does it always last 24 hours or can the ban be increased or for life? - Just last 24 hours. If you get banned you can send request to Traxsource support (please include your IP address) and they will remove ban ASAP.

Nickyg001 commented 2 months ago

Ho l'ultima versione 1.7 di OneTagger (l'ho scoperta e installata ieri per la prima volta) e non sapevo del ban da Traxsource! Quindi il problema non è stato ancora risolto?

* Does the ban last 24 hours?

* Does it always last 24 hours or can the ban be increased or for life?

* Does it only happen for massive requests for TAGs?
  I scanned 80 files and it happened...

* If I use it with a few files at a time can the BAN happen?

* beatport Don't you have these IP ban 502 problems?

* Are there other similar problems known with other sites e.g. Drezer, Spotify...?

Il ban dura 24 ore? - Sì, lo è. Dura sempre 24 ore o può essere aumentato o a vita? - Dura solo 24 ore. Se vieni bannato puoi inviare una richiesta al supporto Traxsource (includi il tuo indirizzo IP) e rimuoveranno il ban il prima possibile.

What do I tell Customer Service about my BAN? I was tagging a massive amount of files from your servers?? Is this legal? So theoretically if they remove my IP from the BAN by whitelisting me I should never get BAN again? This is frustrating.

rolly-dev commented 2 months ago

Ho l'ultima versione 1.7 di OneTagger (l'ho scoperta e installata ieri per la prima volta) e non sapevo del ban da Traxsource! Quindi il problema non è stato ancora risolto?

* Does the ban last 24 hours?

* Does it always last 24 hours or can the ban be increased or for life?

* Does it only happen for massive requests for TAGs?
  I scanned 80 files and it happened...

* If I use it with a few files at a time can the BAN happen?

* beatport Don't you have these IP ban 502 problems?

* Are there other similar problems known with other sites e.g. Drezer, Spotify...?

Il ban dura 24 ore? - Sì, lo è. Dura sempre 24 ore o può essere aumentato o a vita? - Dura solo 24 ore. Se vieni bannato puoi inviare una richiesta al supporto Traxsource (includi il tuo indirizzo IP) e rimuoveranno il ban il prima possibile.

What do I tell Customer Service about my BAN? I was tagging a massive amount of files from your servers?? Is this legal? So theoretically if they remove my IP from the BAN by whitelisting me I should never get BAN again? This is frustrating.

Yes, you can tell them that, becouse they actually know that fact, at least if you used the same IP address which has been used when you logged on their website last time ;) You are not first banned and you aren't last. That's happen from time to time, please feel free to contact with support if you do not want to wait 24 hours. Traxsource is aware about issue related to previous versions of OneTagger, and there's nothing wrong about using 3rd party software, like OneTagger. Real problem is about hundreds of stupid bots which harassing websites like Traxsource with literally millions idiotic search requests per day. You have to set some limits to protect yourself and your legit visitors, unless you are not a whale with infinite resourses.

Ban is based on your actual activity, so all you need to avoid ban in the future - to use updated version of OneTagger, on which Marekkon5 pointed on.

Nickyg001 commented 2 months ago

Ho l'ultima versione 1.7 di OneTagger (l'ho scoperta e installata ieri per la prima volta) e non sapevo del ban da Traxsource! Quindi il problema non è stato ancora risolto?

* Does the ban last 24 hours?

* Does it always last 24 hours or can the ban be increased or for life?

* Does it only happen for massive requests for TAGs?
  I scanned 80 files and it happened...

* If I use it with a few files at a time can the BAN happen?

* beatport Don't you have these IP ban 502 problems?

* Are there other similar problems known with other sites e.g. Drezer, Spotify...?

Il divieto dura 24 ore? - Sì, lo è. Dura sempre 24 ore o può essere aumentata oa vita? - Dura solo 24 ore. Se vieni bannato puoi inviare una richiesta al supporto Traxsource (includere il tuo indirizzo IP) e rimuoverà il ban il prima possibile.

Cosa dico al Servizio Clienti del mio BAN? Stavo taggando un'enorme quantità di file dai vostri server? È legale? Quindi teoricamente se rimuovono il mio IP dal BAN inserendomi nella whitelist non dovrei mai più ricevere un BAN? È frustrante.

Sì, puoi dirglielo, perché in realtà lo sanno, almeno se hai usato lo stesso indirizzo IP che è stato usato quando hai effettuato l'accesso al loro sito web l'ultima volta ;) Non sei il primo bannato e non sei l'ultimo. Succede di tanto in tanto, sentiti libero di contattare l'assistenza se non vuoi aspettare 24 ore. Traxsource è a conoscenza del problema relativo alle versioni precedenti di OneTagger e non c'è niente di sbagliato nell'usare software di terze parti, come OneTagger. Il vero problema riguarda centinaia di stupidi bot che molestano siti web come Traxsource con letteralmente milioni di richieste di ricerca idiote al giorno. Devi stabilire dei limiti per proteggere te stesso e i tuoi legittimi visitatori, a meno che tu non sia una balena con risorse infinite.

Il ban si basa sulla tua attività effettiva, quindi tutto ciò di cui hai bisogno per evitare di essere bannato in futuro è usare la versione aggiornata di OneTagger, su cui ha fatto riferimento Marekkon5.

I'll start by saying that I only discovered OneTagger yesterday and they've already banned me 🤣🤣🤣. I downloaded version 1.7, it seems to me that it can be downloaded from the site, I did not understand how and where to download this new version of OneTagger... And I do not understand why it is not implemented in the download Standard from site eg V.1.8 ! So with this version it would be possible to bypass the block?

Marekkon5 commented 2 months ago

@Nickyg001 1.8 is sadly not ready yet due to personal circumstances. However the development versions are available, and should be stable. We advise that in every issue.

You can download it from the Actions tab on top, click the latest build with a green checkmark, and on bottom find the files for your platform.

Nickyg001 commented 2 months ago

Even today after more than 24 hours (I think it's been even well over 48 hours) I still have the 502 Bad Gateway problem.

Are you sure it's a 24-hour IP block? Are you sure this problem is solved?

image

bascurtiz commented 2 months ago

@rolly-dev @Nickyg001 As rolly-dev, who is part of Traxsource was saying, I advice you to follow-up his suggestion:

If you get banned you can send request to Traxsource support (please include your IP address) and they will remove ban ASAP.

Nickyg001 commented 2 months ago

@rolly-dev @Nickyg001 Come ha detto rolly-dev, che fa parte di Traxsource, ti consiglio di seguire il suo suggerimento:

Se vieni bannato, puoi inviare una richiesta al supporto Traxsource (includere il tuo indirizzo IP) e loro rimuoveranno il ban il prima possibile.

OK I'll try. Can you give me the email address to write to since I can't access the site? Can I mention OneTagger as the software causing the BAN?

rolly-dev commented 2 months ago

@rolly-dev @Nickyg001 Come ha detto rolly-dev, che fa parte di Traxsource, ti consiglio di seguire il suo suggerimento:

Se vieni bannato, puoi inviare una richiesta al supporto Traxsource (includere il tuo indirizzo IP) e loro rimuoveranno il ban il prima possibile.

OK I'll try. Can you give me the email address to write to since I can't access the site? Can I mention OneTagger as the software causing the BAN?

Ban lasts exactly 24 hours, no one second more. Yes, you need to mention OneTagger as reason of request. https://traxsource.zendesk.com/hc/en-us/requests/new

Nickyg001 commented 2 months ago

Hi Nicky, Sorry for the trouble. Your IP was removed from the no access list. The continued use of OneTagger will block you again, as that app floods Traxsource servers with requests in a short period of time. This is a safety measure we have to have in place. I believe OneTagger is working on a solution, or may already have one.

OK after requesting support they unblocked me :)

But it seems that it is not an eternal thing.

You told me that with the Build present in Actions the problem was solved right? I am using the build 852512db. Technically how was the thing solved? Are there slowdowns in requesting / writing tags?

Thanks for all the help you are giving me... and in any case Onetagger is a fantastic software!!!

image

Marekkon5 commented 2 months ago

The latest actions build is capped to 1 thread internally. If we will still get issues from users, we will add artificial delays.