Headphones Musicbrainz mirror no longer working

[Tuk]

I have been reeving this message for the last several days, does not seem to matter which group/album I search for. WARNING Attempt to query MusicBrainz for Dropkick Murphy s failed (caused by: )

When I switch to the normal musicbrainz server it works with no issue.

[AdeHub]

Look like the forum has gone too:

http://headphones.codeshy.com/forum/index.php

Rembo where are you?

[Tuk]

Yeah I noticed that too...

[Elfman72]

Here's to hoping this is temporary....

[dny238]

Rembo, happy to take over the server management to help you keep it running. It's what I do for a living. Message me back.,

[BbayFist]

Well he may have taken the money and run. All $100 or so. It seems his VIP service which cost $10 is gone as well. He may have been warned?

[kodek]

Give him a break - it's only been a few days, and people have lives outside the internet. In any case, if he actually disappeared, the VIP service is the least of our concern; someone else can easily take over the MB mirror task if necessary.

[BbayFist]

So did you just volunteer?

[dny238]

Seems easier to fix his existing solution than to setup another instance. We discussed direct musicbrainz traffic with musicbrainz and they throttle pretty aggressively. A mirror dedicated to this is the correct solution.

[AdeHub]

Agree it would be easier to fix the existing solution but Rembo hasn't been round these parts for quite some time and I think it would be a good idea to get an alternate mirror up and running

[dny238]

Sounds logical, he hasn't responding to my emails either. I'll get something setup and report back.

Unless someone else wants to take a stab at it.

[hamett]

Hope REMBO is ok. Hopefully on a beach somewhere.

[ghost]

Using the musicbrainz VM, it's surprisingly easy to set up your own mirror.

https://musicbrainz.org/doc/MusicBrainz_Server/Setup

[dny238]

Already have a server online I can use, so I am adding this to it. Harder than the vm no doubt. We will have it online in the next day or two.

We can load test with a few people here to see how it holds up.

[ghost]

Awesome. I tried a quick go at getting it to work natively on my 'buntu box, but the install wasn't going as smoothly as I was hoping. Scrapped it due to no time to figure it out and gave the VM a go the next day. After downloading it took minutes to get it running. Access control is my only issue right now, though I think I'll just do IP filtering to open it up to some friends.

[bresmith]

I'd be happy to help load test the mirror, dny238. Let me know how I can help.

[sammcj]

This is a real pain as headphones is now unusable. I'm going to setup my own local mirror but it kind of defeats the purpose.

[projectzero]

An alternative mirror would be great so I can get back to work on my library.

[sammcj]

I have a fix!

edit lib/musicbrainzngs/musicbrainz.py

change

    _useragent = "%s/%s python-musicbrainz-ngs/%s ( %s )" % (app, version, _version, contact)
else:
    _useragent = "%s/%s python-musicbrainz-ngs/%s" % (app, version, _version)

to something other than python-musicbrainz-ngs, I think headphones has been blacklisted from using musicbrainz API.

i.e.

    _useragent = "%s/%s i-really-like-fish-n-chips/%s ( %s )" % (app, version, _version, contact)
else:
    _useragent = "%s/%s i-really-like-fish-n-chips/%s" % (app, version, _version)

and everything will work again.

How long will this last before musicbrainz blacklists you / does something else to block headphones users? I don't know.

[kodek]

I just tried that and it was a no-go. I was still getting errors. I remember reading that they throttle by other parameters as well as user agent string. Either way, that would only be a temporary solution, as MB will eventually block it off.

I guess it's time to set up a community mirror; I'll see what can be done this week.

[sammcj]

Might be worth setting up a cheap VPS to do it if we can't find a sponsor.

I've had a really good run with ramdnode.com

Or take a look at:

http://serverbear.com/compare?Sort=Monthly+Cost&Order=asc&Server+Type=VPS&Monthly+Cost=-3000000&HDD=30000000000-&RAM=1000000000-&Bandwidth=-

Happy to pitch in for running costs.

On 5 March 2013 19:58, kodek notifications@github.com wrote:

I just tried that and it was a no-go. I was still getting errors. I remember reading that they throttle by other parameters as well as user agent string. Either way, that would only be a temporary solution, as MB will eventually block it off.

I guess it's time to set up a community mirror; I'll see what can be done this week.

— Reply to this email directly or view it on GitHubhttps://github.com/rembo10/headphones/issues/1031#issuecomment-14429661 .

[sammcj]

Great! Not sure what kind of disk size we'd need, the stock VM expands out to around 18GB, but I think it will grow once you download the replication data from the mirror. RAM should be OK with 1-2GB.

On 5 March 2013 20:10, Benjamin Hodgetts notifications@github.com wrote:

I work for Namesco/PoundHost/CheapVPS so I can probably get a VPS for us on the cheap. I'll check pricing when I get in today.

— Reply to this email directly or view it on GitHubhttps://github.com/rembo10/headphones/issues/1031#issuecomment-14430097 .

[TBallS]

I cant help technically, but im willing to make a small monthly donation for the mirror. i just hope that REMBO is ok and he is just afk.

[Diovarus]

This are stressfull days...

[rjgould]

Hope rembo is ok, when paired with beets this is a really good tool for figuring out at a glance which albums I needed to buy.

[LoLig]

Rembo I hope you're OK!!

Meanwhile I'm happy to try another mirror :+1: .....and donate if it's working ;)

[dny238]

This is much easier to setup with the VM than install on CentOS. Almost there guys. Thanks for the patience...

[bresmith]

Also tried getting VM setup on CentOS but ran into some hardware virtualization issues once I booted the VM.

I'm down for donating if there is a working mirror I could use.

[Maelstromeous]

Mine seems to be working, but has to be refreshed a few times in order to grab the album info. I managed to download a NZB of Pendulum however :-)

[Maelstromeous]

I've not changed a thing with Headphones, no custom servers or anything. Seems to be proper broke now though :-/

[rembo10]

Guys - so so sorry i dropped the ball on this. I had a situation i had to attend to and had literally zero time to do anything on headphones. Back in action now so im gonna get the mirror and forum back up and then will start taking a look at the backlog of issues and bug fixes.

Hopefully we can get this all back on track and get it working again

Thanks so much for the concern and for sticking with me through this.... It's very much appreciated

[Maelstromeous]

Welcome back. Heres to hoping headphones will be awesome once more :D

[rembo10]

Thank you. Hoping the same :-) On Mar 7, 2013 11:13 AM, "Maelstrome26" notifications@github.com wrote:

Welcome back. Heres to hoping headphones will be awesome once more :D

— Reply to this email directly or view it on GitHubhttps://github.com/rembo10/headphones/issues/1031#issuecomment-14542187 .

[holucon]

Hey rembo10, please let us know when the musicbrainz mirror is back up! :)

[cybrosis]

Great to see you back!

[sammcj]

Welcome back! Hope everything is ok.

[Freso]

@sammcj

I think headphones has been blacklisted from using musicbrainz API.

It isn't blacklisted, but it is heavily throttled. But, hey, maybe there actually is a reason for this? After all, MusicBrainz loves people using their data, and they think it's great that headphones is using it too, and definitely wants you guys to continue doing so. However, at one point headphones was taking up more than half (like, a good deal more than half - I can't remember the numbers right now) of the entire traffic to and from MusicBrainz. This resulted in MusicBrainz being slow for everyone - people using MusicBrainz Picard, beets, Jaikoz, morituri, and other programs and web sites using the web service - as well as people doing the actual data entry and editing on the site. We didn't think it was fair that one application should take up more than half of the traffic available to everybody, hence why headphones began getting throttled - and hence why we urged people to set up their own mirrors to sync against.

There's also a monetary aspect. Bandwidth isn't free, electricity isn't free, hardware isn't free. And while MusicBrainz has sponsors and some good deals, headphones traffic was getting us dangerously close to crossing the threshold where we would have had to spend a good deal more on bandwidth, to accommodate the amount of data being transferred in and out of the servers.

All in all, hacking your way around the headphones throttling is not a very solidary move, if you don't also donate the dough to MetaBrainz that will allow them to keep the service running at the level you want.

(Oh, and also: I'm not officially affiliated with Meta- or MusicBrainz, just a long-time contributor and volunteer. :))

[Cantafordit]

HI All, This is my first post here... I would love to get MB working with Headphones again so count me in for testing any new mirrors. I run HP on an UnRaid server, does anybody know if you can install a local mirror in UnRaid, like on the cache drive or something?

[Enverex]

Mirror is up on musicbrainz.xnode.org (port 80). It works for scraping but I'm having some issues getting the web part of it to work properly at the moment (the CSS is missing for some weird reason, had to merge it manually).

[skidelo]

Mirror works for me. Thanks Enverex! Glad your alive Rembo...

[flippedcracker]

Thank you so much Enverex. The mirror helps. My headphones link to mb broke half way through scanning my library, so I had a bunch of incomplete stuff. I can now finally finish it.

[Enverex]

No worries, the machine has 2.4TB of bandwidth and is only ~£5 so it's not like it's breaking the bank. 2.4TB should be enough bandwidth right?... right?

MusicBrainz running native rather than inside a VM so that should help speed things along as well.

[flippedcracker]

If possible, I would definitely set some kind of usage alarm. I don't know how much data each request uses.

[holucon]

Enverex any idea why its still saying blacklisted or VA?

http://puu.sh/2dGJE

Didnt find one single artist.

Edit: I fixed it by completely wiping out the old Headphones and running from a new dir/new install. Fresh install works perfect using Enverex's mirror. Thanks man!

[TBallS]

works perfectly Enverex & very fast

[JHeat]

Enverex, LEGIT. If you got a link for donations, lemme know.

[cybrosis]

@Enverex Thanks dude! Also willing to donate to the cause, just let us know.

[Redlake73]

Good to see you back @rembo10, was starting to think my digital world was about to collapse.

Now fix this shit! ;o)

[good-dr-robotnik]

@Enverex Please let us know if you want donations for bandwidth, you're doing God's work keeping us all afloat while rembo gets the main mirror back up and running.

[Redlake73]

Thanks for the mirror @Enverex, my full folder is slowly clearing :+1:

[Dream-Killer]

@Enverex thanks for the mirror!