Keep getting server error

[Nightfirecat]

I'm seeing it intermittently, but it seems to work after attempting to refresh a couple times.

[michael-elbert]

Please review my pull request #115 , as I tried my best to get rid of the Yahoo API.

[Nightfirecat]

Is it specifically the Yahoo API that's failing?

[michael-elbert]

059 said here and here that the Yahoo API is what's causing the issue

[Nightfirecat]

Not reproducing this myself.

$ cat yql-test.sh && echo && yql-test.sh
#!/bin/bash
responseBody=''; \
for i in {1..10}
do
        # get response body (remove the "created" key which contains the current time)
        testResponseBody="$(
                curl 'https://query.yahooapis.com/v1/public/yql?format=json&=select%20*%20from%20xml%20where%20url=%22https://realmofthemadgodhrd.appspot.com/char/list%22' 2>/dev/null |
                sed -E 's|"created":"[^"]+",||'
        )"

        # set the base text for comparison if it isn't set already
        if [[ -z "$responseBody" ]]; then
                responseBody="$testResponseBody"
        fi

        # check to see if they're different
        if [[ "$responseBody" != "$testResponseBody" ]]; then
                echo 'Mismatch found!'
                echo "Initial response:\n" + \
                     "$responseBody"
                echo "New response:\n" + \
                     "$testResponseBody"
                echo "$responseBody" > responseBody.txt
                echo "$testResponseBody" > testResponseBody.txt
                exit 1
        fi
done; \
echo 'No mismatches found.'; \
exit 0
No mismatches found.

@atomizer Is there a reason we're using YQL in the first place? (besides XML -> JSON conversion) Is it a CORS concern?

[michael-elbert]

The realm API has a CORS header missing that prevents you from locally getting it (Access-Control-Allow-Origin). This prevents muledump from working because everyone just runs it straight from their file directory, and not from a web facing resource. The reason it works with the yahoo API is because it's loading from a web resource, not a local resource. With my fix I just incorporated Cors-Anywhere to load the API, and used x2js for the conversion of XML to JSON.

Also, apparently people were having issues with Yahoo providing inconsistent results or even not loading at all. I figure it would be better to load the response on the API (semi)directly and do the conversion client-sided, rather than letting a third party do it.

[Nightfirecat]

It might be nice to have Yahoo as a fallback (how reliable is Cors-Anywhere?). Eventually, it might be nice to run some kind of local proxy service to bypass the CORS requirement and the need for an external intermediate, but that's outside the scope of this particular issue.

[atomizer]

YQL was used because it was the only open and "trusted" CORS proxy. Also, it has very high rate limit, and not limited per server, only per client. On the other hand, here the developer of cors-anywhere says that the demo server is just that, a demo server. This might cause problems since I imagine muledump generates a decent amount of traffic. I found https://crossorigin.me - might be a better option?

[loveens]

I started getting this server error today all the time - at the moment I can't update any of my accounts. I get a response:

"loading data server error full response:{"count":0,"created":"2016-10-24T07:27:40Z","lang":"ru-RU","results":null}"

It happens both in Chrome and Firefox, latest versions of both browsers. What can I do to fix that?

[JohnBlackburne]

Started happening to me. Only noticed it today but it may be 2 or 3 days the last time I tried using it. Same error with both a Kongregate account and regular account. Safari on macOS.

[batof]

Can we set up a local server in our computer to get the xml and convert it to json?

[phil-el]

@Nightfirecat, your script miss a q before =select, what you get with it is a consistent error about that syntax error, anyway even after correcting that yql return a correct answer but always fail atm with an url ala ?guid=...yql is not reliable since a few weeks.

Erm, even your query return often '{"query":{"count":0,"lang":"en-US","results":null}}', anyway your script doesn't test if the api work but if you get the same response, so getting the same error is found as ok by your script.

[phil-el]

It works by forcing an ip for a working yql server, in linux I did 68.142.242.184 query.yahooapis.com No idea how to do that with windows, it's obviously ugly to do that as yahoo can change the server ip

[LordBunny]

I think it's happening to everyone because i got it aswell

[LordBunny]

today

[jerzabek]

For me @phil-el s sollution worked! Yeah it may be ugly, but nothing else is working right now (at least not for me). BTW for Windows you have to edit your hosts file (Win7 it's in c:/windows/system32/drivers/etc I'm not sure if it's the same in Win10), and add

68.142.242.184 query.yahooapis.com

at the end of the file. Then you save it Thanks

[Mrunibro]

Hi @JarzaGM . Could you elaborate on how to do this? I know very little about how all of this works. I went to the directory you said to go to ( C:\Windows\System32\drivers\etc). Which file here exactly do i put

68.142.242.184 query.yahooapis.com

at the end? here's a pic of all the files I have in that folder: https://gyazo.com/a04266d218d1ee11c4253d886b8c1eaa

Thanks

[LordBunny]

hosts he said it

[Mrunibro]

@LordBunny I am not a clever man. thanks

[Mrunibro]

Anyway just to confirm i'm doing it right (and for all those future googlers) this is how its supposed to look like? https://gyazo.com/504c5b92ce72aa33a4e1afd675493c33

EDIT: it is.

[toddw123]

@batof Thats what i did so i wouldnt have to worry about the yahoo api not working. I have muledump running on a web server on my local machine and then i also created a script that you pass the API url to and it will return the results in the format that muledump expects. So here is the script: http://pastebin.com/4m3Bk6fL. I wanted to make it return the same format and type of data the muledump already uses so now all you have to do is open the file "lib/realmapi.js", scroll down to where you see "url: 'https://query.yahooapis.com/v1/public/yql'" and change that you "url: 'URLTOTHESCRIPT'". And also change the line right below that currently says "q: 'select * from xml where url="' + url + '"'," to just read: "q: url,"

So thats it. Now my muledump will function off my machines local server and wont go down if the yahoo api is down.

[LordBunny]

@Mrunibro Does it work now?

[JohnBlackburne]

Works for me now, having added query.yahooapis.com to /etc/hosts

[loveens]

@JarzaGM thanks, your solution worked for me on Win7.

[XthEsimon]

@toddw123 The local server thing is not working for me right now. Would like to do it since i hope it will be faster than the yahoo api ^^

[toddw123]

@XthEsimon you have the php script i posted running on a server? Make sure you have that running somewhere (you can use apache2 on linux, or iis on windows, or whatever really), and then you redirect the javascript code to point to the script instead of the yahoo api.