Redirects ruining GET parameters using axios

[mlhomedev]

I am using axios to retrieve data from this API https://archives.nd.edu/cgi-bin/wordz.pl?keyword=input where input is a word that is passed through as a GET parameter, and the API messes with it on their end and spits out some HTML. Here is my axios fetch script.

let link = "https://archives.nd.edu/cgi-bin/wordz.pl";
let proxy = "https:/MyCors-AnywhereProxy.heroku.app/";
let input = "canis"; // for example
axios.get(proxy+link, { params: { keyword: input}}).then(resp => console.log(resp.data));

The problem that occurs is when I look in the console and see that the URL that actually got fetched was https://archives.nd.edu/cgi-bin/wordz.pl?keyword=input&keyword=input which causes the API to not work. When looking into the request in chrome dev tools, I found that in the request headers, x-cors-redirect-1 is what added this extra parameter. Is there a way to disable redirects in the cors anywhere server, or is my code incorrect somewhere? I will note that in python using request(), I can simply grab the HTML by adding the input onto the link +"?keyword=" and there are no redirects at all. Thank you for your time!

Edit: This whole project is in firebase, so if you need to recreate my environment to test the issue, I wouldn't mind deploying it and dropping a link.

[Rob--W]

To investigate this issue, you need to paste the exact request headers and response headers (+status code) of the original request + proxy response, and the original request + response without proxy. CORS Anywhere does not randomly create/follow redirects, it does what the client (browser) normally does.

[mlhomedev]

Sure thing! With Proxy:

Request URL: https://appname.herokuapp.com/http://archives.nd.edu/cgi-bin/wordz.pl?keyword=canis
Request Method: GET
Status Code: 200 OK
Remote Address: 54.243.129.215:443
Referrer Policy: strict-origin-when-cross-origin
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: content-type,server,x-xss-protection,content-security-policy,date,connection,content-length,x-final-url,access-control-allow-origin
Connection: keep-alive
Content-Length: 180
Content-Security-Policy: script-src 'self' http://archives.nd.edu style-src 'self' http://archives.nd.edu
Content-Type: text/html
Date: Thu, 20 Apr 2023 13:13:58 GMT
Server: Microsoft-IIS/10.0
Via: 1.1 vegur
X-Cors-Redirect-1: 302 https://archives.nd.edu/cgi-bin/wordz.pl?keyword=canis&keyword=canis
X-Final-Url: https://archives.nd.edu/cgi-bin/wordz.pl?keyword=canis&keyword=canis
X-Request-Url: http://archives.nd.edu/cgi-bin/wordz.pl?keyword=canis
X-Xss-Protection: 1; mode=block
Accept: application/json, text/plain, */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Connection: keep-alive
Host: appname.herokuapp.com
Origin: http://127.0.0.1:5002
Referer: http://127.0.0.1:5002/
sec-ch-ua: "Google Chrome";v="111", "Not(A:Brand";v="8", "Chromium";v="111"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Windows"
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: cross-site
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36

Without proxy:

Request URL: http://archives.nd.edu/cgi-bin/wordz.pl?keyword=canis
Request Method: GET
Status Code: 302 Redirect
Referrer Policy: strict-origin-when-cross-origin
Content-Length: 195
Content-Security-Policy: script-src 'self' http://archives.nd.edu style-src 'self' http://archives.nd.edu
Content-Type: text/html; charset=UTF-8
Date: Thu, 20 Apr 2023 13:16:52 GMT
Location: https://archives.nd.edu/cgi-bin/wordz.pl?keyword=canis&keyword=canis
Server: Microsoft-IIS/10.0
X-XSS-Protection: 1; mode=block
Accept: application/json, text/plain, */*
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9
Connection: keep-alive
Host: archives.nd.edu
Origin: http://127.0.0.1:5002
Referer: http://127.0.0.1:5002/
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36

I do note that the redirect also seems to happen without the proxy which is curious. I went and checked the old python request script, and it also seems to have this problem now, so I guess something in the website has changed. Turns out the problem is with the API. Thanks for the help!