kurisubrooks / tfnsw-pids

Sydney Trains Passenger Information Display (PIDs)
15 stars 3 forks source link

Example CORS Proxy #3

Open Anon-4 opened 2 years ago

Anon-4 commented 2 years ago

Hi there (again lmao),

I've managed to get the program running but upon opening localhost:3000, it is returning an error: Unhandled Rejection (TypeError): Cannot read properties of undefined (reading 'departures').

Screen Shot 2022-04-28 at 4 07 02 pm
kurisubrooks commented 2 years ago

Can you show me how you set up the proxy?

Anon-4 commented 2 years ago

I'm currently hosting a CORS proxy on my own machine using Cors-Anywhere.

My config.js file for tfnsw-pid is:

const config = {
  proxy: {
    url: '127.0.0.1:8080',
    fetchOptions: {}
  }
};

export default config;

Here's the code for the CORS proxy for your reference (server.js from Cors-Anywhere):

// Listen on a specific host via the HOST environment variable
var host = process.env.HOST || '0.0.0.0';
// Listen on a specific port via the PORT environment variable
var port = process.env.PORT || 8080;

// Grab the blacklist from the command-line so that we can update the blacklist without deploying
// again. CORS Anywhere is open by design, and this blacklist is not used, except for countering
// immediate abuse (e.g. denial of service). If you want to block all origins except for some,
// use originWhitelist instead.
var originBlacklist = parseEnvList(process.env.CORSANYWHERE_BLACKLIST);
var originWhitelist = parseEnvList(process.env.CORSANYWHERE_WHITELIST);
function parseEnvList(env) {
  if (!env) {
    return [];
  }
  return env.split(',');
}

// Set up rate-limiting to avoid abuse of the public CORS Anywhere server.
var checkRateLimit = require('./lib/rate-limit')(process.env.CORSANYWHERE_RATELIMIT);

var cors_proxy = require('./lib/cors-anywhere');
cors_proxy.createServer({
  originBlacklist: originBlacklist,
  originWhitelist: originWhitelist,
  requireHeader: ['origin', 'x-requested-with'],
  checkRateLimit: checkRateLimit,
  removeHeaders: [
    'cookie',
    'cookie2',
    // Strip Heroku-specific headers
    'x-request-start',
    'x-request-id',
    'via',
    'connect-time',
    'total-route-time',
    // Other Heroku added debug headers
    // 'x-forwarded-for',
    // 'x-forwarded-proto',
    // 'x-forwarded-port',
  ],
  redirectSameOrigin: true,
  httpProxyOptions: {
    // Do not add X-Forwarded-For, etc. headers, because Heroku already adds it.
    xfwd: false,
  },
}).listen(port, host, function() {
  console.log('Running CORS Anywhere on ' + host + ':' + port);
});

Lmao I had a feeling it had something to do with the proxy. I geniunely had no idea how to use a CORS proxy as I've never used or hosted one before and I also didn't really know what to put in the config file lmaoooo.

Thanks so much for your help!

kurisubrooks commented 2 years ago

Your server example should work just fine so I'm not sure. Try adding some logging to the listen function in the server to see what's being passed through and tweak from there I guess. Sorry I can't help further.

Anon-4 commented 2 years ago

That's Okay. I'll probably hold it off for now.

If you don't mind me asking, when you originally created the program, what CORS proxy did you use?

kurisubrooks commented 2 years ago

I wrote my own. Should do the same thing, just takes the URL and spits outs the contents in the response. https://github.com/kurisubrooks/sherlock2/blob/master/api/proxy/cors.js

msoffice95 commented 5 months ago

That's Okay. I'll probably hold it off for now.

If you don't mind me asking, when you originally created the program, what CORS proxy did you use?

Did you get it working? Same issues not really sure why it needs a cors proxy seems to be a rare use case

kurisubrooks commented 5 months ago

The CORS proxy was because it makes the requests to another web service from the browser, which is heavily restricted by CORS as I don't own the service that I'm using (despite having permission to use it). I have plans to rewrite this app as a Next.js web app which can make use of React Server Components so a CORS proxy may no longer be necessary in the future. No ETA however.

JonA02 commented 5 months ago

@msoffice95 nah never did unfortunately. I ultimately gave up cause I couldn't fix the problem at all HAHAHA

But yeah @kurisubrooks that sounds good!!! Can't wait for it!

mrpresidentjk commented 2 months ago

I'm currently hosting a CORS proxy on my own machine using Cors-Anywhere.

My config.js file for tfnsw-pid is:

const config = {
  proxy: {
    url: '127.0.0.1:8080',
    fetchOptions: {}
  }
};

export default config;

Here's the code for the CORS proxy for your reference (server.js from Cors-Anywhere):

// Listen on a specific host via the HOST environment variable
var host = process.env.HOST || '0.0.0.0';
// Listen on a specific port via the PORT environment variable
var port = process.env.PORT || 8080;

// Grab the blacklist from the command-line so that we can update the blacklist without deploying
// again. CORS Anywhere is open by design, and this blacklist is not used, except for countering
// immediate abuse (e.g. denial of service). If you want to block all origins except for some,
// use originWhitelist instead.
var originBlacklist = parseEnvList(process.env.CORSANYWHERE_BLACKLIST);
var originWhitelist = parseEnvList(process.env.CORSANYWHERE_WHITELIST);
function parseEnvList(env) {
  if (!env) {
    return [];
  }
  return env.split(',');
}

// Set up rate-limiting to avoid abuse of the public CORS Anywhere server.
var checkRateLimit = require('./lib/rate-limit')(process.env.CORSANYWHERE_RATELIMIT);

var cors_proxy = require('./lib/cors-anywhere');
cors_proxy.createServer({
  originBlacklist: originBlacklist,
  originWhitelist: originWhitelist,
  requireHeader: ['origin', 'x-requested-with'],
  checkRateLimit: checkRateLimit,
  removeHeaders: [
    'cookie',
    'cookie2',
    // Strip Heroku-specific headers
    'x-request-start',
    'x-request-id',
    'via',
    'connect-time',
    'total-route-time',
    // Other Heroku added debug headers
    // 'x-forwarded-for',
    // 'x-forwarded-proto',
    // 'x-forwarded-port',
  ],
  redirectSameOrigin: true,
  httpProxyOptions: {
    // Do not add X-Forwarded-For, etc. headers, because Heroku already adds it.
    xfwd: false,
  },
}).listen(port, host, function() {
  console.log('Running CORS Anywhere on ' + host + ':' + port);
});

Lmao I had a feeling it had something to do with the proxy. I geniunely had no idea how to use a CORS proxy as I've never used or hosted one before and I also didn't really know what to put in the config file lmaoooo.

Thanks so much for your help!

I am not sure if this helps, but fyi I helped my mate out with getting this up and running. There were two things that I had to tweak in your setup:

1) Change the proxy url from 127.0.0.1:8080 to http://127.0.0.1:8080 - without specifying protocol the web app will treat the proxy url as a relative path which you can see in the network tab in chrome 2) If you have done the above correctly, proxy anywhere should now be receiving the requests. Because the author used his own bespoke proxy, the url used to make the requests will not be formatted correctly. You will have to update the source code to get it to work. Replace this line with const url = `${this.proxy.url}/${reqUrl}`;. This will proxy the calls in accordance with the docs shown here.