search

aws / aws-sdk-js

AWS SDK for JavaScript in the browser and Node.js
https://aws.amazon.com/developer/language/javascript/
Apache License 2.0
7.59k stars 1.55k forks source link

Proxy support #26

Closed gramakri closed 11 years ago

gramakri commented 11 years ago

Allow the API to be used with HTTP and Socks Proxies.

gramakri commented 11 years ago

For anyone else who wants this, proxychains and node work well for the moment.

erikjjohnson commented 11 years ago

Proxychains isn't as elegant as native support for this feature (like in the other AWS sdks). The other sdks add an AWS.config.proxy property to set the field, and then in the http.js request, read this property and pass it along to the underlying library if it supports it (request library).

Anyway, I hope this enhancement is considered for those of us stuck behind a proxy.

erikjjohnson commented 11 years ago

Greetings! The above fix results in an XMLParserError somewhere inside the XDK. For example:

AWS.config.update({ httpOptions: { proxy: 'http://myproxy:myport" }}); var s3 = new AWS.S3(); s3.client.listObjects({ Bucket: settings.bucket }, function (err, listing) { console.log("S3 error ", err); }

reports:

S3 error { [XMLParserError: Unexpected close tag Line: 6 Column: 7 Char: >] code: 'XMLParserError', name: 'XMLParserError', statusCode: 400, retryable: false }

If I don't set the httpOptions, but then run the same code through proxy chains, the listObject call works fine.

ampgcat commented 11 years ago

I have the very same problem. I get the same XMLParserError from getObject and putObject. XMLParserError: Unexpected close tag Line: 6 Column: 7 Char: > and calls work perfectly if I go out of the firewall and comment out AWS.config.update({ httpOptions: { proxy: 'http://myproxy:myport" }});

lsegal commented 11 years ago

See #108. Our proxy support currently does not support CONNECT proxies. Are you using CONNECT based proxies?

ampgcat commented 11 years ago

Thank you very much for a quick response. I don't know. I'll try to find out and get back to you. Thank you.

lsegal commented 11 years ago

@ampgcat it would also help if you could add the following to your callback:

console.log(this.httpResponse.body.toString());

The response body would be useful to figure out what data you are getting from the response.

ampgcat commented 11 years ago 
<HTML><HEAD>
<TITLE>Request Error</TITLE>
</HEAD>
<BODY>
<FONT face="Helvetica">
<big><strong></strong></big><BR>
</FONT>
<blockquote>
<TABLE border=0 cellPadding=1 width="80%">
<TR><TD>
<FONT face="Helvetica">
<big>Request Error (invalid_request)</big>
<BR>
<BR>
</FONT>
</TD></TR>
<TR><TD>
<FONT face="Helvetica">
Your request could not be processed. Request could not be handled
</FONT>
</TD></TR>
<TR><TD>
<FONT face="Helvetica">
This could be caused by a misconfiguration, or possibly a malformed request.
</FONT>
</TD></TR>
<TR><TD>
<FONT face="Helvetica" SIZE=2>
<BR>
For assistance, contact your network support team.
</FONT>
</TD></TR>
</TABLE>
</blockquote>
</FONT>
</BODY></HTML>
lsegal commented 11 years ago

This looks like the proxy server is not liking the request. Do you have details on what kind of proxy software is being used here?

ampgcat commented 11 years ago

Hi. I just talked with the company IT and they have no idea.. :<

lsegal commented 11 years ago

Unless we know why the request is being considered invalid, it will be hard to fix this. Googling the contents of that response brings up a few similar issues other users have had, but it's unclear what the proxy server is.

One thing it could be is an incompatibility with HTTPS. Have you tried creating your S3 object with SSL off?

var s3 = new AWS.S3({sslEnabled: false});
ampgcat commented 11 years ago

No. That seems to work. Thanks!

lsegal commented 11 years ago

If this works it seems that your proxy does not support SSL. I would be very cautious about using any connection (especially a proxy) that does not support SSL as this leaves you open for possible man-in-the-middle attacks. The data you are sending is completely in the clear, so you should be very careful when sending and receiving sensitive data from the services you use. I would avoid using this proxy if possible.

Hope that helps.

ampgcat commented 11 years ago

Thanks!!!

Mido22 commented 10 years ago

Hi Isegal, I am getting problems similar to that of ampgcat while trying to send a mail through AWS.SES in node server, I have set the proxy in the config.json as 

      ...
      "httpOptions": {
        "proxy": "http://0.0.0.0:0000"
      }, ...

you can find the http response below, 

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<HTML><HEAD><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<TITLE>ERROR: The requested URL could not be retrieved</TITLE>
<STYLE type="text/css"><!--BODY{background-color:#ffffff;font-family:verdana,sans-serif}PRE{font-family:sans-serif}--></STYLE>
</HEAD><BODY>
<H1>ERROR</H1>
<H2>The requested URL could not be retrieved</H2>
<HR noshade size="1px">
<P>
While trying to retrieve the URL:
<A HREF="https://email.us-west-2.amazonaws.com/">https://email.us-west-2.amazonaws.com/</A>
<P>
The following error was encountered:
<UL>
<LI>
<STRONG>
Connection to 54.240.252.56 Failed
</STRONG>
</UL>

<P>
The system returned:
<PRE><I>    (71) Protocol error</I></PRE>

<P>
The remote host or network may be down.  Please try the request again.
<P>Your cache administrator is <A HREF="mailto:webmaster">webmaster</A>.

<BR clear="all">
<HR noshade size="1px">
<ADDRESS>
Generated Fri, 15 Aug 2014 02:01:58 GMT by proxy.yyy.xxx.com (squid/2.7.STABLE6)
</ADDRESS>
</BODY></HTML>

some help please.

yamsellem commented 9 years ago

Using https-proxy-agent worked for me.

var ProxyAgent = require('https-proxy-agent');

var options = {
  accessKeyId: '..', 
  secretAccessKey: '..',
  httpOptions: {agent: new ProxyAgent({host, port})}
};
var s3 = new AWS.S3(options);
restebanez commented 9 years ago

http://blogs.aws.amazon.com/javascript/post/Tx3CV0OSVN2TBPS/Using-the-AWS-SDK-for-JavaScript-from-Behind-a-Proxy

ivarrian commented 9 years ago

@yamsellem This is the solution that helped me! Thanks.

license2e commented 8 years ago

@yamsellem Thank you! This helped me as well...

alain-odea-vgh commented 8 years ago

Got hit by this too. My Squid proxy was refusing access, returning HTTP 403 itself, and presenting a user-facing access denied HTML page. Not sure if there's a reliable way to differentiate that from a AWS API error. The Content-Type is text/html regardless of Accept headers, but I suppose that isn't enforceable for S3 since each Object can have a custom Content-Type including text/html.

lock[bot] commented 5 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.