implicit redirect with go-middleware template

rajatjindal commented 5 years ago

my usecase is to have a function that receives a URL as parameter, and after doing some processing redirects to that URL.

Now the redirect is working, but the URL in the browser still points to the original function URL.

I believe it can be fixed using client.CheckRedirect thing in golang, but i am not exactly sure where will that change be needed in openfaas cloud.

To reproduce: https://rajatjindal.o6s.io/redirect-test -> this returns OK, no redirect https://rajatjindal.o6s.io/redirect-test?redirect=https://google.com -> this get o/p from google.com, but the URL in browser still remains the same.

source code of this function: https://github.com/rajatjindal/redirect-test

alexellis commented 5 years ago

We need to add "don't follow redirect" here https://github.com/openfaas-incubator/golang-http-template/blob/master/template/golang-middleware/main.go

Check the watchdog for an example of how to implement it. I'd accept a PR.

rajatjindal commented 5 years ago

i created local docker image of of-watchdog, and further created docker image of the function using middleware template, and could not reproduce this issue when running locally.

here is the image built locally for this function:

rajatjindal/redirect-test:issue-31

to verify i did following:

started the function locally:

➜  golang-middleware git:(master) ✗ docker run -p 8080:8080 -p 8082:8082 -p 8081:8081 rajatjindal/redirect-test:issue-31
Forking - ./handler []
2019/09/25 05:07:25 Started logging stderr from function.
2019/09/25 05:07:25 Started logging stdout from function.
2019/09/25 05:07:25 OperationalMode: http
2019/09/25 05:07:25 Timeouts: read: 10s, write: 10s hard: 10s.
2019/09/25 05:07:25 Listening on port: 8080
2019/09/25 05:07:25 Writing lock-file to: /tmp/.lock
2019/09/25 05:07:25 Metrics listening on port: 8081

Hit the function directly (bypass the of-watchdog):

➜  golang-middleware git:(master) ✗ curl -vXGET 'http://localhost:8082?redirect=https://google.com'

Note: Unnecessary use of -X or --request, GET is already inferred.
* Rebuilt URL to: http://localhost:8082/?redirect=https://google.com
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8082 (#0)
> GET /?redirect=https://google.com HTTP/1.1
> Host: localhost:8082
> User-Agent: curl/7.54.0
> Accept: */*
> 
< HTTP/1.1 307 Temporary Redirect
< Content-Type: text/html; charset=utf-8
< Location: https://google.com
< Date: Wed, 25 Sep 2019 05:07:37 GMT
< Content-Length: 54
< 
<a href="https://google.com">Temporary Redirect</a>.

* Connection #0 to host localhost left intact
➜

Hit the function via of-watchdog

➜  golang-middleware git:(master) ✗ curl -vXGET 'http://localhost:8080?redirect=https://google.com'
Note: Unnecessary use of -X or --request, GET is already inferred.
* Rebuilt URL to: http://localhost:8080/?redirect=https://google.com
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8080 (#0)
> GET /?redirect=https://google.com HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.54.0
> Accept: */*
> 
< HTTP/1.1 307 Temporary Redirect
< Content-Length: 54
< Content-Type: text/html; charset=utf-8
< Date: Wed, 25 Sep 2019 05:09:03 GMT
< Location: https://google.com
< X-Duration-Seconds: 0.001594
< 
<a href="https://google.com">Temporary Redirect</a>.

* Connection #0 to host localhost left intact

while hitting the function deployed on openfaas-cloud, we get implicit redirect done

➜  golang-middleware git:(master) ✗ curl -vXGET 'https://rajatjindal.o6s.io/redirect-test?redirect=

Note: Unnecessary use of -X or --request, GET is already inferred.
*   Trying 67.207.70.150...
* TCP_NODELAY set
* Connected to rajatjindal.o6s.io (67.207.70.150) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=*.o6s.io
*  start date: Aug 26 15:11:17 2019 GMT
*  expire date: Nov 24 15:11:17 2019 GMT
*  subjectAltName: host "rajatjindal.o6s.io" matched cert's "*.o6s.io"
*  issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7fc200809400)
> GET /redirect-test?redirect=https://google.com HTTP/2
> Host: rajatjindal.o6s.io
> User-Agent: curl/7.54.0
> Accept: */*
> 
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 200 
< server: nginx/1.15.8
< date: Wed, 25 Sep 2019 05:09:59 GMT
< content-type: text/html; charset=ISO-8859-1
< vary: Accept-Encoding
< alt-svc: quic=":443"; ma=2592000; v="46,43,39"
< cache-control: private, max-age=0
< expires: -1
< p3p: CP="This is not a P3P policy! See g.co/p3phelp for more info."
< set-cookie: 1P_JAR=2019-09-25-05; expires=Fri, 25-Oct-2019 05:09:59 GMT; path=/; domain=.google.com; SameSite=none
< set-cookie: NID=188=cPY4PEtMKHdowTmTHT9bKW_Lfdg7d_Inn00_Ca92v5eRPgoBU1rgQnOM7eNvVhZTy_lFG3U1c6rjFwUrHzdZfCfY4ZnGcAtE-Tbu3BaRqY5DUCKjGRVGLCrfkrZBwG9RAyRU-9fzHVmJHil7_Ljby_hBVI8NFEaUHghyGW3psAM; expires=Thu, 26-Mar-2020 05:09:59 GMT; path=/; domain=.google.com; HttpOnly
< x-frame-options: SAMEORIGIN
< x-xss-protection: 0
< strict-transport-security: max-age=15724800; includeSubDomains
< 
<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="en-GB"><head><meta content="text/html; charset=UTF-8" http-equiv="Content-Type"><meta content="/images/branding/googleg/1x/googleg_standard_color_128dp.png" itemprop="image"><title>Google</title><script nonce="yZhC7Kc19BCkVO3UmFxUzw==">(function(){window.google={kEI:'p_aKXYa4Ho_GaZD5rMAH',kEXPI:'0,18168,1335578,4439,1224,731,223,510,1065,2081,1071,377,207,1017,53,1438,570,2,124,10,713,320,18,48,142,349,670,128,1131128,144,1197719,329548,1294,12383,4855,32691,2075,13173,867,12163,5281,1100,3335,2,2,6801,363,3320,5505,224,2212,5948,1119,2,578,728,2431,1362,4325,4966,773,2248,1409,4483,9,981,987,6191,1719,1808,1478,7,3,488,2044,8909,5297,897,1119,38,920,873,1214,1367,4347,48,1510,1503,2,631,3240,193,7253,620,2884,20,317,235,884,904,101,1049,975,1,372,2774,520,399,992,840,445,8,109,1107,438,1142,889,78,48,553,11,14,1279,2213,201,37,130,156,5,68,1184,840,324,193,318,1156,48,158,662,2620,818,109,151,52,1136,3,2063,606,1839,184,595,1325,377,257,43,386,1071,190,244,503,211,8,98,112,44,207,1,791,78,25,328,1284,16,84,336,25,56,2426,1639,608,473,11,1166,162,29,700,19,1039,669,2423,135,773,1216,332,524,7,586,142,592,523,928,123,1600,735,859,1361,184,344,426,957,2994,167,465,407,1,11,1,24,51,613,43,9,648,289,174,58,540,565,16,297,3,188,2,42,4,477,164,2,5,197,2,470,588,14,32,1,335,115,312,214,509,38,71,498,60,1281,771,21,225,496,59,719,398,144,268,216,212,93,21,52,253,42,65,589,393,116,5889336,13,6000454,187,2799686,4,1572,549,333,444,1,2,80,1,900,583,9,304,1,8,1,2,2132,1,1,1,1,1,414,1,748,141,59,726,3,7,563,1,3787,141,33,4,7,30,17,22303119,1662865',authuser:0,kscs:'c9c918f0_p_aKXYa4Ho_GaZD5rMAH',kGL:'GB',kBL:'yHmd'};google.sn='webhp';google.kHL='en-GB';google.jsfs='Ffpdje';})();(function(){google.lc=[];google.li=0;google.getEI=function(a){for(var b;a&&(!a.getAttribute||!(b=a.getAttribute("eid")));)a=a.parentNode;return b||google.kEI};google.getLEI=function(a){for(var b=null;a&&(!a.getAttribute||!(b=a.getAttribute("leid")));)a=a.parentNode;return b};google.https=function(){return"https:"==window.location.protocol};google.ml=function(){return null};google.time=function(){return(new Date).getTime()};google.log=function(a,b,e,c,g){if(a=google.logUrl(a,b,e,c,g)){b=new Image;var d=google.lc,f=google.li;d[f]=b;b.onerror=b.onload=b.onabort=function(){delete d[f]};google.vel&&google.vel.lu&&google.vel.lu(a);b.src=a;google.li=f+1}};google.logUrl=function(a,b,e,c,g){var d="",f=google.ls||"";e||-1!=b.search("&ei=")||(d="&ei="+google.getEI(c),-1==b.search("&lei=")&&(c=google.getLEI(c))&&(d+="&lei="+c));c="";!e&&google.cshid&&-1==b.search("&cshid=")&&"slh"!=a&&(c="&cshid="+google.cshid);a=e||"/"+(g||"gen_204")+"?atyp=i&ct="+a+"&cad="+b+d+f+"&zx="+google.time()+c;/^http:/i.test(a)&&google.https()&&(google.ml(Error("a"),!1,{src:a,glmm:1}),a="");return a};}).call(this);(function(){google.y={};google.x=function(a,b){if(a)var c=a.id;else{do c=Math.random();while(google.y[c])}google.y[c]=[a,b];return!1};google.lm=[];google.plm=function(a){google.lm.push.apply(google.lm,a)};google.lq=[];google.load=function(a,b,c){google.lq.push([[a],b,c])};google.loadAll=function(a,b){google.lq.push([a,b])};}).call(this);google.f={};var a=window.location,b=a.href.indexOf("#");if(0<=b){var c=a.href.substring(b+1);/(^|&)q=/.test(c)&&-1==c.indexOf("#")&&a.replace("/search?"+c.replace(/(^|&)fp=[^&]*/g,"")+"&cad=h")};</script><style>#gbar,#guser{font-size:13px;padding-top:1px !important;}#gbar{height:22px}#guser{padding-bottom:7px !important;text-align:right}.gbh,.gbd{border-top:1px solid #c9d7f1;font-size:1px}.gbh{height:0;position:absolute;top:24px;width:100%}@media all{.gb1{height:22px;margin-right:.5em;vertical-align:top}#gbar{float:left}}a.gb1,a.gb4{text-decoration:underline !important}a.gb1,a.gb4{color:#00c !important}.gbi .gb4{color:#dd8e27 !important}.gbf .gb4{color:#900 !important}
</style><style>body,td,a,p,.h{font-family:arial,sans-serif}body{margin:0;overflow-y:scroll}#gog{padding:3px 8px 0}td{line-height:.8em}.gac_m td{line-height:17px}form{margin-bottom:20px}.h{color:#36c}.q{color:#00c}.ts td{padding:0}.ts{border-collapse:collapse}em{font-weight:bold;font-style:normal}.lst{height:25px;width:496px}.gsfi,.lst{font:18px arial,sans-serif}.gsfs{font:17px arial,sans-serif}.ds{display:inline-box;display:inline-block;margin:3px 0 4px;margin-left:4px}input{font-family:inherit}a.gb1,a.gb2,a.gb3,a.gb4{color:#11c !important}body{background:#fff;color:black}a{color:#11c;text-decoration:none}a:hover,a:active{text-decoration:underline}.fl a{color:#36c}a:visited{color:#551a8b}a.gb1,a.gb4{text-decoration:underline}a.gb3:hover{text-decoration:none}#ghead a.gb2:hover{color:#fff !important}.sblc{padding-top:5px}.sblc a{display:block;margin:2px 0;margin-left:13px;font-size:11px}.lsbb{background:#eee;border:solid 1px;border-color:#ccc #999 #999 #ccc;height:30px}.lsbb{display:block}.ftl,#fll a{display:inline-block;margin:0 12px}.lsb{background:url(/images/nav_logo229.png) 0 -261px repeat-x;border:none;color:#000;cursor:pointer;height:30px;margin:0;outline:0;font:15px arial,sans-serif;vertical-align:top}.lsb:active{background:#ccc}.lst:focus{outline:none}</style><script nonce="yZhC7Kc19BCkVO3UmFxUzw=="></script></head><body bgcolor="#fff"><script nonce="yZhC7Kc19BCkVO3UmFxUzw==">(function(){var src='/images/nav_logo229.png';var iesg=false;document.body.onload = function(){window.n && window.n();if (document.images){new Image().src=src;}
if (!iesg){document.f&&document.f.q.focus();document.gbqf&&document.gbqf.q.focus();}
}
})();</script><div id="mngb"> <div id=gbar><nobr><b class=gb1>Search</b> <a class=gb1 href="https://www.google.co.uk/imghp?hl=en&tab=wi">Images</a> <a class=gb1 href="https://maps.google.co.uk/maps?hl=en&tab=wl">Maps</a> <a class=gb1 href="https://play.google.com/?hl=en&tab=w8">Play</a> <a class=gb1 href="https://www.youtube.com/?gl=GB&tab=w1">YouTube</a> <a class=gb1 href="https://news.google.co.uk/nwshp?hl=en&tab=wn">News</a> <a class=gb1 href="https://mail.google.com/mail/?tab=wm">Gmail</a> <a class=gb1 href="https://drive.google.com/?tab=wo">Drive</a> <a class=gb1 style="text-decoration:none" href="https://www.google.co.uk/intl/en/about/products?tab=wh"><u>More</u> &raquo;</a></nobr></div><div id=guser width=100%><nobr><span id=gbn class=gbi></span><span id=gbf class=gbf></span><span id=gbe></span><a href="http://www.google.co.uk/history/optout?hl=en" class=gb4>Web History</a> | <a  href="/preferences?hl=en" class=gb4>Settings</a> | <a target=_top id=gb_70 href="https://accounts.google.com/ServiceLogin?hl=en&passive=true&continue=https://www.google.com/" class=gb4>Sign in</a></nobr></div><div class=gbh style=left:0></div><div class=gbh style=right:0></div> </div><center><br clear="all" id="lgpd"><div id="lga"><img alt="Google" height="92" src="/images/branding/googlelogo/1x/googlelogo_white_background_color_272x92dp.png" style="padding:28px 0 14px" width="272" id="hplogo"><br><br></div><form action="/search" name="f"><table cellpadding="0" cellspacing="0"><tr valign="top"><td width="25%">&nbsp;</td><td align="center" nowrap=""><input name="ie" value="ISO-8859-1" type="hidden"><input value="en-GB" name="hl" type="hidden"><input name="source" type="hidden" value="hp"><input name="biw" type="hidden"><input name="bih" type="hidden"><div class="ds" style="height:32px;margin:4px 0"><input style="color:#000;margin:0;padding:5px 8px 0 6px;vertical-align:top" autocomplete="off" class="lst" value="" title="Google Search" maxlength="2048" name="q" size="57"></div><br style="line-height:0"><span class="ds"><span class="lsbb"><input class="lsb" value="Google Search" name="btnG" type="submit"></span></span><span class="ds"><span class="lsbb"><input class="lsb" id="tsuid1" value="I'm Feeling Lucky" name="btnI" type="submit"><script nonce="yZhC7Kc19BCkVO3UmFxUzw==">(function(){var id='tsuid1';document.getElementById(id).onclick = function(){if (this.form.q.value){this.checked = 1;if (this.form.iflsig)this.form.iflsig.disabled = false;}
else top.location='/doodles/';};})();</script></span></span></td><td class="fl sblc" align="left" nowrap="" width="25%"><a href="/advanced_search?hl=en-GB&amp;authuser=0">Advanced search</a><a href="/language_tools?hl=en-GB&amp;authuser=0">Language tools</a></td></tr></table><input id="gbv" name="gbv" type="hidden" value="1"><script nonce="yZhC7Kc19BCkVO3UmFxUzw==">(function(){var a,b="1";if(document&&document.getElementById)if("undefined"!=typeof XMLHttpRequest)b="2";else if("undefined"!=typeof ActiveXObject){var c,d,e=["MSXML2.XMLHTTP.6.0","MSXML2.XMLHTTP.3.0","MSXML2.XMLHTTP","Microsoft.XMLHTTP"];for(c=0;d=e[c++];)try{new ActiveXObject(d),b="2"}catch(h){}}a=b;if("2"==a&&-1==location.search.indexOf("&gbv=2")){var f=google.gbvu,g=document.getElementById("gbv");g&&(g.value=a);f&&window.setTimeout(function(){location.href=f},0)};}).call(this);</script></form><div id="gac_scont"></div><div style="font-size:83%;min-height:3.5em"><br></div><span id="footer"><div style="font-size:10pt"><div style="margin:19px auto;text-align:center" id="fll"><a href="/intl/en/ads/">Advertising�Programmes</a><a href="/services/">Business Solutions</a><a href="/intl/en/about.html">About Google</a><a href="https://www.google.com/setprefdomain?prefdom=GB&amp;prev=https://www.google.co.uk/&amp;sig=K_WqQm3gWtxI51xTVhP1gcec4a0dk%3D">Google.co.uk</a></div></div><p style="color:#767676;font-size:8pt">&copy; 2019 - <a href="/intl/en/policies/privacy/">Privacy</a> - <a href="/intl/en/policies/terms/">Terms</a></p></span></center><script nonce="yZhC7Kc19BCkVO3UmFxUzw==">(function(){window.google.cdo={height:0,width:0};(function(){var a=window.innerWidth,b=window.innerHeight;if(!a||!b){var c=window.document,d="CSS1Compat"==c.compatMode?c.documentElement:c.body;a=d.clientWidth;b=d.clientHeight}a&&b&&(a!=google.cdo.width||b!=google.cdo.height)&&google.log("","","/client_204?&atyp=i&biw="+a+"&bih="+b+"&ei="+google.kEI);}).call(this);})();(function(){var u='/xjs/_/js/k\x3dxjs.hp.en.meu_UbG4Irs.O/m\x3dsb_he,d/am\x3dHhZsAg/d\x3d1/rs\x3dACT90oFx5MkUgHP-hKfNsU5ydfeNMhGbfA';setTimeout(function(){var a=document.createElement("script");a.src=u;google.timers&&google.timers.load&&google.tick&&google.tick("load","xjsls");document.body.appendChild(a)},0);})();(function(){window.google.xjsu='/xjs/_/js/k\x3dxjs.hp.en.meu_UbG4Irs.O/m\x3dsb_he,d/am\x3dHhZsAg/d\x3d1/rs\x3dACT90oFx5MkUgHP-hKfNsU5ydfeNMhGbfA';})();function _DumpException(e){throw e;}
function _F_installCss(c){}
* Connection #0 to host rajatjindal.o6s.io left intact
(function(){google.spjs=false;google.snet=true;google.em=[];google.emw=false;})();(function(){var pmc='{\x22JnTczA\x22:{},\x22Qnk92g\x22:{},\x22RWGcrA\x22:{},\x22U5B21g\x22:{},\x22YFCs/g\x22:{},\x22YQeDTA\x22:{},\x22ZI/YVQ\x22:{},\x22d\x22:{},\x22mVopag\x22:{},\x22sb_he\x22:{\x22agen\x22:true,\x22cgen\x22:true,\x22client\x22:\x22heirloom-hp\x22,\x22dh\x22:true,\x22dhqt\x22:true,\x22ds\x22:\x22\x22,\x22ffql\x22:\x22en\x22,\x22fl\x22:true,\x22host\x22:\x22google.com\x22,\x22isbh\x22:28,\x22jsonp\x22:true,\x22msgs\x22:{\x22cibl\x22:\x22Clear Search\x22,\x22dym\x22:\x22Did you mean:\x22,\x22lcky\x22:\x22I\\u0026#39;m Feeling Lucky\x22,\x22lml\x22:\x22Learn more\x22,\x22oskt\x22:\x22Input tools\x22,\x22psrc\x22:\x22This search was removed from your \\u003Ca href\x3d\\\x22/history\\\x22\\u003EWeb History\\u003C/a\\u003E\x22,\x22psrl\x22:\x22Remove\x22,\x22sbit\x22:\x22Search by image\x22,\x22srch\x22:\x22Google Search\x22},\x22ovr\x22:{},\x22pq\x22:\x22\x22,\x22refpd\x22:true,\x22rfs\x22:[],\x22sbpl\x22:24,\x22sbpr\x22:24,\x22scd\x22:10,\x22sce\x22:5,\x22stok\x22:\x22OzFV1OLcPl5N5DZK4rQk72qtH-o\x22,\x22uhde\x22:false}}';google.pmc=JSON.parse(pmc);})();</script>        </body></html>%                                                                                           
➜  golang-middleware git:(master) ✗

rajatjindal commented 5 years ago

I am suspecting something else on openfaas-cloud is doing the magic here? or the version of of-watchdog on openfaas-cloud is not latest?

I do plan to setup openfaas-cloud on my kubernetes cluster, and try it out there, if you have any suggestions/pointers, it will be great.

alexellis commented 5 years ago

This is the code I was thinking of https://github.com/openfaas-incubator/of-watchdog/commit/1ac9c410a3ef50c686c1b6963409142903578ba8

But in retrospect, this code is if you are using a HTTP client, which we are not here.

Look at the edge-router to see whether it follows redirects or not? It may be the part which you're hitting.

I did client-side JavaScript redirects for the OpenFaaS official functions -> https://github.com/openfaas/cloud-functions/blob/master/slack/handler.go

This work-around will fix things for you, for now, but if you can find time to explore the edge-router, I'd recommend it. It appears to be that a component is "following" redirects, rather than passing them on.

https://github.com/openfaas/openfaas-cloud/blob/master/edge-router/main.go

rajatjindal commented 5 years ago

got it. thanks for the pointer Alex.

is it possible to check logs of edge-router on openfaas-cloud? I tried using ofc-bot, but it does not show those logs (rightfully so i guess)

I'll add edge-router to my local setup tonight to narrow down the issue.

alexellis commented 5 years ago

I've made a fix for the edge-router.

For example try out:

https://github.com/alexellis/redirect-http/commits/master

Alex

rajatjindal commented 5 years ago

yep, works now. thanks for the quick fix.

openfaas / golang-http-template

implicit redirect with go-middleware template #31