HTTPS for the API

[timwaters]

1. change / document https in the API docs
2. In API response: in the self links etc of the api resource it should return the same scheme

e.g. https://warper.wmflabs.org/api/v1/layers/7

"kml":"http://warper.wmflabs.org/layers/7.kml"

1. tests for authentication etc

Note: the existing interface needs https to be off for map and layers as these use the iD editor interface which communicates with openhistoricalmap.org. OHM only has http not https.

[Abbe98]

Currently when trying to access https://wikimaps.mapwarper.net/ a time-out error is returned.

[timwaters]

@Abbe98 there is no https on wikimaps.mapwarper.net

There is https on https://warper.wmflabs.org/api/v1 (but where https calls in the browser on non /api/v1URLs are redirected to http)

[Abbe98]

oh sorry, I got the endpoints wrong...

[timwaters]

Okay, I've pushed some code to add the correct protocol (https / http) in the api responses, so that should be better now. Also authentication of the warper via https via https://timwaters.github.io/warper_oauth_token_auth_demo/# appears to be working.

Will leave open until I deploy and see if its all working correctly.

[Abbe98]

Currently I get a 502 Bad Gateway error when trying to use Oauth over HTTPS and HTTP?

[timwaters]

via https://timwaters.github.io/warper_oauth_token_auth_demo ? or your own script? On all browsers? More info pls.

I believe this may be what we discussed over email:

http://warper.wmflabs.org/u/auth/mediawiki?auth_origin_url=https%3A%2F%2Ftimwaters.github.io%2Fwarper_oauth_token_auth_demo%2F%23&config_name=default&omniauth_window_type=newWindow&favorite_color=purple

in chrome works okay
in firefox not so

even sent via https https://timwaters.github.io/warper_oauth_token_auth_demo/#

Now... if I remove or even just rename the auth_origin_url string, I
get no error.
warper.wmflabs.org/u/auth/mediawiki?config_name=default&omniauth_window_type=newWindow&favorite_color=purple
 works in FF

The error is from the labs proxy as our server doesnt run nginx ... So
something weird is going on with that query parameter name... I will
see if I can change the jtoker JS code...

[Abbe98]

Yep true, it's that bug.

2016-06-22 12:24 GMT+02:00 Tim Waters notifications@github.com:

via https://timwaters.github.io/warper_oauth_token_auth_demo ? or your own script? On all browsers? More info pls.

I believe this may be what we discussed over email:

http://warper.wmflabs.org/u/auth/mediawiki?auth_origin_url=https%3A%2F%2Ftimwaters.github.io%2Fwarper_oauth_token_auth_demo%2F%23&config_name=default&omniauth_window_type=newWindow&favorite_color=purple

in chrome works okay in firefox not so

even sent via https https://timwaters.github.io/warper_oauth_token_auth_demo/#

Now... if I remove or even just rename the auth_origin_url string, I get no error.warper.wmflabs.org/u/auth/mediawiki?config_name=default&omniauth_window_type=newWindow&favorite_color=purple works in FF

The error is from the labs proxy as our server doesnt run nginx ... So something weird is going on with that query parameter name... I will see if I can change the jtoker JS code...

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/wikimaps-dev/mapwarper/issues/106#issuecomment-227703999, or mute the thread https://github.com/notifications/unsubscribe/ACgoJwFCJegIsKUuxFUPycFmHYiesfEcks5qOQ1YgaJpZM4I4Yid .

[timwaters]

Will change the JS and see if it makes a difference, or if it's to do with the WMF Labs proxy.

[timwaters]

@Abbe98 okay, I've updated the code on https://timwaters.github.io/warper_oauth_token_auth_demo/# and it appears to not be giving the gateway error... please try on your side also :)

[Abbe98]

Currently the github link redirects me to http://warper.wmflabs.org/ and when using Mediawiki I still gets the gate way error.

2016-06-22 16:20 GMT+02:00 Tim Waters notifications@github.com:

@Abbe98 https://github.com/Abbe98 okay, I've updated the code on https://timwaters.github.io/warper_oauth_token_auth_demo/# and it appears to not be giving the gateway error... please try on your side also :)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/wikimaps-dev/mapwarper/issues/106#issuecomment-227759138, or mute the thread https://github.com/notifications/unsubscribe/ACgoJ1RUL4SJCxXsVw7aKRlZ9zZ9_ms0ks5qOUSugaJpZM4I4Yid .

[timwaters]

odd.

For github (and possibly also wikimedia) it could be because the redirect url set up was http and not https. I might have to set up two auths.

[timwaters]

• [x] https protocols enabled for api endpoints now

[timwaters]

Note: I have removed auth_origin_url from calls on https://timwaters.github.io/warper_oauth_token_auth_demo/#

@Abbe98 please see if that still gives a 502

[Abbe98]

It does still not always work but it still seams like a issue with parameters, when I removed config_name=default this time it worked...

[Abbe98]

This is relay weird, OAuth works in private a window, but even when using the auth_origin_url it still redirects me to warper.wmflabs.org. And sometimes it works in the regular window on second try when modifying the parameters.

I has to continue investigate.

[timwaters]

I believe it's a labs proxy issue - that is, if its not related with github

[timwaters]

I'm creating new oauth registrations for the https endpoint - testing locally made the results with github disappear.

I am also adding https links to oauth sign ins in the website to match.

Will deploy code when the oauth registration is approved

[timwaters]

Changed: OSM Oauth is https working now Github OAauth is https now -

and github should work now on https://timwaters.github.io/warper_oauth_token_auth_demo/#

Commons OAuth requires approval: https://meta.wikimedia.org/wiki/Special:OAuthListConsumers/view/d3308b00209d49e5a1d764aee019f83c

https://meta.wikimedia.org/wiki/Steward_requests/Miscellaneous#Request_to_Approve_OAuth_Consumer_for_Wikimaps_Warper_v.2.2

But the http callback works for the website. When approved, there shouldnt be any weird gateway errors, hopefully.

@Abbe98 do try github in the meantime before I switch to the new Commons Oauth configuration

[Abbe98]

Github Oauth still gives me an gateway error I'm afraid.

[timwaters]

using https://timwaters.github.io/warper_oauth_token_auth_demo/# ?

[Abbe98]

Yes

[timwaters]

okay, the OAuth proposal was approved. Please try with mediawiki / commons oauth login now. Both work fine for me on https://timwaters.github.io/warper_oauth_token_auth_demo/

I'm suspecting that it's the labs proxy that is at fault...

[Abbe98]

Okey it works now!

I found the issue causing the 502 error: I was logged into my main Wikimedia account on warper.wmflabs.org while trying to get Github/Mediawiki OAuth working(logging out fixed it)...

[timwaters]

okay, thats great - lets keep an eye on this for the moment!

[timwaters]

ok, so if user is logged in via oauth and they try to login again, with the long url, it overflows the buffers of the nginx proxy. So we can send a smaller url and it works.

Now, it also looks like the warper is setting two identical cookies and this is pushing it over. And/Or the oauth library is putting too much into the env / sessions / cookies.

The two cookies may be a rails config issue

Will investigate further....

[timwaters]

looking at the tcpdump logs on warper.wmflabs.org using

sudo stdbuf -oL -eL /usr/sbin/tcpdump -A -s0 "tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)" | egrep -a --line-buffered ".+(GET |HTTP\/|POST )|^[A-Za-z0-9-]+: " | perl -nle 'BEGIN{$|=1} { s/.*?(GET |HTTP\/[0-9.]* |POST )/\n$1/g; print }'

however trying the same thing on wikimaps.mapwarper.net only has the expected 1 cookie set - possibly because its not using the proxy. Will continue investigating... probably see what Rails is actually trying to set.

[timwaters]

Okay, I've added a few things.

1) using altered mediawiki oauth library to put less things in the session (e.g. no need to put wiki user rights) 2) if the user is logged in and calls /u/auth/{provider} then the warper logs them out and carrys on with the request as before 3) the session information about whether the call is via the API or via the normal website is kept between these.

So it should be working better now.

Clients should make sure that logged in users cannot log in via oauth again, of course, but the system should have slightly less chance of an error.

Am closing this now, as its workaroundable, and end of the milestone.

[timwaters]

Have also seen where the 2 identical cookies were being sent, so only one should be sent now. (we can revert the change for logging out if we want)