Client does not appear to set Origin: header

[bradwood]

Hi,

Is there a way I can set this via calling code?

And, more generally, any other headers? Or is this a wsproto thing?

I have a rather badly behaved server (it's a sky set top box) and it seems to be not responding unless an origin header is present...

thanks

Brad

[mehaase]

It is a limitation in wsproto, but it is being worked on upstream: python-hyper/wsproto/issues/85 and eventually trio-websocket will allow control over HTTP headers.

[bradwood]

Ok... Here is my situation:

WORKING (go-http-client):
---req---
GET /as/system/status HTTP/1.1
Host: skyq:9006
User-Agent: Go-http-client/1.1
Connection: Upgrade
Origin: http://skyq:9006/as/system/status
Sec-WebSocket-Key: 3pDOFIejJw9aFZ84xAus+A==
Sec-WebSocket-Version: 13
Upgrade: websocket

---res---
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: 0INWfq7txLmN/1LGLgpPUl+OfA8=

<json payload follows>

NOT WORKING (trio-websockets + wsproto):
---req---

GET /as/system/status? HTTP/1.1
host: skyq:9006
upgrade: WebSocket
connection: Upgrade
sec-websocket-key: xNtNd7KbRC0kZu14wayhRA==
sec-websocket-version: 13

---res---
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: xCfjTlQDTFLgO1IFiV8a8eK1Ar8=

<empty body :(>

[bradwood]

Any ideas on workarounds? This basically blocks me completely :(

[bradwood]

Also note the trailing ? on the trio/wsproto URL --- I didn't add that -- is it coming from the lib somewhere?

[mehaase]

If you want to stick with Trio and need an immediate fix, the best suggestion I have is to clone wsproto locally and hack your origin header into it. This is the location you'd need to edit: https://github.com/python-hyper/wsproto/blob/master/wsproto/connection.py#L113

If asyncio is an option, this is a high quality websocket library that I've used extensively: https://github.com/aaugustin/websockets

Also note the trailing ? on the trio/wsproto URL --- I didn't add that -- is it coming from the lib somewhere?

Yeah, that's a bug that's currently fixed on master. I can post a new release on PyPI if you need it.

[bradwood]

I had this issue with aiohttp, but never with aaugustin's stuff under asyncio, which just worked, so I agree -- its high quality. But I'm off asyncio now and onto trio so looking to stick with that now.

Re the ? -- if its not too much trouble to do a patch release on pyPI I'd appreciate it.

thanks for your help and responsiveness Mark!

[bradwood]

BTW, I just looked at that bit of wsproto code and the headers there are Capitalised, but in my dump they're not. Are you lowercasing them somewhere else? I know the RFC says case-insensitive, but my server might be pinickity about that too.

[mehaase]

It looks like lowercasing happens in h11, which is the sans I/O HTTP library that wsproto depends on:

https://h11.readthedocs.io/en/latest/api.html#headers-format

[mehaase]

Version 0.3.0 just released on PyPI.

[bradwood]

thanks again @mehaase 👍

[njsmith]

You might want to try confirming that it's the origin header first... What i would probably do here is write a silly little client that opens a direct TCP connection to the box, sends a hard-coded bytestring (copied from the working Wireshark dump), and then prints whatever it gets back. And then play around with the headers you send until you figure out what the minimal change is that switches it from working to not working.

[bradwood]

turns out my issue was with the case of headers @mehaase but the questionmark is now nicely removed thanks...

[njsmith]

The h11 issue is here: https://github.com/python-hyper/h11/issues/31