Web access returns 400 Invalid Request

[realies]

Running the following docker run -p 7890:7890 -v "/mnt/user/appdata/goaccess/data":"/srv/data":rw -v "/mnt/user/appdata/goaccess/html":"/srv/report":rw -v "/mnt/user/appdata/letsencrypt/log/nginx":"/srv/logs":ro allinurl/goaccess goaccess /srv/logs/access.log -o report.html --real-time-html --no-global-config --config-file=/srv/data/goaccess.conf parses the access log and returns WebSocket server ready to accept new client connections.

Unfortunately accessing the the front-end results in an empty page, curl confirms:

# curl http://localhost:7890/ -vvvv
*   Trying localhost...
* TCP_NODELAY set
* Connected to localhost (localhost) port 7890 (#0)
> GET / HTTP/1.1
> Host: localhost:7890
> User-Agent: curl/7.57.0
> Accept: */*
> 
< HTTP/1.1 400 Invalid Request
* no chunk, no close, no size. Assume close to signal end
< 
* Closing connection 0

Using different ports for the front-end and the WebSocket server (--ws-url=0.0.0.0:7891 --port=7890)produces the same result.

[thornyweb]

Did you ever find a solution to this? I'm seeing it now.

[1arrow]

following

[allinurl]

@1arrow @thornyweb are you able to run and debug this outside of Docker? I'm thinking it should be fairly easy to debug this on gdb. Let me know and I can post some steps on how to do this. Thanks

[realies]

@thornyweb, just tested my instance and it seems I still haven't found a way around it.

Update: Think I've closed the issue when I found that exposing the report outside of the container and serving it via Nginx works, contrary to my initial understanding that goaccess would be self-hosted. Unfortunately, I don't keep the configuration for this, perhaps @allinurl can shed more light?

[allinurl]

@realies So just to be clear, if you run goaccess outside Docker, then you are able to get real-time data?

I'm thinking this most be some sort configuration or version issue since I'm able to run goaccess in a Docker container while getting real-time data out of it. Again, if any of you are able to reproduce this issue outside of Docker and run it through gdb would be awesome.

[thornyweb]

@allinurl I can run it in docker with out the real time html flag and it runs ok it’s the real time html flag that seems to be causing me problems.

@realies are you saying that you use a volume to mount the report html and then serve that on your Docker host with Nginx?

[realies]

@thornyweb, it seems that goaccess does not serve the HTML report page and the exposed port is meant for WebSocket connections only. Serving the HTML report file via a web server does work as long as the WebSocket port is correctly exposed from the container to the serving host.

My expectation for the default behaviour is thatgoaccess is accessible via its port and responding with the report page when accessed via HTTP and providing the WebSocket connection if accessed via WS.

[allinurl]

That's right. You need to serve the HTML report through a web server, e.g., nginx, apache, etc. or open the HTML file via your browser (Ctrl+o), i.e., file:///srv/report/report.html. You can find more details here.

[realies]

Would it be difficult to host the HTML file via the same port?

[thornyweb]

Thanks @allinurl I think what’s not clear from the documentation is that it requires a secondary service to front the HTML. Would you take a PR for any changes to this?

@realies. Having had this confirmed now. I think the easiest thing to do would be to mount the reports dir in as a volume from your host then run nginx as a separate service, both could be defined in a single compose file which could use the same volume from the host.

[realies]

Sure, would be cool if proxy_pass is the only thing needed, similarly to other containerised web apps.

[thornyweb]

@realies This is my compose file I've got working locally with Docker

version: "3.2"
services:
  nginx:
   container_name: goaccess_web
    image: nginx:stable-alpine
    ports:
      - 80:80
    volumes:
      - C:/Users/Michael/Dev/projects/goaccess/LOCAL/html:/usr/share/nginx/html:ro

  app:
    container_name: goaccess_service
    image: allinurl/goaccess:latest
    ports:
      - 7890:7890
    volumes:
      - C:/Users/Michael/Dev/projects/goaccess/LOCAL/data:/srv/data
      - C:/Users/Michael/Dev/projects/goaccess/LOCAL/logs:/srv/logs
      - C:/Users/Michael/Dev/projects/goaccess/LOCAL/html:/srv/report

Then just going to http://localhost hits the nginx and serves the html from the mapped in folder. I'm not even passing in any nginx conf, just the folder of HTML I want to use as my web dir, with a read only flag as it's being mastered by the goaccess container.

[realies]

Thanks for this but my point is that perhaps with little modification of goaccess there wouldn't be any need for a webserver when there is already a websocket server built-in.

[allinurl]

The original assumption was that if you're parsing an access log, you probably have a webserver that you could use to serve the HTML file. That probably doesn't hold completely in a Docker environment.. There was a discussion to have a built-in web server to simply serve the HTML file. I still need to look into this.

@thornyweb, please feel free to submit a PR and I'll be happy to merge it and update the docs. Thanks.

[realies]

It just seems more elegant to provide the html via the same port if it is relatively easy to implement.

[BirkhoffLee]

I'm also having a same issue of WebSocket connection returning 400 Invalid Request. I'm not sure but shouldn't there be any additional messages in log when a successful WS connection establishes? I only have this:

$ docker-compose logs -f goaccess
Attaching to goaccess_goaccess_1
WebSocket server ready to accept new client connections

I'm still investigating because this have a number of factors to happen. My setup is quite complicated but I'll keep it simple: GoAccess runs in a Docker environment where behind a reverse proxy called Traefik. I serve GoAccess web server and the WebSocket server on different containers and domains. Both of them are behind CloudFlare in which supports WebSocket, and shouldn't be a problem.

Traefik (the reverse proxy) returns the below on attempt to connect to the GoAccess WS server:

vulcand/oxy/forward/websocket: Error dialing "(websocket server here)": websocket: bad handshake with resp: 400 400 Invalid Request

and +1 for

It just seems more elegant to provide the html via the same port if it is relatively easy to implement.

[BirkhoffLee]

After some testing, having any cookies sent to WS server will result in a 400 Bad Request error returning from the WS server. The solution is to (simply) clear all cookies that will be sent to the server and try again. It works for me :)

This should be considered a bug, I guess, maybe @allinurl can take a look? Is anyone able to reproduce this?

[Ehekatl]

I don't have any cookie, still get 400 no matter how I configure (I have tried all means on readme, including different platform docker/linux/macos, curl & chrome never works, raw websocket does work use client like wsta)

[allinurl]

@BirkhoffLee Are you able to provide the steps to reproduce this? So far I'm unable to replicate this from my side.

@Ehekatl Are you running it inside a Docker container as well?

[Ehekatl]

@allinurl either it's inside docker or outside, it's always http 400

I don't think docker matters, I have experience running a lot application inside docker, including websocket ones.

It should be some simple mistake during websocket http handshake, since raw websockets always work. Another issue I'm not sure if it's related, goaccess access.log -o /tmp/report.html --real-time-html running goaccess like this, the report.html won't get update unless I restart goaccess

[Ehekatl]

@allinurl finally I'm able to reproduce this bug 100% time.

ws server return http 400 when cookie is larger than a certain size.

I can reproduce this by request with a large cookie

[allinurl]

@Ehekatl Thanks for posting this finding. Just to be sure, so no cookie it works, large cookie doesn't work? Can you please share the length of the cookie you are sending so I can play with it from my side? Thanks!

[ltm]

@allinurl I just experienced this issue as well. I believe it occurs when the total size of the request is larger than 1024 bytes, which is likely to happen if you have several cookies already set for the domain.

As such, a request of 1025 bytes (note printf '%820s') returns 400:

$ curl -v http://localhost:7890/ \
    -H 'Connection: Upgrade' \
    -H 'Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==' \
    -H 'Sec-WebSocket-Version: 13' \
    -H 'Upgrade: websocket' \
    -H 'User-Agent: Mozilla/5.0' \
    -H 'Cookie: foo='$(printf '%820s' | tr ' ' 'x')
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 7890 (#0)
> GET / HTTP/1.1
> Host: localhost:7890
> Accept: */*
> Connection: Upgrade
> Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
> Sec-WebSocket-Version: 13
> Upgrade: websocket
> User-Agent: Mozilla/5.0
> Cookie: foo=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
> 
< HTTP/1.1 400 Invalid Request
* no chunk, no close, no size. Assume close to signal end
< 
* Closing connection 0

While a request of 1024 bytes (note printf '%819s') returns 101:

$ curl -v http://localhost:7890/ \
    -H 'Connection: Upgrade' \
    -H 'Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==' \
    -H 'Sec-WebSocket-Version: 13' \
    -H 'Upgrade: websocket' \
    -H 'User-Agent: Mozilla/5.0' \
    -H 'Cookie: foo='$(printf '%819s' | tr ' ' 'x')
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 7890 (#0)
> GET / HTTP/1.1
> Host: localhost:7890
> Accept: */*
> Connection: Upgrade
> Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
> Sec-WebSocket-Version: 13
> Upgrade: websocket
> User-Agent: Mozilla/5.0
> Cookie: foo=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
> 
< HTTP/1.1 101 Switching Protocols
< Upgrade: websocket
< Connection: Upgrade
< Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
< 
?;Y{"general": ...

[allinurl]

@ltm Thanks a lot for pointing this out. Let me try to reproduce this on my end and I'll post back.

[EpochalEngineer]

Did you have any success replicating?

I was able to replicate on my end with the curl commands and with a cookie set returning 400. My setup is simply docker + goaccess from the ticket that referenced this, no other modifications. There is a simple workaround is simply opening incognito (so other tickets would be of higher priority, like 117)

[conet]

I can confirm that the problem was a too long cookie on that domain.

[allinurl]

Thanks for reporting this. I've added as a milestone for v1.4 which it's coming out soon.

[brunoluiz]

For anyone who comes across this issue and use traefik: you can remove the header by setting Cookie=. In my case, Cloudflare always adds a cookie to my requests, so it needs to be transformed traefik side. My docker-compose at the moment:

    labels:
      ...
      - "traefik.http.middlewares.removecookies.headers.customrequestheaders.Cookie="
      - "traefik.http.routers.goaccess.middlewares=removecookies"

[allinurl]

Thanks everyone, this has been fixed and pushed upstream. Feel free to test it out.

This was compiler dependent as it was using BUFSIZ. It now uses a more reasonable fixed buffer size for the request headers of 8192. It will be pushed out in the upcoming release. Stay tuned :)

[synthead]

I'm still getting this error with the latest tag (v1.5.2) and the latest commit from master as of this writing (f7b1e3ce1f40e251c43dacd34e8dd632041d2d9c):

[aqos156]

I'm facing the same issue on latest docker image

[allinurl]

@aqos156 can you please share some details on your environment and the exact error you are getting via the browser's console and if you could try accessing telnet host 7890. thanks.

[aqos156]

@allinurl when I run cat traefik.log | docker run --rm -i allinurl/goaccess -a -o html --log-format COMBINED - > report.html then report.html is generated as expected. When I try to use the web server example from docs cat traefik.log | docker run -p 7890:7890 --rm -i allinurl/goaccess -a -o html --log-format COMBINED --real-time-html - > report.html

here is excempt from docs

Is there anything more I have to do? The way I understand this from the docs I expect this to work.

When I curl localhost:7890 I get the same responses with 400 error as mentioned before.

[aqos156]

@allinurl just to add, here is the telnet output

root@mp-01:~# telnet localhost 7890
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
GET / HTTP/1.0
Host: localhost

HTTP/1.1 400 Invalid Request

Connection closed by foreign host.
root@mp-01:~# 

Also both of the curl commands in this comment work for me, but when I remove any upgrade headers it stops working and returns 400.

[allinurl]

@aqos156 Sorry for the late reply. I wonder if you are behind some sort of proxy? Sounds like an environment issue. Here's what I get on a fresh image: