directus / v8-archive

Directus Database API — Wraps Custom SQL Databases with a REST/GraphQL API
https://docs.directus.io/api/reference.html
505 stars 204 forks source link

Telemetry requests are blocking, potential denial of service #1932

Open MiniDigger opened 4 years ago

MiniDigger commented 4 years ago

See https://github.com/directus/api/issues/1932#issuecomment-645867051

OG Description:

Directus Version: 8.7.2 Running in OpenShift, using the directus:v8-apache docker image, but without running the user as root. DB is mariadb-102-rhel, installed on OpenShift too.

I setup a project via the install interface and I am now trying to login. The app got gateway timeouts, but after further investigation (by trying to auth via localhost and curl), it turns out, that the authenticate endpoint is incredible slow, taking more than 2 minutes to complete:

/var/directus/$ time curl 'localhost:8080/sad-service/auth/authenticate' \
>   -H 'Content-Type: application/json;charset=UTF-8' \
>   --data-binary '{"email":"xxxx","password":"xxxxx","mode":"cookie"}'
{"data":{"user":{"id":"1","status":"active","role":"1","first_name":"Admin","last_name":"User","email":"xxxxxx","timezone":"UTC","locale":"en-US","locale_options":null,"avatar":null,"company":null,"title":null,"external_id":null,"theme":"auto","2fa_secret":null,"password_reset_token":null}},"public":true}
real    2m7.579s
user    0m0.018s
sys     0m0.016s

other requests, like /server/projects or /sad-service/collections (manually acquired a JWT for that) complete normally. I can see all the collections directus created in my db, so its really not a connectivity issue.

the apache log doesnt say anything special (::1 - - [16/Jun/2020:13:31:34 +0200] "POST /sad-service/auth/authenticate HTTP/1.1" 200 882 "-" "curl/7.64.0"), nor do I see anything in directus/logs/ that is related.

MiniDigger commented 4 years ago

After more investigation (aka, adding error_log to code I dont understand, lol), this is where the two minute delay is: https://github.com/directus/api/blob/c2acecb1c5cf34633df2fd12a64650d759b5fd60/src/core/Directus/Services/AuthService.php#L63 in particular, calling listener with id 27 takes ages: https://github.com/directus/api/blob/c2acecb1c5cf34633df2fd12a64650d759b5fd60/src/core/Directus/Hook/Emitter.php#L291 found out that id 27 is auth.request:credentials

MiniDigger commented 4 years ago

ok, it all makes sense now. directus tries to send telemetry stuff here: https://github.com/directus/api/blob/265f492df35788ac1291e3a4bca8ae4908805700/src/core/Directus/Application/CoreServicesProvider.php#L667

that stuff doesn't fly in the env I am running this in (corporate cloud, no internet access without proxy and firewall, etc), so the request just times out.

update directus_settings SET value = 0 where id = 8; fixed it

so I guess my suggestion would be to do this request async/non blocking? I have no idea about php and how one would do that, but even so my setup might not be common, similar issues would arrive if the central telemetry service time out request, potentially taking off every directus installation with telemetry enabled, making the telemetry service a single point of failure and a nice ddos target...