Allow global profiles

[figassis]

When hosting multiple apps, sometimes it's a good idea to have a general set of rules protecting all applications, and then specific rules for different types of applications.

This helps avoid duplicating rules in similar profiles.

One way to do this would be to allow users to configure a parent profile when creating or editing a profile.

[zit-hb]

Thanks for the suggestion. I agree that it would be handy to share some rules with multiple profiles. I also would like to see a way to group rules and assign groups to profiles. But this will require deep changes and much work. It is not likely that this will be added to Shadow Daemon 2 any more. For example, even something like the translations can be a blocker, since it will be hard to find someone to translate the missing parts for all languages. I would like to start Shadow Daemon 3 at some point, this would give the possibility to add as many breaking changes as wanted. This is a private project though, so the time I can spent on it is limited. I do not plan to start on it this year.

[figassis]

Interesting you say that, I loved this project as I am moving away from wordfence. a WAF should be outside the app. But My experience in Golang is far superior to my C++ experience, so I was planning on porting the whole of shadowd to Go and also do away with connectors, so it can work as a reverse proxy and require no application level configuration.

Profiles can be matched to applications by mapping domains to profiles.

I want to make it api based and also provide a repository for community validated profiles and rules so anyone can simply launch their instance and simply assign domains to pre existing profiles, or add their private ones.

With a rest api, people can build their own UIs or integrate into their infrastructures.

[zit-hb]

My ideas are going in a similar direction, Shadow Daemon 3 will also be API-driven. I will stick to the connector approach though (but write new ones). If you try to create a WAF with a reverse proxy approach you have to parse the requests yourself. It is incredibly hard to do this well because many web servers, languages, and frameworks handle certain situations differently (for example, a partially invalid request). Shadow Daemon lets the language/framework parse the request first and then intercepts it. This makes it a bit harder to activate it but I think it is totally worth it. Though, the WAF is really limited to the application, it will not see requests that the web server does not forward to the application. That is fine with me though.

I would love to see a new WAF though! There did not happen a lot in the open-source WAF space in the last years. Also good to have different approaches.

[zit-hb]

Another thing I would like to explore in a new version is to hook functions and parse the input to them. For example, I would like to hook the function that executes MySQL queries and forward the queries to the analysis server. The analysis server parses the queries and extracts some information, e.g. is there normally an UNION SELECT in the query or not. This should be a great source of information if a request is malicious or not.

[figassis]

That would be awesome, but would require so much work by app developers what I would not expect them to adhere so easily.

One way to solve this is to also have a sql proxy. For example there's MySQL Proxy which supports lua scripts for processing both queries and server responses. On useful script is the logging script, which you can decide to ship to wherever you want and then do your analysis. Of course, you lose some context as you don't know which function executed the query, but I think the app developer could easily tell, especially if you can group queries by database, table and other fields, which they could then attach a function name to, via tagging.

You should probably be able to see performance by subtracting time between request and response log entries if they're linked by a unique identifier.

Not sure it would all work until I try, but I just like to avoid making devs change their code to add external integrations unless its impossible to do otherwise.

[figassis]

But I guess your way works for more than just sql queries, you could hook a function to just about anything. Also, to avoid getting to deep into code, most apps are setup with logging functions, where you log with levels INFO, DEBUG, etc. so you could hook into those entries, which in modern apps can be json because ELK.

devs can then call log("ANALYSIS", parameters...), when the function starts/exits, which would give you {"app-version": "xyz", "level": "ANALYSIS", "timestamp": "Sat, 04 Jul 2020 22:36:51.999 UTC", "fields": {"field1": "value", ...}}

And there are tools that can ship this info straight to you. This would not give you the ability to stop a query from running (in that case you really need a hook), but it will provide insight that you may be able to use to on rules. Bonus if you can match http request parameters to sql query parameters so you can link a query to a specific request.