rashidkpc commented 9 years ago

Kibana authentication layer, phase 1

In the first phase of Kibana’s authentication system we’re focus on just that: authentication. That is, a clean way of allowing for login, logout and sessions that doesn’t require HTTP basic auth to be configured or a proxy to be setup. Authentication will be disabled by default, but enabled via kibana.yml

The login screen

Like any other web application this provides the ability to enter a username and password which will be sent to the server for validation. If validation succeeds the user is allowed into the application, if not they are returned to the login screen.

The session

Before even presenting the login screen we need to check if the user is already logged in. This means we need to keep a session.

Because Elasticsearch has no built in authentication mechanism we don’t consider it a secure location for session storage, thus the user’s client will need to be the authority on the session. Here we’ll use an encrypted cookie containing username:password:timestamp which will be encrypted by the server using a key provided in kibana.yml.

Potential pitfall: Session stealing

Because the cookie will contain everything the user needs to login, it will be vulnerable to session stealing like many other applications. To mitigate this concern we 2 lines of defense:

Session timeouts Also configurable in kibana.yml should be the session timeout. The server should validate the timestamp on the cookie and redirect the user to the login screen if it is older than the configured timeout allows. The timestamp on the cookie should be updated on every request. This could be infinite, however we should not recommend this.
Required SSL If the user is using authentication, SSL is required. The server should fail to start if the user does not have SSL turned on, but does have authentication turned on.
Authentication strategies

Regardless of if the user is coming with existing credentials in a cookie, or via a new login, the user will need to be authenticated server side in some way. The user will in fact be re-validated with every request, thus allowing for users to be quickly invalidated simply by removing them from the auth system. Initially we’ll provide a pluggable authentication system with 2 pre-built modules:

.htpasswd This simple method works just like apache’s, or Shield’s user list. A file containing user names and hashed passwords. The kibana backend will check this file and either allow or deny the user.
Basic auth proxy In this case, the user’s credentials will be relayed to Elasticsearch’s / endpoint for authentication. If elasticsearch returns a 200, we let the user in, otherwise the user is logged out and returned to the login screen. In addition, the user’s credentials are included on every request to Elasticsearch from the backend. If at any point we receive a 401, the user will be logged out.
Implementing authentication strategies

The auth strategy module should contain 3 functions:

init(server) Run to initialize the authentication strategy.
authenticate(request, username, password) Run when attempting to establish the session for the first time, if the user does not have a cookie. It should return a promise that succeeds or fails. If it succeeds the user is issued a cookie, otherwise they are returned to the login screen.
validate(request, session) (Optional) Run on every request, returns a promise that succeeds or fails. If it fails the user is logged out. The request can be modified here before being sent, including modifying cookies, headers, etc.
Logout

We should add a logout button to the navigation bar. When the user logs out they should be return to the login screen.

monotek commented 8 years ago

I had asked for the subscription and its just not afordable. When i asked it was around 25000€ / per year. Hopefully this does not lead to a fork of Kibana which would split communities. Would love to see a free plugin for that.

delfuego commented 8 years ago

I couldn't agree more strongly with the above sentiments — putting authentication behind Shield makes it functionally inaccessible to 90+% of potential (and current!) Kibana users. We too asked for quotes for subscriptions, and were absolutely stunned at the cost — for our simple setup, purely used for logging, we'd have to spend nearly $50K a year, so that's the price of not wanting our Kibana data available on our entire network.

@rashidkpc, please PLEASE rethink this decision.

rashidkpc commented 8 years ago

Sorry gang, not my decision to make

jhovell commented 8 years ago

Why isn't a reverse proxy with oauth or any other auth a viable solution for Kibana 4? I set this up on my network and it seems to work fine. There is no ACL obviously. It sounds like for that you need a commercial license. Seems fair enough. What am I missing?

rtoma commented 8 years ago

@rashidkpc, I understand. Its a company decision. Github is not the place to talk to a company, so I did this tweet: https://twitter.com/rtoma_/status/694897025440169984 - Let's see what happens.

elvarb commented 8 years ago

@jhovell that is exactly the point. The only access control you can do with a reverse proxy is authentication and then method calls, limit PUT, POST, DELETE and so on but that does not work as well as it did in Kibana3 as far as I know.

The best we can do for now is limiting to methods on Kibana4 or at worst we will have to limit the methods between Kibana4 and Elastcsearch and run two instances of Kibana4 with different authentication profiles. One for read only and one for read and write.

sobrique commented 8 years ago

I've been experimenting a little with on the fly request rewriting of POSTed data. It's not a perfect ACL by any means, but does mean you can add "should" filters to the various API calls (like _msearch, that Kibana uses a lot) to restrict the result set.

This uses ngx_http_perl module in Nginx - it does effectively mean though, I'm double-proxying Kibana - once at the front to do user authentication, and once at the back to do the authorization elements.

trevan commented 8 years ago

If Kibana allowed a plugin to hook into the "mapUri" piece of the es proxy, then a plugin could hack in authentication to prevent access to certain areas of es. I've done that to limit access to certain indices to certain people and it has worked fairly well.

thalesfsp commented 8 years ago

So, it's possible or not to enable authentication only in Kibana? I'm talking about the basic login/logout screen, user and password. @rashidkpc

ofavre commented 8 years ago

Kibana forwards any basic HTTP auth, if one is required to access the elasticsearch backend. So this enables you to protect a Kibana that is accessible on the Internet. And your elasticsearch backend is only accessed through Kibana since v4, so it is not exposed.

However this alone does not perform any filtering on the requests send to elasticsearch.

For this, you'll have to write a proxy (using nginx with lua scripting, for instance) before elasticsearch, that filters and rewrites requests, and responses if necessary. You can even load per credentials configuration to allow different users to access different indices/aliases, or views. Basically this is like rewriting Shield, intercepting and checking every access.

robison commented 8 years ago

If not your decision to make @rashidkpc, then whose?

kimchy commented 8 years ago

hey, I answered similar questions in the context of Elasticsearch when we released shield. I highly recommend reading it: https://github.com/elastic/elasticsearch/issues/664#issuecomment-73944252, it is valid to this discussion as well. I would also read the comments to what I wrote, I found them to be very constructive.

Nothing much changed since then. We still plan at one point to make parts of shield free, no concrete dates though. We do offer almost of all shield functionality as part of our Elasticsearch as a Service Standard offering (https://www.elastic.co/found) as its simpler there. One thing actually did change, we are now ~150 engineers :).

elvarb commented 8 years ago

Thanks for the update @kimchy, glad that mini shield is on the plans. Would make everyone very happy if that will be available at the same time kibana authentication with big shield is released. One big announcement, useable for all.

delfuego commented 8 years ago

@kimchy On that Elasticsearch as a Service page, Shield is explicitly called out as only part of the Premium service — are you saying that's not the case?

(And that page highlights another issue — it's easy to know how much we'd be in for if we went with the Standard service, but in order to find out more about how much Premium would cost, there's just "contact us". I'm sure I'm not the first, and not even the hundredth, to say that that in and of itself means you lose potential customers... it is much harder to consider a product when there are barriers to getting accurate pricing information at the time the research is being done.)

kimchy commented 8 years ago

@delfuego in the context of Found (our Elasticsearch and Kibana as a Service), agreed, we need to better list the features you get in each lane. Certain aspects are embedded in the Found service regardless (message signing for example). We recently decided to offer most of the features of Shield in our Standard offering (so you can expect RBAC and so on to be part of it regardless of size, and Kibana session management). We are working on updating the site to properly reflect it.

Also, regarding Premium, which includes our dedicated support (see more here: https://discuss.elastic.co/t/typical-response-times-for-found-support/39925/2), we took our time (this is new!) to refine our pricing model for it (as you an imagine, a dedicated support engineer has cost associated with it). I believe now we are in a good place, so definitely considering publishing the pricing.

trevan commented 8 years ago

I wrote up a plugin that enables oauth2 authentication using Bell (https://github.com/hapijs/bell). The plugin is at https://github.com/trevan/oauth2. I believe I can also enhance it to do authorization (such as prevent editing of dashboards/visualizations or restrict which indices are available per user).

megastef commented 8 years ago

@trevan cool! :)

membersheep commented 8 years ago

@trevan thank you for sharing it.

pieterlange commented 8 years ago

Others hitting this issue might also be interested in https://github.com/floragunncom/search-guard

kadishmal commented 8 years ago

We run Kibana behind the proxy and require users to be logged in.

Here is a Kibana plugin (https://github.com/kadishmal/kibana-express-session) that reads the session data from Redis saved previously using express-session in our main app.

The actual login process is handled by our main app outside Kibana.

johnnyshields commented 7 years ago

@kimchy has there been any further clarity on when a free basic version of Shield would become available?

kimchy commented 7 years ago

@johnnyshields no, nothing concrete yet.

bluepuma77 commented 7 years ago

Why it is a good idea to provide basic authentication and authorization. Yes, it's probably the admin's fault, but still the software is delivered with all ports wide open by default.

"Online databases dropping like flies, with >10k falling to ransomware groups" http://arstechnica.com/security/2017/01/more-than-10000-online-databases-taken-hostage-by-ransomware-attackers/