search

karakun / OpenWebStart

Run Web Start based applications after the release of Java 11
https://openwebstart.com
Other
417 stars 48 forks source link

Apache LoadBalancer with OpenWebStart: Session Stickiness Problem #244

Closed vskladov closed 4 years ago

vskladov commented 4 years ago

Hi,

I use OpenWebStart at the moment to run our WebStart applications. But I have a problem with session stickiness, trying to build a Apache LoadBalancer Cluster. The required cookie, configured in a LoadBalancer, is not being sent with the request.

The problem appears only while using OpenWebStart, it works fine with Oracle JDK8. It seems to do with the following note I found on the RedHat site: "IcedTea-Web does not support sending cookie information (such as JSESSIONID)...".

Unfortunately, I can't find any info or how-to, pointing me in the right direction, how this issue could be fixed, if ever. I do not believe, that OpenWebStart can not be used in LoadBalancer environment. Would be grateful for any tip helping to solve this problem.

My environment: two application servers (cluster members) and one LoadBalancer, Ubuntu 16.04, Apache 2.4.18, Tomcat 8.0.32, mod_jk 1.2.41. OpenWebStart 1.1.7 on the client side (Windows 10).

Thank you in advance!

Best wishes, Viktor

sclassen commented 4 years ago

We are sorry, but since the plugin support has been dropped there is no integration with the browser. Therefore we cannot access any cookie values or similar places where the JSESSIONID is stored.

It is possible to work around this limitation in an corporate environment. Please contact openwebstart@karakun.com if you are in need for consultancy in order to implement such a custom solution

vskladov commented 4 years ago

I fixed it. The solution that works for me was simply to install a system wide CookieHandler, that cares for reading cookies from response (e.g. JSESSIONID from Tomcat) and adding them to every request.

sclassen commented 4 years ago

Could you give some more details. This would help others with the same setup

vskladov commented 4 years ago

OK, below more details about my environment and how exactly I solved the problem.

My environment:

Step 1) Configuring Apache Load Balancer (requires modules mod_proxy, mod_proxy_balancer, mod_proxy_ajp, mod_lbmethod_byrequests): Add the folllowing directives in your Virtual Host, usually placed in /etc/apache/sites-available (note, I use ajp protocol with port 8009 for communication between load balancer and Tomcat). Important: if you'd like your client sessions to be sticky, this means that all requests of your clients should be always redirected by balancer to the same app server, because e.g. you save any information on the server during the session, add stickysession=JSESSIONID directive to the load balancer configuration as shown below. I use cookie based stickyness, alternatively URL encoding session stickyness can be used, please consult Apache LoadBalancer docs for that.

    <Proxy balancer://my_cluster>
            BalancerMember ajp://appserver1.mydomain.de:8009 loadfactor=50 retry=60 max=8 smax=4 ttl=300 timeout=900 route=app1
            BalancerMember ajp://appserver2.mydomain.de:8009 loadfactor=50 retry=60 max=8 smax=4 ttl=300 timeout=900 route=app2
            ProxySet lbmethod=byrequests stickysession=JSESSIONID      
    </Proxy>

    ProxyPass /webapp1/ balancer://my_cluster/webapp1/
    ProxyPassReverse /webapp1/ balancer://my_cluster/webapp1/
    ProxyPass /webapp2/ balancer://my_cluster/webapp2/
    ProxyPassReverse /webapp2/ balancer://my_cluster/webapp2/
    ProxyPass / balancer://my_cluster/webapp1/
    ProxyPassReverse / balancer://my_cluster/webapp1/

Step 2) Activate AJP 1.3 Connector and add jvmRoute attribute to Engine in Tomcat's server.xml (/etc/tomcat8) on every application server. jvmRoute should match the route value in BalancerMember directive (app1 on appserver1, app2 on appserver 2 an so on):

<Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />
<Engine name="Catalina" defaultHost="localhost" jvmRoute="app1">

How it works, or at least, how it should work:

once a new session is opened (usually it happens with a very first request from a client), tomcat sends a session id information as a JSESSIONID cookie (the name configurable) in http response back to the client, adding a jvmRoute value, as configured in its Engine, to the end of the string. It looks like this:

JSESSIONID=7042C8058708FF5202E2E77D53061944.app1; Path=/webapp1/; HttpOnly

And now follows the most important part of it all: this cookie, JSESSIONID, must be sent by your client WebStart application back to the load balancer. This is very important, because the Load Balancer looks up the configured cookie name (in our case JSESSIONID) in incoming requests from the client, extracts the value of the cookie and looks for a member worker with route equal to that value. This mechanism secures that the client always stays on the same application server. Note: if we don't care about the session id, or if your back-end doesn't send the route information, we can ask Apache itself to inform us about the route the load balancer had initially chosen, using mod_headers and its "Header add Set-Cookie" directive. Please see configuration examples in Apache Load Balancer docs. In our case it does't actually matter how exactly the session cookie will be sent to the client.

And now the problem:

the session cookie, sent by the back-end (Tomcat or Apache), is not added by Java application on the client side to the subsequent requests and therefore does not come back to the load balancer. The balancer does not find the route information and decides by itself to which app server the client will be forwarded. Thus the client can end up with every request on the other app machine! But this is exactly what we'd like to avoid.

Solution:

To solve this issue I ended up with the configuring a system-wide CookieHandler in a WebStart Java application. CookieHandler class has a purpose of providing a callback mechanism for the benefit of the HTTP protocol handler. It retrieves cookies from the response headers of the HTTP response from the given URI every time a response is received and adds them to the requestHeaders just before a request is made. Please see JDK Api for further details.

Let's look at how it could be used based on my example:

import java.net.CookieHandler;
import java.net.CookieManager;
import java.net.CookiePolicy;
import java.net.CookieStore;
import java.net.URL;
import java.net.URLConnection;

public class ServletConnect {
    /* servlet we want to send our data to */
    private URL servlet = null;
    /* CookieManager object */
    static java.net.CookieManager cookieManager = null;

    public ServletConnect(String url) {
        // Installing a system-wide CookieHandler 
        // should be made before the connection to servlet is opened
        initCookieManager();
        servlet = new URL(url);
    }

    public DataFromServer sendToServlet(RequestFromClient out)  {
           // open connection to servlet
           URLConnection toServletCon = servlet.openConnection();
           toServletCon.setDoInput(true);
           toServletCon.setDoOutput(true);
           toServletCon.setUseCaches(false);
           toServletCon.setDefaultUseCaches(false);
           toServletCon.setRequestProperty("Content-Type", "application/octet-stream");
           // send object to the servlet
           ObjectOutputStream output = new ObjectOutputStream(toServletCon.getOutputStream());
           output.writeObject(out);
           output.flush();
           output.close();

           // receive data from servlet
    }

     // Initializing CookieHandler and CookieManager
     static void initCookieManager() {

         if(cookieManager == null) {
             // Create a new cookie manager with specified cookie store and cookie policy.
             // A CookieStore object represents a storage for cookie. Can store and retrieve cookies. 
             // if store parameter is null, cookie manager will use a default one, 
             // which is an in-memory CookieStore implementation
             cookieManager = new CookieManager(null, CookiePolicy.ACCEPT_ALL);

             // Registering a system-wide CookieHandler 
             // CookieHandler class has a purpose of providing a callback mechanism 
             // for the benefit of the HTTP protocol handler.
             // It's method put(uri, responseHeaders) saves any cookies to the cookie store. 
             // These cookies are retrieved from the response headers of the HTTP response from the given URI. 
             // It's called every time a response is received.
             // A related API method – get(uri,requestHeaders) retrieves the cookies saved under the given URI 
             // and adds them to the requetHeaders. It's called just before a request is made.

             // Sets (or unsets) a system-wide cookie handler.
             CookieHandler.setDefault(cookieManager);
         }

     }
}

That's it. The Cookie Handler cares about retrieving of response cookies, saving them in its Cookie Store and adding them to every subsequent request. You don't need to do it by yourself, the Cookie Handler cares about it. All you need to do is configuring it in a right way. Now, the Load Balancer works great, all sessions remain sticky!

I would be glad, if my solution could help anybody in solving similar issues. Sure you can find more configuration examples of CookieHandler and adjust it to your needs, but for me it was quite enough to solve the session stickyness problem in my environment.

Concluding notes

As I wrote in my very first comment, this problem appears only when using OpenWebStart, with Oracle JRE 1.8 it works as expected out of the box. I suppose (I am not sure, just supposing), that CookieHandler (or similar mechanism) is already implemented in Oracle Java per default... Anyway, the guys from Karakun are doing a great job and give our WebStart tools a second chance ... :-)

sclassen commented 4 years ago

So the sticky session is for the communication of the application with a backend server?

I initially understood the issue is about having a sticky session for downloading the resources (jars). For downloading the resources we currently do not support cookies.

vskladov commented 4 years ago

So the sticky session is for the communication of the application with a backend server?

Yes

I initially understood the issue is about having a sticky session for downloading the resources (jars). For downloading the resources we currently do not support cookies.

It could be of importance for somebody, but in our case it does not matter from which server the jars are downloaded.