search

javaee / grizzly

Writing scalable server applications in the Java™ programming language has always been difficult. Before the advent of the Java New I/O API (NIO), thread management issues made it impossible for a server to scale to thousands of users. The Grizzly NIO framework has been designed to help developers to take advantage of the Java™ NIO API.
https://javaee.github.io/grizzly/
Other
222 stars 60 forks source link

statistics problem #376

Closed glassfishrobot closed 14 years ago

glassfishrobot commented 15 years ago

i have some difficult while trying to get stats from a grizzlywebserver. Here is a snippet of the code. This code throw a nullpointer on the stats.getThreadPoolStatistics().getCountQueued()); and return 0 for all the other req. btw, i have copy this piece of code somewhere on grizzly blog, but i can't remember where

---snippet- public static void main(String args[]) { GrizzlyWebServer ws = new GrizzlyWebServer(8080, 40,"/Users/ramarama/html"); ScheduledThreadPoolExecutor ste = new ScheduledThreadPoolExecutor(1); final Statistics stats = ws.getStatistics(); stats.startGatheringStatistics(); ste.scheduleAtFixedRate(new Runnable() { public void run() { try

{ System.out.println("Current connected users: " + stats.getKeepAliveStatistics().getCountConnections()); System.out.println("How many requests since startup:" + stats.getRequestStatistics().getRequestCount()); System.out.println("How many connection we queued because of all " + "thread were busy: " + stats.getThreadPoolStatistics().getCountQueued()); }

catch (Exception e)

{ System.out.println(e); }

return; } }, 0, 1,TimeUnit.SECONDS); System.out.println("Grizzly WebServer listening on port 8080"); try

{ ws.start(); }

catch (IOException e)

{ // TODO Auto-generated catch block e.printStackTrace(); }

}

Environment

Operating System: All Platform: Macintosh

Affected Versions

[1.9.3]

glassfishrobot commented 15 years ago

Reported by ramarro@java.net

glassfishrobot commented 15 years ago

jfarcand@java.net said: This is a regression from

https://grizzly.dev.java.net/issues/show_bug.cgi?id=346

fix coming, 1.8.x must not suffer the issue.

glassfishrobot commented 15 years ago

jfarcand@java.net said: First fix

Sending src/main/java/com/sun/grizzly/http/SelectorThread.java Transmitting file data . Committed revision 2170.

glassfishrobot commented 15 years ago

jfarcand@java.net said: Sending src/main/java/com/sun/grizzly/http/DefaultProcessorTask.java Transmitting file data . Committed revision 2171.

glassfishrobot commented 15 years ago

jfarcand@java.net said: Fix target

glassfishrobot commented 15 years ago

ramarro@java.net said: the problem with the nullpointer is solved, but the stats report 0 with my test.

--test-- WebServer w = Listener.istances.get; //webserver extends GrizzlyWebServer Statistics s = w.getStatistics();

println("
ISTANCE "i" Port: "+w.getSelectorThread().getPort()); println("
OpenConnection "+ s.getRequestStatistics().getCountOpenConnections()); println("
Max Open Connection: "+ s.getRequestStatistics().getMaxOpenConnections());

println("
Request Queued (15min avg): "+ s.getThreadPoolStatistics().getCountQueued15MinuteAverage()); println("
Request Queued (total): "+ s.getThreadPoolStatistics().getCountTotalQueued()); println("
Byte Sent: "+ s.getRequestStatistics().getBytesSent()); println("
200 Count: "+ s.getRequestStatistics().getCount200()); println("
404 Count: "+ s.getRequestStatistics().getCount404()); println("

");

all the data are 0, apart from port number that is displayed correctly

glassfishrobot commented 15 years ago

ramarro@java.net said: after a little test i am able to give more informations

--> report 0, even if not correct stats.getRequestStatistics().getBytesSent() (data are send via GrizzlyAdapter using res.getResponse().doWrite(chunk)

--> report 0 getRequestStatistics().something (all req stats report0)

--> report 0 stats.getThreadPoolStatistics().getCountQueued1MinuteAverage() and all the other avg (even if running for more than 15mins, with ab consuming all the threads)

--> report 197 (correct ab is running with -c 200, and there are 3 worker thread) stats.getThreadPoolStatistics().getCountQueued() --> report 0 stats.getThreadPoolStatistics().getCountTotalQueued() (for me is not clear the difference between counttotal and countqueued)

--> report 0 stats.getThreadPoolStatistics().getCountTotalConnections()

--> seems to work getKeepAliveStatistics() (but it's difficult to test with ab, using a browser, the number seems to be correct for getcount, not sure for getcountrefusals)

best regards

glassfishrobot commented 15 years ago

jfarcand@java.net said: Salut,

I've updated the test case/demo located:

http://download.java.net/maven/2/com/sun/grizzly/samples/grizzly-embed-samples/1.9.5-SNAPSHOT/

public class GrizzlyEmbedWebServer {

// Simple scheduler that will outpot stats every 5 seconds. private static ScheduledThreadPoolExecutor ste = new ScheduledThreadPoolExecutor(1);

public static void main( String args[] ) throws Exception { String path = args[0]; if (args[0] == null || path == null)

{ System.out.println("Invalid static resource path"); System.exit(-1); }

GrizzlyWebServer ws = new GrizzlyWebServer(path); ws.enableJMX(new Management() {

public void registerComponent(Object bean, ObjectName oname, String type) throws Exception

{ Registry.getRegistry().registerComponent(bean,oname,type); }

public void unregisterComponent(ObjectName oname) throws Exception

{ Registry.getRegistry(). unregisterComponent(oname); }

});

final Statistics stats = ws.getStatistics(); stats.startGatheringStatistics();

ste.scheduleAtFixedRate(new Runnable() { public void run()

{ System.out.println("Current connected users: " + stats.getKeepAliveStatistics().getCountConnections()); System.out.println("How many requests since startup:" + stats.getRequestStatistics().getRequestCount()); System.out.println("How many connection we queued because of all" + "thread were busy: " + stats.getThreadPoolStatistics().getCountQueued()); System.out.println("Max Open Connection: "+ stats.getRequestStatistics().getMaxOpenConnections()); System.out.println("Request Queued (15min avg): "+ stats.getThreadPoolStatistics().getCountQueued15MinuteAverage()); System.out.println("Request Queued (total): "+ stats.getThreadPoolStatistics().getCountTotalQueued()); System.out.println("Byte Sent: "+ stats.getRequestStatistics().getBytesSent()); System.out.println("200 Count: "+ stats.getRequestStatistics().getCount200()); System.out.println("404 Count: "+ stats.getRequestStatistics().getCount404()); return; }

}, 0, 10,TimeUnit.SECONDS); System.out.println("Grizzly WebServer listening on port 8080"); ws.start(); }

I just do:

java -jar grizzly-embed-samples-1.9.5-SNAPSHOT.jar /s1/domains/doma in1/docroot/

and do requests and the stats will be displayed on the console.

For me it works. Which version of grizzly are you using? I'm closing it as fixed, please re-open a new one as I suspect this is not the same.

glassfishrobot commented 15 years ago

ramarro@java.net said: seems that if i don't use the ws.enableJMX(new Management() {

public void registerComponent(Object bean, ObjectName oname, String type) throws Exception

{ Registry.getRegistry().registerComponent(bean,oname,type); }

public void unregisterComponent(ObjectName oname) throws Exception

{ Registry.getRegistry(). unregisterComponent(oname); }

});

the stats aren't displayed.

Is the enableJMX management required for use statistics?

best regards

glassfishrobot commented 15 years ago

jfarcand@java.net said: This has already been fixed by:

https://grizzly.dev.java.net/issues/show_bug.cgi?id=423

Which version are you using?

glassfishrobot commented 15 years ago

ramarro@java.net said: grizzly-http-webserver-1.9.8.jar

sorry for the long post, this is the JSP page that should display the stats

on Listener.istance i put every istance that i create (i support multiple spawn on different port for webserver)

Also, this "work" more or less, cos some data are displayed :S

Tomorrow, i'll do a more simple test (without using the jsp, just print on stdout) and let you know what's up

<%@ page import="com.eg.euler.gs.Block, com.eg.euler.ipc.http.Listener, com.eg.euler.ipc.http.User, com.eg.euler.ipc.http.newHttp.WebServer"%> <%@ page import="com.eg.euler.ipc.pull.Login" %> <%@ page import="com.sun.grizzly.http.embed.Statistics" %> <%@ page import="java.util.concurrent.Callable" %> <%@ page import="java.util.concurrent.ScheduledThreadPoolExecutor" %> <%@ page import="java.util.concurrent.TimeUnit" %> <%@ page extends="com.eg.euler.ehtml.EServlet" %>

<% for (int i = 0;i
Application Server Stats (Port <%=w.getSelectorThread().getPort()%>)
KeepAlive
ThreadPool
Connections Queued <%=stats.getThreadPoolStatistics().getCountQueued()%>
Requested Queued (15m/avg) <%=stats.getThreadPoolStatistics().getCountQueued15MinuteAverage()%>
Total Connections <%=stats.getThreadPoolStatistics().getCountTotalConnections()%>
Total Queued <%=stats.getThreadPoolStatistics().getCountTotalQueued()%>
Peak Queued <%=stats.getThreadPoolStatistics().getPeakQueued()%>
Other
Restart This SelectorThread


<% } %>
glassfishrobot commented 15 years ago

ramarro@java.net said: ok, i have tested again, using your example, i get

Current connected users: 0 How many requests since startup:32733 How many connection we queued because of allthread were busy: 0 Max Open Connection: 0 Request Queued (15min avg): 0 Request Queued (total): 0 Byte Sent: 0 200 Count: 32409 404 Count: 0

Numbers are for sure incorrect (can't say nothing about 200 count or req count, but for sure keepalive stats, max conn queued and byte sent are wrong)

This is how i spawn the webserver

public class WebServer extends GrizzlyWebServer {

private final String root1; private final String root2; private final int cacheMin; private final Parser2 parser; private final Element CONF;

/default settings for file caching & error msg/ protected ConcurrentHashMap<String, File> cache = new ConcurrentHashMap<String, File>(); protected static final String NOTFOUNDMSG = "

ERROR 404 - PAGE NOT FOUND

"

  • "Oops! Looks like the page you're looking for was moved or never existed. "
  • "Make sure you typed the correct URL or followed a valid link.
";

/settings for pushing System/ static final public ConcurrentLinkedQueue toPush = new ConcurrentLinkedQueue();

public WebServer(int port, String root1,String root2,int cache, Parser2 parser,Element CONF,boolean secure,int maxThread)

{ super(port,"",secure); this.setMaxThreads(maxThread); this.root1 = root1; this.root2 = root2; this.cacheMin = cache; this.parser = parser; this.CONF = CONF; Util.warn("Starting WS on "+port, WebServer.class.getName()); init(secure); }

private static ScheduledThreadPoolExecutor ste = new ScheduledThreadPoolExecutor(1); public void init(boolean secure) { //this.useAsynchronousWrite(true); this.getSelectorThread().setDisplayConfiguration(true); this.getStatistics().startGatheringStatistics(); this.getSelectorThread().setCompression("on"); this.getSelectorThread().setSelectorReadThreadsCount(6); this.addGrizzlyAdapter(new EulerResAdapter(root1, root2,cacheMin,parser,CONF,cache),new String[]

{"/Adapter-1"}

); this.addAsyncFilter(new EulerAsyncAdapter());

final Statistics stats = this.getStatistics(); stats.startGatheringStatistics();

ste.scheduleAtFixedRate(new Runnable() { public void run()

{ System.out.println("Current connected users: " + stats.getKeepAliveStatistics().getCountConnections()); System.out.println("How many requests since startup:" + stats.getRequestStatistics().getRequestCount()); System.out.println("How many connection we queued because of all" + "thread were busy: " + stats.getThreadPoolStatistics().getCountQueued()); System.out.println("Max Open Connection: "+ stats.getRequestStatistics().getMaxOpenConnections()); System.out.println("Request Queued (15min avg): "+ stats.getThreadPoolStatistics().getCountQueued15MinuteAverage()); System.out.println("Request Queued (total): "+ stats.getThreadPoolStatistics().getCountTotalQueued()); System.out.println("Byte Sent: "+ stats.getRequestStatistics().getBytesSent()); System.out.println("200 Count: "+ stats.getRequestStatistics().getCount200()); System.out.println("404 Count: "+ stats.getRequestStatistics().getCount404()); return; }

}, 0, 10, TimeUnit.SECONDS); } }

eulerresadapter basically do

fis = new FileInputStream(resource); byte b[] = new byte[16384]; ByteChunk chunk = new ByteChunk(); int rd=0; while ((rd=fis.read(b)) > 0)

{ chunk.setBytes(b, 0, rd); res.getResponse().doWrite(chunk); }

(could be useful to track why sent byte is wrong)

glassfishrobot commented 15 years ago

jfarcand@java.net said: Fix millestone

glassfishrobot commented 15 years ago

jfarcand@java.net said: Incremental fix

Sending src/main/java/com/sun/grizzly/http Sending src/main/java/com/sun/grizzly/http/ProcessorTask.java Transmitting file data . Committed revision 2675.

glassfishrobot commented 15 years ago

jfarcand@java.net said: Fixed.

Sending http/src/main/java/com/sun/grizzly/http/SelectorThreadHandler.java Sending http/src/main/java/com/sun/grizzly/http/SelectorThreadKeyHandler.java Sending http/src/main/java/com/sun/grizzly/http/ThreadPoolStatistic.java Transmitting file data ... Committed revision 2676.

glassfishrobot commented 15 years ago

ramarro@java.net said: Still not working

I'll submit a fully "working" class

The result of this class is Current connected users: 0 [WRONG] (req was on keepalive) How many requests since startup:12278 How many connection we queued because of allthread were busy: 0 [WRONG] (for sure, req was queued, since i have used ab -c 250 -n 1000 blablabla) Max Open Connection: 0 [i guess that this is wrong] Request Queued (15min avg): 0 [it's depend on how it's calculated, but probably wrong] Request Queued (total): 0 [FOR SURE WRONG] Byte Sent: 0 [WRONG FOR SURE] 200 Count: 12308 404 Count: 0

The code seems to be correct, more or less is copied by the example on the blog. best regards!

import com.sun.grizzly.http.embed.GrizzlyWebServer; import com.sun.grizzly.http.embed.Statistics; import com.sun.grizzly.tcp.http11.GrizzlyAdapter; import com.sun.grizzly.tcp.http11.GrizzlyRequest; import com.sun.grizzly.tcp.http11.GrizzlyResponse; import com.eg.euler.ipc.http.newHttp.EulerAsyncAdapter; import java.util.concurrent.ScheduledThreadPoolExecutor; import java.util.concurrent.TimeUnit; import java.io.IOException;

public class test extends GrizzlyWebServer {

private static ScheduledThreadPoolExecutor ste = new ScheduledThreadPoolExecutor(1);

test() { this.getSelectorThread().setDisplayConfiguration(true); this.getStatistics().startGatheringStatistics(); this.getSelectorThread().setCompression("on"); this.getSelectorThread().setSelectorReadThreadsCount(6); this.addGrizzlyAdapter(new GrizzlyAdapter() { public void service(GrizzlyRequest grizzlyRequest, GrizzlyResponse grizzlyResponse) { try

{ grizzlyResponse.getWriter().write("123456"); }

catch (Exception e) {

} } },new String[]

{"/"}

); this.addAsyncFilter(new EulerAsyncAdapter());

final Statistics stats = this.getStatistics(); stats.startGatheringStatistics();

ste.scheduleAtFixedRate(new Runnable() { public void run()

{ System.out.println("Current connected users: " + stats.getKeepAliveStatistics().getCountConnections()); System.out.println("How many requests since startup:" + stats.getRequestStatistics().getRequestCount()); System.out.println("How many connection we queued because of all" + "thread were busy: " + stats.getThreadPoolStatistics().getCountQueued()); System.out.println("Max Open Connection: "+ stats.getRequestStatistics().getMaxOpenConnections()); System.out.println("Request Queued (15min avg): "+ stats.getThreadPoolStatistics().getCountQueued15MinuteAverage()); System.out.println("Request Queued (total): "+ stats.getThreadPoolStatistics().getCountTotalQueued()); System.out.println("Byte Sent: "+ stats.getRequestStatistics().getBytesSent()); System.out.println("200 Count: "+ stats.getRequestStatistics().getCount200()); System.out.println("404 Count: "+ stats.getRequestStatistics().getCount404()); return; }

}, 0, 10, TimeUnit.SECONDS); try

{ this.start(); }

catch (IOException e)

{ e.printStackTrace(); }

} public static void main(String args[])

{ new test(); }

}

glassfishrobot commented 15 years ago

jfarcand@java.net said: Hum, are you sure you are using 1.9.9? Can you build the workspace from scratch? The sample under samples/http/http-embed is showing the proper data....

Thanks!!

glassfishrobot commented 15 years ago

ramarro@java.net said: using grizzly-http-webserver-1.9.9.jar (file size: 799964) downloaded a couple of hrs ago.

Workspace was rebuilded, but indeed this is a "singleclass" stupid gitzmo that i have write right now, so i am pretty sure that it was compiled and using grizzly 1.9.9

have you had a test with this class? the last example is a "copy paste" so it will be quite easy to test.

best regards

glassfishrobot commented 15 years ago

jfarcand@java.net said: Hum, using the demo under sample/http-embed, I do get the proper statistic. Can you take a look and tell me the demo doesn't work for you? Thanks!!

glassfishrobot commented 15 years ago

ramarro@java.net said: Using the demo example i got

Current connected users: 0 How many requests since startup:392913 How many connection we queued because of allthread were busy: 0 Max Open Connection: 0 Request Queued (15min avg): 0 Request Queued (total): 0 Byte Sent: 0 200 Count: 289480 404 Count: 28599

There are too many 0s to be correct, for sure byte sent, max open connection, and request queued are wrong.

best regards

glassfishrobot commented 15 years ago

jfarcand@java.net said: Rama, sorry for not being able to reproduce. From you log I see Byte Sent always 0, which is supposed to be fixed. I've just svn up and I'm getting

jfarcand@jfarcand-desktop:/appserv80/grizzly/trunk/samples/http/http-embed$ java -jar target/grizzly-embed-samples-1.9.11-SNAPSHOT.jar /s1/domains/d omain1/docroot/ Grizzly WebServer listening on port 8080

Current connected users: 0 How many requests since startup:0 How many connection we queued because of allthread were busy: 0 Max Open Connection: 0 Request Queued (15min avg): 0 Request Queued (total): 0 Byte Sent: 0 200 Count: 0 404 Count: 0

Current connected users: 0 How many requests since startup:2100 How many connection we queued because of allthread were busy: 0 Max Open Connection: 100 Request Queued (15min avg): 0 Request Queued (total): 0 Byte Sent: 10107300 200 Count: 2100 404 Count: 0

when doing:

jfarcand@jfarcand-desktop:~/appserv80/jersey/trunk/jersey$ ab -k -c100 -n 2000 http://localhost:8080/index.html This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0 Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient) Completed 200 requests Completed 400 requests Completed 600 requests Completed 800 requests Completed 1000 requests Completed 1200 requests Completed 1400 requests Completed 1600 requests Completed 1800 requests Finished 2000 requests

Server Software: Server Hostname: localhost Server Port: 8080

Document Path: /index.html Document Length: 4813 bytes

Concurrency Level: 100 Time taken for tests: 0.498561 seconds Complete requests: 2000 Failed requests: 0 Write errors: 0 Keep-Alive requests: 2000 Total transferred: 9879710 bytes HTML transferred: 9626000 bytes Requests per second: 4011.55 /sec (mean) Time per request: 24.928 [ms] (mean) Time per request: 0.249 [ms] (mean, across all concurrent requests) Transfer rate: 19351.69 [Kbytes/sec] received

Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.4 0 3 Processing: 1 23 19.1 18 108 Waiting: 1 23 19.0 18 107 Total: 1 23 19.4 18 110

Percentage of the requests served within a certain time (ms) 50% 18 66% 24 75% 29 80% 32 90% 39 95% 59 98% 101 99% 106 100% 110 (longest request)

glassfishrobot commented 15 years ago

jfarcand@java.net said: Rama, any update? I want this to be fixed in 1.9.11

glassfishrobot commented 15 years ago

ramarro@java.net said: Hi!

Unfortunatly i haven't any update for this issue, using the 1.9.11 snapshot (http://download.java.net/maven/2/com/sun/grizzly/grizzly-http-webserver/1.9.11- SNAPSHOT/grizzly-http-webserver-1.9.11-20090403.105011-22.jar)

i am still having the problem that i have already explain here This data are wrong for sure: Byte Sent: 0 Request Queued (15min avg): 0 Request Queued (total): 0

This data are provided with a ultra simple test class

I can also say that this problem isn't platform dependant (try on linux) and not related to java VM (try with 1.5 on mac and 1.6 on linux)

also, when the test class is up and running (and it's recompiled every time that i launch it) i do an ab -c 100 -n 1000 http://localhost:8080/test.fla

please notice that test.fla exist, and i receive it correctly (doing some calculation on ab result or using a webbrowser)

getting a big file will cause request to be queued (not displayed) and byte sent to be > 0.

here is the class test

public class test {

// Simple scheduler that will outpot stats every 5 seconds. private static ScheduledThreadPoolExecutor ste = new ScheduledThreadPoolExecutor(1);

public static void main( String args[] ) throws Exception { String path = "/Users/ramarama/html";

GrizzlyWebServer ws = new GrizzlyWebServer(path); final Statistics stats = ws.getStatistics(); stats.startGatheringStatistics();

ste.scheduleAtFixedRate(new Runnable() { public void run()

{ System.out.println("Current connected users: " + stats.getKeepAliveStatistics().getCountConnections()); System.out.println("How many requests since startup:" + stats.getRequestStatistics().getRequestCount()); System.out.println("How many connection we queued because of all" + "thread were busy: " + stats.getThreadPoolStatistics().getCountQueued()); System.out.println("Max Open Connection: "+ stats.getRequestStatistics().getMaxOpenConnections()); System.out.println("Request Queued (15min avg): "+ stats.getThreadPoolStatistics().getCountQueued15MinuteAverage()); System.out.println("Request Queued (total): "+ stats.getThreadPoolStatistics().getCountTotalQueued()); System.out.println("Byte Sent: "+ stats.getRequestStatistics().getBytesSent()); System.out.println("200 Count: "+ stats.getRequestStatistics().getCount200()); System.out.println("404 Count: "+ stats.getRequestStatistics().getCount404()); return; }

}, 0, 1,TimeUnit.SECONDS); System.out.println("Grizzly WebServer listening on port 8080"); ws.start(); } }

glassfishrobot commented 15 years ago

jfarcand@java.net said: Salut, running your code I'm getting:

Current connected users: 0 How many requests since startup:11 How many connection we queued because of allthread were busy: 0 Max Open Connection: 1 Request Queued (15min avg): 0 Request Queued (total): 0 Byte Sent: 52943 200 Count: 11 404 Count: 0

Current connected users: 0 How many requests since startup:11 How many connection we queued because of allthread were busy: 0 Max Open Connection: 1 Request Queued (15min avg): 0 Request Queued (total): 0 Byte Sent: 52943 200 Count: 11 404 Count: 0

Current connected users: 0 How many requests since startup:11 How many connection we queued because of allthread were busy: 0 Max Open Connection: 1 Request Queued (15min avg): 0 Request Queued (total): 0 Byte Sent: 52943 200 Count: 11 404 Count: 0

Current connected users: 0 How many requests since startup:11 How many connection we queued because of allthread were busy: 0 Max Open Connection: 1 Request Queued (15min avg): 0 Request Queued (total): 0 Byte Sent: 52943 200 Count: 11 404 Count: 0

Current connected users: 0 How many requests since startup:11 How many connection we queued because of allthread were busy: 0 Max Open Connection: 1 Request Queued (15min avg): 0 Request Queued (total): 0 Byte Sent: 52943 200 Count: 11 404 Count: 0

I think Hudson is not uploading the artifacts and that's why you can't test it. I'm closing the issue as fixed and will make sure the artifact are properly uploaded. Please don't hesitate to re-opne with more information. I'm using Ubuntu BTW.

glassfishrobot commented 15 years ago

Was assigned to jfarcand@java.net

glassfishrobot commented 7 years ago

This issue was imported from java.net JIRA GRIZZLY-376

glassfishrobot commented 14 years ago

Marked as fixed on Wednesday, December 16th 2009, 6:14:16 pm