MySQL Backend Performance Testing with apache benchmark (ab)

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Create a MySQL DB/Table that will be consumed by perservere w/ a table
address:
+-------------+------------------+------+-----+---------+----------------+
| Field       | Type             | Null | Key | Default | Extra          |
+-------------+------------------+------+-----+---------+----------------+
| id          | int(10) unsigned | NO   | PRI | NULL    | auto_increment | 
| street_name | text             | NO   |     | NULL    |                | 
| street_dir  | text             | YES  |     | NULL    |                | 
| city        | text             | NO   |     | NULL    |                | 
| county      | text             | YES  |     | NULL    |                | 
| state       | text             | YES  |     | NULL    |                | 
| country     | text             | NO   |     | NULL    |                | 
| postal_code | text             | NO   |     | NULL    |                | 
+-------------+------------------+------+-----+---------+----------------+
2. Point to it in config/mysql-db.json
        "name": "address",
        "sourceClass": "org.persvr.datasource.DatabaseTableDataSource",
        "driver": "com.mysql.jdbc.Driver",
        "connection":
"jdbc:mysql://localhost/persevere?user=root&password=root",
        "table": "address",
        "idColumn": "id",
        "camelCaseColumnNames": false,
        "dataColumns": ["street_name", "street_dir", "city", "county",
"state", "country", "postal_code"],
        "schema": {
            "prototype": {},
            "data": {
                "$ref": "../address/"
            }
        }

3. Start up persevere and point to http://localhost:8080/address (should
display empty set if there is no data in DB)
4. Run ab against it with the command line:
ab -r -n 1000000 -c 100 -T "application/javascript" -p post_data.txt
http://atgdev.ca.com:8080/address
and post_data.txt:
[{street_name:'1 CA
Plaza',street_dir:'',city:'Islandia',state:'NY',county:'Suffolk',country:'USA',p
ostal_code:'11749'}]

What is the expected output? What do you see instead?
1,000,000 records should be created.
Only 150k records or so get created and then eventually it 'stops responding'.
ab errors out with:
Benchmarking localhost (be patient)
Completed 100000 requests
apr_poll: The timeout specified has expired (70007)
Total of 156495 requests completed

On the server we see:
Aug 25, 2009 10:44:17 PM org.persvr.remote.PersevereFilter$DefaultHandler call
WARNING:
java.lang.OutOfMemoryError: Java heap space
        at java.util.LinkedHashMap.createEntry(LinkedHashMap.java:424)
        at java.util.LinkedHashMap.addEntry(LinkedHashMap.java:406)
        at java.util.HashMap.put(HashMap.java:385)
        at
sun.util.resources.OpenListResourceBundle.loadLookup(OpenListResourceBundle.java
:118)
        at
sun.util.resources.OpenListResourceBundle.loadLookupTablesIfNecessary(OpenListRe
sourceBundle.java:97)
        at
sun.util.resources.OpenListResourceBundle.handleGetObject(OpenListResourceBundle
.java:58)
        at
sun.util.resources.TimeZoneNamesBundle.handleGetObject(TimeZoneNamesBundle.java:
59)
        at java.util.ResourceBundle.getObject(ResourceBundle.java:378)
        at java.util.ResourceBundle.getObject(ResourceBundle.java:381)
        at java.util.ResourceBundle.getStringArray(ResourceBundle.java:361)
        at
sun.util.TimeZoneNameUtility.retrieveDisplayNames(TimeZoneNameUtility.java:100)
        at
sun.util.TimeZoneNameUtility.retrieveDisplayNames(TimeZoneNameUtility.java:81)
        at java.util.TimeZone.getDisplayNames(TimeZone.java:399)
        at java.util.TimeZone.getDisplayName(TimeZone.java:350)
        at java.text.SimpleDateFormat.subFormat(SimpleDateFormat.java:1110)
        at java.text.SimpleDateFormat.format(SimpleDateFormat.java:899)
        at java.text.SimpleDateFormat.format(SimpleDateFormat.java:869)
        at java.text.DateFormat.format(DateFormat.java:316)
        at
org.persvr.remote.PersevereFilter$DefaultHandler.call(PersevereFilter.java:855)
        at org.persvr.remote.PersevereFilter.doFilter(PersevereFilter.java:357)
        at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.jav
a:1115)
        at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:361)
        at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
        at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
        at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
        at
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
        at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollecti
on.java:230)
        at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
        at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
        at org.mortbay.jetty.Server.handle(Server.java:324)
        at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)
        at
org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:879)

What version of the product are you using? On what operating system?
Ubuntu 9.04 w/ verion 1.0 beta 6.

Please provide any additional information below.
When run against a similarly configured JavascriptDB backend, the 1,000,000
records complete as expected.

Original issue reported on code.google.com by beau.cro...@gmail.com on 27 Aug 2009 at 3:12

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

This may be solved by #220

Original comment by kris...@gmail.com on 29 Aug 2009 at 3:12

[GoogleCodeExporter]

The fix for #220 is checked now, have you had a chance to rerun this test with 
the 
updated code? (Note that DatabaseTableDataSource was updated with the changes 
in 
PooledDataSourceTable, so you can still use DatabaseTableDataSource or 
MySqlSource)

Original comment by kris...@gmail.com on 5 Sep 2009 at 2:43

• Changed state: Accepted