"View Results Tree" fails to load saved results

[asfimport]

Mike Hampton (Bug 30779): I added a "View Results Tree" listener to my jmeter test, and specified a file in the filename textfield. I ran the test and verified that the file existed and had data. After closing and then opening jmeter, i tried to load the file into the listener, but nothing appeared in the gui.

I closed jmeter, and modified the "jmeter.properties" to have "log_level.jmeter=DEBUG".

I opened jmeter, added a new "View Results Tree" listener into an empty test plan and tried to open the file again. I saw a NumberFormatException in the jmeter.log.

The log snippet follows:

2004/08/20 13:21:49 INFO - jmeter.visualizers.gui.AbstractVisualizer: getting new collector 2004/08/20 13:21:49 DEBUG - jmeter.gui.AbstractJMeterGuiComponent: setting element to enabled: true 2004/08/20 13:21:49 DEBUG - jmeter.visualizers.ViewResultsFullVisualizer: Start : clear1 2004/08/20 13:21:49 DEBUG - jmeter.visualizers.ViewResultsFullVisualizer: clear1 : total child - 0 2004/08/20 13:21:49 DEBUG - jmeter.visualizers.ViewResultsFullVisualizer: End : clear1 2004/08/20 13:21:50 DEBUG - jmeter.visualizers.gui.AbstractVisualizer: Error occurred while loading file java.lang.NumberFormatException: <?xml version="1.0" encoding ="UTF-8"?> at java.lang.Long.parseLong(Long.java:305) at java.lang.Long.parseLong(Long.java:358) at org.apache.jmeter.save.SaveService.makeResultFromDelimitedString (SaveService.java:298) at org.apache.jmeter.reporters.ResultCollector.loadExistingFile (ResultCollector.java:186) at org.apache.jmeter.visualizers.gui.AbstractVisualizer.stateChanged (AbstractVisualizer.java:231) at org.apache.jmeter.gui.util.FilePanel.fireFileChanged (FilePanel.java:128) at org.apache.jmeter.gui.util.FilePanel.actionPerformed (FilePanel.java:141) at javax.swing.AbstractButton.fireActionPerformed (AbstractButton.java:1445) at javax.swing.AbstractButton$ForwardActionEvents.actionPerformed (AbstractButton.java:1499) at javax.swing.DefaultButtonModel.fireActionPerformed (DefaultButtonModel.java:373) at javax.swing.DefaultButtonModel.setPressed (DefaultButtonModel.java:245) at javax.swing.plaf.basic.BasicButtonListener.mouseReleased (BasicButtonListener.java:211) at java.awt.Component.processMouseEvent(Component.java:3710) at java.awt.Component.processEvent(Component.java:3539) at java.awt.Container.processEvent(Container.java:1159) at java.awt.Component.dispatchEventImpl(Component.java:2588) at java.awt.Container.dispatchEventImpl(Container.java:1208) at java.awt.Component.dispatchEvent(Component.java:2492) at java.awt.LightweightDispatcher.retargetMouseEvent (Container.java:2451) at java.awt.LightweightDispatcher.processMouseEvent(Container.java:2216) at java.awt.LightweightDispatcher.dispatchEvent(Container.java:2125) at java.awt.Container.dispatchEventImpl(Container.java:1195) at java.awt.Window.dispatchEventImpl(Window.java:923) at java.awt.Component.dispatchEvent(Component.java:2492) at java.awt.EventQueue.dispatchEvent(EventQueue.java:334) at java.awt.EventDispatchThread.pumpOneEventForHierarchy (EventDispatchThread.java:126) at java.awt.EventDispatchThread.pumpEventsForHierarchy (EventDispatchThread.java:93) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:88) at java.awt.EventDispatchThread.run(EventDispatchThread.java:80)

Severity: normal OS: All

[asfimport]

Sebb (migrated from Bugzilla): Looks like it thinks the results file is CSV, whereas it is actually XML.

This would account for it trying to parse <?xml version="1.0" encoding ="UTF-8"?> as a Long.

If the output format disagrees with the file format, then JMeter won't read the file properly.

Note that if one creates a results file using xml format, and then changes the output format to csv, JMeter will append the new results to the existing file, thus creating a file which is a mixture of XML and CSV formats, which will not load successfully using either format.

It would be worth checking the result format setting in jmeter.properties, and that the whole results file is one format or the other ...

Without seeing the files, my analysis could be wrong, so if the two formats do agree, please could you attach the failing results file and jmeter.properties?

[asfimport]

Mike Hampton (migrated from Bugzilla): Created attachment nonguiTest2.zip: Example results with error.

[asfimport]

Mike Hampton (migrated from Bugzilla): I tried to open the attached file with an aggregate listener and got the same reaction.

[asfimport]

Mike Hampton (migrated from Bugzilla): Created attachment jmeter.properties: Jmeter config

jmeter.properties

````properties ################################################################################ # Apache JMeter Property file ################################################################################ ## $Id: jmeter.properties,v 1.100.2.2 2004/05/20 23:07:49 sebb Exp $ ## Copyright 2001-2004 The Apache Software Foundation ## ## Licensed under the Apache License, Version 2.0 (the "License"); ## you may not use this file except in compliance with the License. ## You may obtain a copy of the License at ## ## http://www.apache.org/licenses/LICENSE-2.0 ## ## Unless required by applicable law or agreed to in writing, software ## distributed under the License is distributed on an "AS IS" BASIS, ## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ## See the License for the specific language governing permissions and ## limitations under the License. #Preferred GUI language. Comment out to use the JVM default locale's language. #language=de # Netscape HTTP Cookie file cookies=cookies # Authorization authorization=authorization #Working directory user.dir=. # XML Reader(Parser) - Must implement SAX 2 specs xml.parser=org.apache.xerces.parsers.SAXParser #Classname of the ssl provider to be used (to enable testing of https urls) #And the package name where Stream Handlers can be found #These provided defaults can be uncommented, and they will work if you are using #Sun's JSSE implementation. ssl.provider=com.sun.net.ssl.internal.ssl.Provider #ssl.provider=iaik.security.jsse.provider.IAIKJSSEProvider ssl.pkgs=com.sun.net.ssl.internal.www.protocol #The location of the truststore (trusted certificates) and keystore ( if other than the default. #you can uncomment this and change the path to the correct location. #javax.net.ssl.trustStore=/path/to/cacerts #javax.net.ssl.keyStore=/path/to/keystore #The password to your keystore #javax.net.ssl.keyStorePassword=password #Alternative protocol of the ssl provider for IAIK JCE + iSaSiLk #You can also override IAIK specific Providers as well. #ssl.pkgs=iaik.protocol #ssl.provider=com.mycompany.security.provider.SmartProvider #iaik.provider=com.mycompany.security.provider.SmartSSLProvider #Flag for whether to output debug messages to System.err #To enable it, set the value to "all" Note, for it to work with #JSSE, it needs to be done from the Java command (i.e. -Djavax.net.debug=all) javax.net.debug=all #Classname of the Swing default UI #Installed Look and Feel classes on Windows are: # �Metal � = javax.swing.plaf.metal.MetalLookAndFeel # �Motif � = com.sun.java.swing.plaf.motif.MotifLookAndFeel # �Windows = com.sun.java.swing.plaf.windows.WindowsLookAndFeel jmeter.laf=javax.swing.plaf.metal.MetalLookAndFeel #icons -> moved to program code #timer.tree.icon=timer.gif #listener.tree.icon=ear.gif #bench.tree.icon=clipboard.gif #thread.tree.icon=thread.gif #control.tree.icon=knob.gif #plan.tree.icon=beaker.gif #config.tree.icon=leafnode.gif # Remote Hosts - comma delimited remote_hosts=127.0.0.1,30.72.13.106 #Components to not display in JMeter GUI not_in_menu=Remote Method Configuration,JNDI Configuration,JNDI Lookup Configuration,JNDI Request,Default Controller,org.apache.jmeter.control.DynamicController, org.apache.jmeter.protocol.http.control.Cookie,org.apache.jmeter.protocol.http.control.Authorization,org.apache.jmeter.config.LoginConfig,Header,org.apache.jmeter.protocol.http.config.MultipartUrlConfig #--------------------------------------------------------------------------- # Logging Configuration #--------------------------------------------------------------------------- # Note: JMeter uses Avalon LogKit # Logging Format # see http://avalon.apache.org/logkit/api/org/apache/log/format/PatternFormatter.html # # Default format: #log_format=%{time:yyyy/MM/dd HH:mm:ss} %5.5{priority} - %{category}: %{message} %{throwable} # \n is automatically added to the end of the string # # Predefined formats in the JMeter LoggingManager: #log_format_type=default #log_format_type=thread_prefix #log_format_type=thread_suffix # default is as above # thread_prefix adds the thread name as a prefix to the category # thread_suffix adds the thread name as a suffix to the category # Note that thread name is not included by default, as it requires extra processing. # # To change the logging format, define either log_format_type or log_format # If both are defined, the type takes precedence # Note that these properties cannot be defined using the -J or -D JMeter # command-line flags, as the format will have already been determined by then # However, they can be defined as JVM properties #Logging levels for the logging categories in JMeter. Correct values are FATAL_ERROR, ERROR, WARN, INFO, and DEBUG # To set the log level for a package or individual class, use: # log_level.[package_name].[classname]=[PRIORITY_LEVEL] # But omit "org.apache" from the package name. The classname is optional. Further examples below. #log_level.jmeter=INFO log_level.jmeter=DEBUG log_level.jmeter.junit=DEBUG #log_level.jmeter.engine=WARN #log_level.jmeter.gui=WARN #log_level.jmeter.testelement=DEBUG #log_level.jmeter.util=WARN #log_level.jmeter.util.classfinder=WARN #log_level.jmeter.test=DEBUG #log_level.jmeter.protocol.http=DEBUG #log_level.jmeter.protocol.ftp=WARN #log_level.jmeter.protocol.jdbc=WARN #log_level.jmeter.protocol.java=WARN #log_level.jmeter.testelements.property=DEBUG log_level.jorphan=INFO #Log file for log messages. # You can specify a different log file for different categories via: # log_file.[category]=[filename] # category is equivalent to the package/class names described above # Combined log file (for jmeter and jorphan) log_file=jmeter.log # To redirect logging to standard output, try the following: # (it will probably report an error, but output will be to stdout) #log_file= # Or define separate logs if required: #log_file.jorphan=jorphan.log #log_file.jmeter=jmeter.log # Sample logging levels for HttpClient # Note that full category names are used, i.e. must include the org.apache. # Info level produces no output: #log_level.org.apache.commons.logging=debug # Might be useful: #org.apache.commons.httpclient.Authenticator=trace # wire debug produces a lot of output; consider using separate file: #log_level.httpclient.wire=debug #log_file.httpclient=httpclient.log # Further logging configuration # Excalibur logging provides the facility to configure logging using # configuration files written in XML. This allows for such features as # log file rotation which are not supported directly by JMeter. # # If such a file specified, it will be applied to the current logging # hierarchy when that has been created. # #log_config=logkit.xml #--------------------------------------------------------------------------- # Results file configuration #--------------------------------------------------------------------------- # This section helps determine how result data will be saved. # The commented out values are the defaults. # legitimate values: xml, csv, db. Only xml and csv are currently supported. jmeter.save.saveservice.output_format=xml # true when field should be saved; false otherwise # assertion_results_failure_message only affects CSV output #jmeter.save.saveservice.assertion_results_failure_message=true #jmeter.save.saveservice.data_type=true #jmeter.save.saveservice.label=true #jmeter.save.saveservice.response_code=true #jmeter.save.saveservice.response_data=false #jmeter.save.saveservice.response_message=true #jmeter.save.saveservice.successful=true #jmeter.save.saveservice.thread_name=true #jmeter.save.saveservice.time=true # legitimate values: none, ms, or a format suitable for SimpleDateFormat #jmeter.save.saveservice.timestamp_format=ms #jmeter.save.saveservice.timestamp_format=MM/dd/yy HH:mm:ss # legitimate values: none, first, all #jmeter.save.saveservice.assertion_results=none # For use with Comma-separated value (CSV) files or other formats # where the fields' values are separated by specified delimiters. #jmeter.save.saveservice.default_delimiter=, #jmeter.save.saveservice.print_field_names=true # File that holds a record of name changes for backward compatibility issues upgrade_properties=/bin/upgrade.properties # If the proxy detects a gap of at least 1s (default) between HTTP requests, # it assumes that the user has clicked a new URL #proxy.pause=1000 # Add numeric prefix to Sampler names (default false) #proxy.number.requests=true # # Define the HTML parser to be used. # Default parser: #htmlParser.className=org.apache.jmeter.protocol.http.parser.HtmlParserHTMLParser # Other parsers: #htmlParser.className=org.apache.jmeter.protocol.http.parser.JTidyHTMLParser #htmlParser.className=org.apache.jmeter.protocol.http.parser.RegexpHTMLParser # Put the start time stamp in logs instead of the end #sampleresult.timestamp.start=true # Turn expert mode on/off: expert mode will show expert-mode beans and properties #jmeter.expertMode=true # To set the buffer size, enter the desired value # monitor.buffer.size=800````

[asfimport]

Mike Hampton (migrated from Bugzilla): There seems to be some problem loading line 3235 (which has some strange stuff in it). As soon as I delete that line the file loads into the listener.

[asfimport]

Sebb (migrated from Bugzilla): Well spotted!

I've had a look at the Result loading code, and the problem becomes clear - if the XML does not parse correctly, the code silently assumes that the file must be in CSV format, and passes it to the CSV parser.

This expects to find a numeric timestamp at the start of the line. Instead it gets the XML header.

In this case, the line contains some invalid XML, viz  and
and possibly others.

I will check in a fix to log some messages to show what is happening.

But how did the invalid XML find its way into the file? It is possibly significant that the responseCode = "-1", which is not a normal HTTP response code. What does the "Load Eligibility Request" sampler do?

[asfimport]

Mike Hampton (migrated from Bugzilla): It actually does very little. I have noticed that there are several '-1' response codes in the file. The samplers that produce these invalid responses are often very straightforward 'get' requests. In some cases thay are just requests for images.

The 'responseMessage' attribute consistantly has strange garbage in it when this happens. I am wondering if this is caused by the server that the sampler is making requests against when placed under load, not sure.

Regardless, I am uncomfortable that the sampler is able to write invalid xml into the file.

[asfimport]

Sebb (migrated from Bugzilla): I agree that the sampler should not write invalid XML into the file.

I've had a look at the code, and can't see anything obvious. This part of the code uses Avalon Config to create the XML. The response message itself is obtained directly from the HttpURLConnection object.

It might be interesting to run a test with CSV output instead of XML, as this appears to save the data as-is. Having obtained the raw data, it should be easier to find out why it is not being encoded correctly.

[BTW, the JavaTest sampler is quite useful for testing what happens to response codes and messages etc as you can supply your own text]

Are you able to supply a test case?

If not, perhaps there is some scope for a debug version of JMeter that detects when the response code is -1 and dumps the raw response message - if the CSV method does not produce useful results.

[asfimport]

Sebb (migrated from Bugzilla): I've created a simple test script that uses a BeanShell Assertion to check the response code and conditionally write the ResponseMessage to a file.

[To use this, you need to add the BeanShell jar from www.beanshell.org to the lib directory.]

It's not ideal, but it might be enough to track down the problem.

BTW, I first tried using the BeanShell Assertion to set a JMeter Variable. This worked, but unfortunately the If Controller seems to resolve the condition just the once, so it did not detect changes in the value.

[asfimport]

Mike Stover (migrated from Bugzilla): In HEAD, .jtl files are written using XStream instead of Avalon's Configuration. Hopefully this will fix this problem.

[asfimport]

Sebb (migrated from Bugzilla): No point fixing this now we are using Xstream. Please re-open if it occurs with Xstream

[asfimport]

Mark Sechrest (migrated from Bugzilla): Created attachment results_a.jtl: Small result file

results_a.jtl

````xml ````

[asfimport]

Mark Sechrest (migrated from Bugzilla): I'm seeing similar behaviour trying to reload a results file created with the SimpleDataListener in 2.1.1.

Jmeter.log: 2006/05/11 09:04:05 INFO - jmeter.visualizers.gui.AbstractVisualizer: getting new collector 2006/05/11 09:04:09 INFO - jmeter.protocol.http.sampler.HTTPSampler2: httpsampler2.basicauth=false 2006/05/11 09:04:09 WARN - jmeter.save.SaveService: Could not set up alias BeanShellSampler java.lang.ClassNotFoundException: org.apache.jmeter.protocol.java.sampler.BeanShellSampler 2006/05/11 09:04:09 INFO - jmeter.save.SaveService: Using SaveService properties file 1.7 2006/05/11 09:04:09 INFO - jmeter.save.SaveService: All converter versions present and correct 2006/05/11 09:04:09 WARN - jmeter.reporters.ResultCollector: File load failure, trying old data format. 2006/05/11 09:04:09 WARN - jmeter.reporters.ResultCollector: Error parsing XML results com.thoughtworks.xstream.converters.ConversionException: sampleResult : sampleResult ---- Debugging information ---- required-type : org.apache.jmeter.save.TestResultWrapper cause-message : sampleResult : sampleResult class : org.apache.jmeter.save.TestResultWrapper message : sampleResult : sampleResult line number : 4 path : /testResults/sampleResult cause-exception : com.thoughtworks.xstream.alias.CannotResolveClassException

2006/05/11 09:04:09 INFO - jmeter.reporters.ResultCollector: Assuming CSV format instead 2006/05/11 09:04:09 WARN - jmeter.save.OldSaveService: Error parsing number java.lang.NumberFormatException: For input string: "<?xml version="1.0" encoding="UTF-8"?>"

Looking at the results file, it seems to be incorrectly formatted after about 20 lines. I tried just getting the lines before the formatting changed to a separate file, and loading that, but it give the same error. I have attached the shorter file.

I can open some saved files. It appears to be longer runs (larger data files) that have this problem.

[asfimport]

Sebb (migrated from Bugzilla): The "Small result file" JTL is incomplete - there is no trailing </testResults> tag.

However, when I added it, it still would not load, even though the XML was then syntactically OK. This is a different problem from the original - the XML in that case was incorrect.

I'll take a look at this later.

[asfimport]

Mark Sechrest (migrated from Bugzilla): FWIW, I checked several other of my results files, and many of them did not have the closing </testResults> tag.

[asfimport]

Sebb (migrated from Bugzilla): I found a bug in the code - if the Avalon XML reader does not like the input (as in this case), the log message uses the old XStream exception, which is rather confusing.

When I fixed this, it showed that the last sampleResult is also missing its closing tag. It looks as though some nested samples may have been missing.

Did the test fail to complete in some way? That could explain the missing output.

The original Avalon JTL files have the following second line: <testResults> and the samples look like <sampleResult timeStamp="1146770278785" ...

whereas later XStream JTL files have a version number: <testResults version="1.1"> and the samples look like <sample t="300" ... or <httpSample t="1002" ...

"Small result file" has a version number, but is otherwise an Avalon file. This means that the property file_format.testlog has been set to 2.0. Unless you particularly need this format, I suggest you revert to the default.

[asfimport]

Sebb (migrated from Bugzilla): I think this is now fixed in the 2.1 branch

Please re-open if not.

[asfimport]

Mark Sechrest (migrated from Bugzilla): (In reply to comment 16)

Did the test fail to complete in some way? No, the test ran fine.

This means that the property file_format.testlog has been set to 2.0. Unless you particularly need this format, I suggest you revert to the default.

file_format.testlog IS set to 2.0, but I didn't set it. It would seem that is the default for the version I installed. (jmeter.properties is v 1.124.2.2)