Possible memory leak in Java toolbox.

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Add the following to startup.m
addpath('C:\work\matlab\tools\nctoolbox\','-end')
setup_nctoolbox

2. Attempt to call ncgeodataset on a large thredds dataset

>> 
url7='http://colossus.dl.stevens-tech.edu:8080/thredds/dodsC/fmrc/NYBight/NYHOPS
_Forecast_Collection_for_the_New_York_Bight_best.ncd'

url7 =

http://colossus.dl.stevens-tech.edu:8080/thredds/dodsC/fmrc/NYBight/NYHOPS_Forec
ast_Collection_for_the_New_York_Bight_best.ncd

>> nc=ncgeodataset(url7);

What is the expected output? What do you see instead?

The result:

>> nc=ncgeodataset(url7);
??? Error using ==> ncdataset>ncdataset.ncdataset at 91
Failed to open 
http://colossus.dl.stevens-tech.edu:8080/thredds/dodsC/fmrc/NYBight/NYHOPS_Forec
ast_Collection_for_the_New_York_Bight_best.ncd

Error in ==> cfdataset>cfdataset.cfdataset at 55
            obj = obj@ncdataset(url);

Error in ==> ncgeodataset>ncgeodataset.ncgeodataset at 20
            obj = obj@cfdataset(url);

Caused by:
    Java exception occurred:
    java.lang.OutOfMemoryError: GC overhead limit exceeded

        at java.util.Arrays.copyOf(Unknown Source)

        at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source)

        at java.lang.AbstractStringBuilder.append(Unknown Source)

        at java.lang.StringBuilder.append(Unknown Source)

        at opendap.util.EscapeStrings.escapeString(EscapeStrings.java:184)

        at opendap.util.EscapeStrings.id2www(EscapeStrings.java:213)

        at opendap.dap.DAPNode.setClearName(DAPNode.java:233)

        at opendap.dap.BaseType.setClearName(BaseType.java:118)

        at opendap.dap.DAPNode.setName(DAPNode.java:223)

        at opendap.dap.DDS.cloneDAG(DDS.java:2191)

        at opendap.dap.DAPNode.cloneDAG(DAPNode.java:301)

        at opendap.dap.DAPNode.cloneDAG(DAPNode.java:318)

        at opendap.dap.BaseType.cloneDAG(BaseType.java:618)

        at opendap.dap.DVector.cloneDAG(DVector.java:304)

        at opendap.dap.DArray.cloneDAG(DArray.java:503)

        at opendap.dap.DAPNode.cloneDAG(DAPNode.java:301)

        at opendap.dap.DAPNode.cloneDAG(DAPNode.java:318)

        at opendap.dap.BaseType.cloneDAG(BaseType.java:618)

        at opendap.dap.DAPNode.clone(DAPNode.java:267)

        at opendap.dap.BaseTypePrimitiveVector.deserialize(BaseTypePrimitiveVector.java:185)

        at opendap.dap.DVector.deserialize(DVector.java:263)

        at opendap.dap.DataDDS.readData(DataDDS.java:117)

        at opendap.dap.DConnect2$DataDDSCommand.process(DConnect2.java:799)

        at opendap.dap.DConnect2.openConnection(DConnect2.java:311)

        at opendap.dap.DConnect2.getData(DConnect2.java:783)

        at opendap.dap.DConnect2.getData(DConnect2.java:1063)

        at ucar.nc2.dods.DODSNetcdfFile.readDataDDSfromServer(DODSNetcdfFile.java:1172)

        at ucar.nc2.dods.DODSNetcdfFile.readArrays(DODSNetcdfFile.java:1226)

        at ucar.nc2.dods.DODSNetcdfFile.<init>(DODSNetcdfFile.java:327)

        at ucar.nc2.dataset.NetcdfDataset.acquireDODS(NetcdfDataset.java:786)

        at ucar.nc2.dataset.NetcdfDataset.openOrAcquireFile(NetcdfDataset.java:677)

        at ucar.nc2.dataset.NetcdfDataset.openDataset(NetcdfDataset.java:424)

What version of the product are you using? On what operating system?

nctoolbox-20110616-alpha2.zip  
Windows 7 (same problems on Linux)
Matlab 2010b

Please provide any additional information below.

If I change the Startup.m file to: 
javaaddpath('C:\work\matlab\tools\toolsUI-4.0.49.jar','-end');
addpath('C:\work\matlab\tools\nctoolbox\','-end')
setup_nctoolbox

and comment out setup_nctoolbox_java in setup_nctoolbox I get:

>> nc=ncgeodataset(url7);
>> nc

nc = 

  ncgeodataset handle

  Properties:
     location: 'http://colossus.dl.stevens-tech.edu:8080/thredds/dodsC/fmrc/NYBight/NYHOPS_Forecast_Collection_for_the_New_York_Bight_best.ncd'
       netcdf: [1x1 ucar.nc2.dataset.NetcdfDataset]
    variables: {36x1 cell}

  Methods, Events, Superclasses

As expected. It looks as though the java toolbox included with nctoolbox has a 
memory leak somewhere. 

Original issue reported on code.google.com by imcsla...@gmail.com on 21 Jul 2011 at 11:57

[GoogleCodeExporter]

Thanks for the bug report. I'll investigate a bit more and let the NetCDF folks 
know if needed. To help me out,  Can you run the following command:

>> ver

and paste the results into a comment on this bug report so I can see the 
details of your OS, Java and Matlab versions? Also, I can't duplicate that 
error. So I'm wondering if the fix might be as simple as tweaking your memory 
settings in Matlab. There's a note on how to do this at 
http://blogs.mathworks.com/desktop/2010/04/26/controlling-the-java-heap-size/ . 
Could you give that a try and see if it resolves your issue? Thanks

Original comment by bschlin...@gmail.com on 21 Jul 2011 at 3:24

[GoogleCodeExporter]

Hi, 

Thanks for the suggestions.

Here is the output of ver:

>> ver
--------------------------------------------------------------------------------
-----
MATLAB Version 7.11.0.584 (R2010b)
MATLAB License Number: 354167
Operating System: Microsoft Windows 7 Version 6.1 (Build 7600)
Java VM Version: Java 1.6.0_17-b04 with Sun Microsystems Inc. Java HotSpot(TM) 
64-Bit Server VM mixed mode
--------------------------------------------------------------------------------
-----
MATLAB                                                Version 7.11       
(R2010b)
Simulink                                              Version 7.6        
(R2010b)
Bioinformatics Toolbox                                Version 3.6        
(R2010b)
Communications Blockset                               Version 5.0        
(R2010b)
Communications Toolbox                                Version 4.6        
(R2010b)
Control System Toolbox                                Version 9.0        
(R2010b)
Curve Fitting Toolbox                                 Version 3.0        
(R2010b)
Database Toolbox                                      Version 3.8        
(R2010b)
Datafeed Toolbox                                      Version 4.0        
(R2010b)
Econometrics Toolbox                                  Version 1.4        
(R2010b)
Filter Design Toolbox                                 Version 4.7.1      
(R2010b)
Financial Derivatives Toolbox                         Version 5.6        
(R2010b)
Financial Toolbox                                     Version 3.8        
(R2010b)
Fixed-Income Toolbox                                  Version 2.0        
(R2010b)
Fixed-Point Toolbox                                   Version 3.2        
(R2010b)
Fuzzy Logic Toolbox                                   Version 2.2.12     
(R2010b)
Global Optimization Toolbox                           Version 3.1        
(R2010b)
Image Acquisition Toolbox                             Version 4.0        
(R2010b)
Image Processing Toolbox                              Version 7.1        
(R2010b)
Instrument Control Toolbox                            Version 2.11       
(R2010b)
MATLAB Builder JA                                     Version 2.2        
(R2010b)
MATLAB Builder NE                                     Version 3.2        
(R2010b)
MATLAB Compiler                                       Version 4.14       
(R2010b)
MATLAB Report Generator                               Version 3.9        
(R2010b)
Mapping Toolbox                                       Version 3.2        
(R2010b)
Model Predictive Control Toolbox                      Version 3.2.1      
(R2010b)
Neural Network Toolbox                                Version 7.0        
(R2010b)
Optimization Toolbox                                  Version 5.1        
(R2010b)
Parallel Computing Toolbox                            Version 5.0        
(R2010b)
Parallel Computing Toolbox                            Version 5.0        
(R2010b)
Partial Differential Equation Toolbox                 Version 1.0.17     
(R2010b)
Real-Time Workshop                                    Version 7.6        
(R2010b)
Robust Control Toolbox                                Version 3.5        
(R2010b)
SEAWATER Library                                      Version 3.2               

SEAWATER Library                                      Version 3.2               

SOM Toolbox                                           Version 2.0beta,    May 
30 
Signal Processing Blockset                            Version 7.1        
(R2010b)
Signal Processing Toolbox                             Version 6.14       
(R2010b)
SimBiology                                            Version 3.3        
(R2010b)
SimMechanics                                          Version 3.2.1      
(R2010b)
SimPowerSystems                                       Version 5.3        
(R2010b)
Simscape                                              Version 3.4        
(R2010b)
Simulink 3D Animation                                 Version 5.2        
(R2010b)
Simulink Control Design                               Version 3.2        
(R2010b)
Simulink Design Optimization                          Version 1.2        
(R2010b)
Simulink Fixed Point                                  Version 6.4        
(R2010b)
Simulink Report Generator                             Version 3.9        
(R2010b)
Simulink Verification and Validation                  Version 3.0        
(R2010b)
Spreadsheet Link EX                                   Version 3.1.2      
(R2010b)
Stateflow                                             Version 7.6        
(R2010b)
Stateflow Coder                                       Version 7.6        
(R2010b)
Statistics Toolbox                                    Version 7.4        
(R2010b)
Symbolic Math Toolbox                                 Version 5.5        
(R2010b)
System Identification Toolbox                         Version 7.4.1      
(R2010b)
Video and Image Processing Blockset                   Version 3.1        
(R2010b)
Wavelet Toolbox                                       Version 4.6        
(R2010b)
>> 

I did try messing with the java memory settings, with no success. 

Thanks,
Eli

Original comment by imcsla...@gmail.com on 21 Jul 2011 at 4:28

[GoogleCodeExporter]

Wow! Is there a Matlab Toolbox that you don't have installed?

Can you do one more thing. Some of the toolboxes included Java jars. Would you 
run the following:

>> javaclasspath

and paste the output into a comment.
Thanks again.

Original comment by bschlin...@gmail.com on 21 Jul 2011 at 4:41

[GoogleCodeExporter]

Hi, 

I'm attaching a file with the javaclasspath output.

Thanks,
Eli

Original comment by imcsla...@gmail.com on 22 Jul 2011 at 2:11

Attachments:

• javaclasspath.txt