GMOD / Apollo

Genome annotation editor with a Java Server backend and a Javascript client that runs in a web browser as a JBrowse plugin.
http://genomearchitect.readthedocs.io/
Other
124 stars 85 forks source link

Tomcat: Too many open files error #162

Closed cmdcolin closed 9 years ago

cmdcolin commented 9 years ago

If a bunch of tracks are opened at once, then many duplicate file handles are opened for config.properties, basically until it hits the system limit.

Normal operation: around 250 file handles open

After opening up about 10 bam tracks and scrolling around: over 800 file handles open, all duplicates of config.properties

It will just increase until the "too many open files error" or the garbage collector picks up the objects

nathandunn commented 9 years ago

@cmdcolin It looks like it opens 1-3 files per track loaded and doesn't seem to increase (most of the time) and will eventually release the file handles.

Loading the bam track (Forager RNA-Seq reads), I see that it loads around 14 . . . goes up to 20 or so and then plateaus.

With 3 BAM files I got it to around 80-100 . . and it kind of maxed out (moving back and forth and changing groups).

That being said, in the code there is this line . . can't but help to fix:

    // TODO: move up so not recalculating each time
    String pathTranslated = request.getPathTranslated();
    String rootPath = pathTranslated.substring(0, pathTranslated.length() - request.getPathInfo().length());
    String configPath = rootPath + "/config/config.properties";

    File propertyFile = new File(configPath);
    String filename = null;
cmdcolin commented 9 years ago

should probably call propertyFile.close()

I wasn't sure whether we should put it in the try{} finally {} block or just manually close it

cmdcolin commented 9 years ago

Actually we should probably just load the file into memory so that it doesn't have to do that on every get request right?

nathandunn commented 9 years ago

I think it is working now . . . It probably wouldn't hurt to have it in the init, but performance wise it doesn't make any difference and I am using the "request" explicitly.

nathandunn commented 9 years ago

That is what I ended up doing. I had a note to do that eventually.

Nathan

On Feb 6, 2015, at 5:18 PM, Colin Diesh notifications@github.com wrote:

Actually we should probably just load the file into memory so that it doesn't have to do that on every get request right?

— Reply to this email directly or view it on GitHub https://github.com/GMOD/Apollo/issues/162#issuecomment-73342220.

nathandunn commented 9 years ago

@cmdcolin If you think this fixed it I would say go ahead and close it.

cmdcolin commented 9 years ago

Thanks it seems to work so i'll close it for now

It probably wouldn't hurt to have it in the init

I would prefer this, it looks a little messy currently, so...whenever we get a chance in our copious amounts of free time...refactor it

cmdcolin commented 9 years ago

Reopen:

I thought this was a fluke but the first time you open webapollo after a deploy, it returns

SyntaxError: expected expression, got ')' when parsing data/trackList.json.

This is reproducable on two different servers but I haven't narrowed down the cause yet.

nathandunn commented 9 years ago

Hmm . . do you have the trackList.json? It could be something introduced by the JBrowse Servlet, but I suspect its a faulty tracklist.json. I just tested this and it worked great.

cmdcolin commented 9 years ago

I'm fairly sure it is independant of trackList.json (i tested two different servers, local dev and production) and both have it

It only happens the first time you access it, after that, presumably, the config.properties is initialized (since it is initialized within the GET request handler)

For completeness though here is a basic trackList.json

==removed==

testing

Again, the error goes away after you refresh again

nathandunn commented 9 years ago

I but it is having all of the asynchronous requests coming through. I guess we’ll have to move it to the init servlet method. Give me a few to fix this.

Nathan

On Feb 9, 2015, at 12:02 PM, Colin Diesh notifications@github.com wrote:

I'm fairly sure it is independant of trackList.json (i tested two different servers, local dev and production) and both have it

It only happens the first time you access it, after that, presumably, the config.properties is initialized (since it is initialized within the GET request handler)

For completeness though here is a basic trackList.json

{ "share_link" : 0, "plugins" : [ { "location" : "./plugins/WebApollo", "name" : "WebApollo" } ], "tracks" : [ { "chunkSize" : 20000, "storeClass" : "JBrowse/Store/Sequence/StaticChunked", "urlTemplate" : "seq/{refseq_dirpath}/{refseq}-", "category" : "Reference sequence", "type" : "SequenceTrack", "label" : "DNA", "key" : "Reference sequence" }, { "autocomplete" : "none", "style" : { "className" : "annot", "renderClassName" : "annot-render", "subfeatureClasses" : { "non_canonical_three_prime_splice_site" : "noncanonical-splice-site", "wholeCDS" : null, "exon" : "container-100pct", "CDS" : "annot-CDS", "non_canonical_five_prime_splice_site" : "noncanonical-splice-site", "UTR" : "annot-UTR" }, "arrowheadClass" : "annot-arrowhead", "alternateClasses" : { "transposable_element" : { "renderClassName" : "blue-ibeam-render", "className" : "blue-ibeam" }, "pseudogene" : { "renderClassName" : "gray-center-30pct", "className" : "light-purple-80pct" }, "snRNA" : { "renderClassName" : "gray-center-30pct", "className" : "brightgreen-80pct" }, "rRNA" : { "renderClassName" : "gray-center-30pct", "className" : "brightgreen-80pct" }, "snoRNA" : { "renderClassName" : "gray-center-30pct", "className" : "brightgreen-80pct" }, "repeat_region" : { "className" : "magenta-80pct" }, "tRNA" : { "renderClassName" : "gray-center-30pct", "className" : "brightgreen-80pct" }, "ncRNA" : { "renderClassName" : "gray-center-30pct", "className" : "brightgreen-80pct" }, "miRNA" : { "renderClassName" : "gray-center-30pct", "className" : "brightgreen-80pct" } }, "uniqueIdField" : "id", "centerSubFeature" : { "non_canonical_three_prime_splice_site" : false, "non_canonical_five_prime_splice_site" : false } }, "key" : "User-created Annotations", "storeClass" : "WebApollo/Store/SeqFeature/ScratchPad", "disableJBrowseMode" : false, "phase" : 0, "compress" : 0, "label" : "Annotations", "type" : "WebApollo/View/Track/AnnotTrack", "subfeatures" : 1 } ], "names" : { "url" : "names/", "type" : "Hash" }, "favicon" : "./plugins/WebApollo/img/webapollo_favicon.ico", "alwaysOnTracks" : "DNA,Annotations", "formatVersion" : 1 } https://cloud.githubusercontent.com/assets/6511937/6114505/3fbfeb1a-b064-11e4-9cc2-970de6742e1f.jpg Again, the error goes away after you refresh again

— Reply to this email directly or view it on GitHub https://github.com/GMOD/Apollo/issues/162#issuecomment-73579260.

nathandunn commented 9 years ago

There were a few problems . . . and it was serendipity that it was working in the first place (properties was checking for a global, but assigning a local).

Both of these are now fixed (doing in the init servlet).

Let me know if it doesn’t fix it for you. Still not sure why it ended up working for me.

Nathan

On Feb 9, 2015, at 12:02 PM, Colin Diesh notifications@github.com wrote:

I'm fairly sure it is independant of trackList.json (i tested two different servers, local dev and production) and both have it

It only happens the first time you access it, after that, presumably, the config.properties is initialized (since it is initialized within the GET request handler)

For completeness though here is a basic trackList.json

{ "share_link" : 0, "plugins" : [ { "location" : "./plugins/WebApollo", "name" : "WebApollo" } ], "tracks" : [ { "chunkSize" : 20000, "storeClass" : "JBrowse/Store/Sequence/StaticChunked", "urlTemplate" : "seq/{refseq_dirpath}/{refseq}-", "category" : "Reference sequence", "type" : "SequenceTrack", "label" : "DNA", "key" : "Reference sequence" }, { "autocomplete" : "none", "style" : { "className" : "annot", "renderClassName" : "annot-render", "subfeatureClasses" : { "non_canonical_three_prime_splice_site" : "noncanonical-splice-site", "wholeCDS" : null, "exon" : "container-100pct", "CDS" : "annot-CDS", "non_canonical_five_prime_splice_site" : "noncanonical-splice-site", "UTR" : "annot-UTR" }, "arrowheadClass" : "annot-arrowhead", "alternateClasses" : { "transposable_element" : { "renderClassName" : "blue-ibeam-render", "className" : "blue-ibeam" }, "pseudogene" : { "renderClassName" : "gray-center-30pct", "className" : "light-purple-80pct" }, "snRNA" : { "renderClassName" : "gray-center-30pct", "className" : "brightgreen-80pct" }, "rRNA" : { "renderClassName" : "gray-center-30pct", "className" : "brightgreen-80pct" }, "snoRNA" : { "renderClassName" : "gray-center-30pct", "className" : "brightgreen-80pct" }, "repeat_region" : { "className" : "magenta-80pct" }, "tRNA" : { "renderClassName" : "gray-center-30pct", "className" : "brightgreen-80pct" }, "ncRNA" : { "renderClassName" : "gray-center-30pct", "className" : "brightgreen-80pct" }, "miRNA" : { "renderClassName" : "gray-center-30pct", "className" : "brightgreen-80pct" } }, "uniqueIdField" : "id", "centerSubFeature" : { "non_canonical_three_prime_splice_site" : false, "non_canonical_five_prime_splice_site" : false } }, "key" : "User-created Annotations", "storeClass" : "WebApollo/Store/SeqFeature/ScratchPad", "disableJBrowseMode" : false, "phase" : 0, "compress" : 0, "label" : "Annotations", "type" : "WebApollo/View/Track/AnnotTrack", "subfeatures" : 1 } ], "names" : { "url" : "names/", "type" : "Hash" }, "favicon" : "./plugins/WebApollo/img/webapollo_favicon.ico", "alwaysOnTracks" : "DNA,Annotations", "formatVersion" : 1 } https://cloud.githubusercontent.com/assets/6511937/6114505/3fbfeb1a-b064-11e4-9cc2-970de6742e1f.jpg Again, the error goes away after you refresh again

— Reply to this email directly or view it on GitHub https://github.com/GMOD/Apollo/issues/162#issuecomment-73579260.

cmdcolin commented 9 years ago

Looks good now