VIVO-1669: RDFServiceGraph flush() : out of memory

[chenejac]

alessandro galasso (Migrated from VIVO-1669) said:

The append method of StringBuilder has a character count max limit of "Integer.MAX_VALUE",

so the serialization in serializeGraph(Graph graph) is very likely to go in out of memory or out of bound.

Tests based on real production data, with an upload of 2gb size rdf file show the need to adress this issue,

something like this shoud do the job:

 

 
if(!additionsGraph.isEmpty()) {
 String triples = "";
 StringBuilder sb = new StringBuilder(51000000);
 Iterator<Triple> tripIt = graph.find(null, null, null);
 while(tripIt.hasNext()) {
 sb.append(" \n");
 sb.append(serialize(tripIt.next()));
 if(sb.length()>50000000) {
 triples = sb.toString();
 sb = null;
 sb = new StringBuilder();
 changeSet.addAddition(RDFServiceUtils.toInputStream(triples), 
 RDFService.ModelSerializationFormat.N3, graphURI);
 triples = null;
 rdfService.changeSetUpdate(changeSet);
 changeSet = rdfService.manufactureChangeSet();
 }
 }
 triples = sb.toString();
 sb = null;
 changeSet.addAddition(RDFServiceUtils.toInputStream(triples), 
 RDFService.ModelSerializationFormat.N3, graphURI);
 triples = null;
 additionsGraph.clear();
 rdfService.changeSetUpdate(changeSet);
 }

[chenejac]

Andrew Woods said:

Which class(es) are you suggesting being updated?

[chenejac]

alessandro galasso said:

[~accountid:60785ded115da6006f540529]  this class should be fixed because sb go easlily  out of bound

//class:

  Vitro/api/src/main/java/edu/cornell/mannlib/vitro/webapp/dao/jena/RDFServiceGraph.java

// method

    private synchronized String serializeGraph(Graph graph) {
        StringBuilder sb = new StringBuilder();
        Iterator<Triple> tripIt = graph.find(null, null, null);
        while(tripIt.hasNext()) {
            serialize(sb.append(" \n"), tripIt.next());
        }
        return sb.toString();
    }