Unidata / IDV

The Integrated Data Viewer (IDV) from Unidata is a framework for analyzing and displaying geoscience data.
http://www.unidata.ucar.edu/software/idv/
Other
79 stars 37 forks source link

Fix buffer bug, reads greater than 2 GB were failing [1483] #79

Closed tommyjasmin closed 10 years ago

tommyjasmin commented 10 years ago

Hi guys - very important this one gets in. Definite buffer overflow issue in IOUtil.writeTo() The counter was an int and of course that means our read/write limit is 2 GB. I had to track it down because MUG reported Suomi M-BAND bundles worked fine, but I-BAND bundles would fail to save and reload. The only obvious difference to me was size (I-BAND granules are MUCH bigger).

Watch the writeTo logging I added overflow Java signed int limits:

11:56:39.193 [AWT-EventQueue-0] TRACE e.w.ssec.mcidasv.PersistenceManager - writeTo: Transferred 2147300 Kbytes 11:56:39.194 [AWT-EventQueue-0] TRACE e.w.ssec.mcidasv.PersistenceManager - writeTo: num read: 100000 11:56:39.194 [AWT-EventQueue-0] TRACE e.w.ssec.mcidasv.PersistenceManager - writeTo: Transferred 2147400 Kbytes 11:56:39.195 [AWT-EventQueue-0] TRACE e.w.ssec.mcidasv.PersistenceManager - writeTo: num read: 100000 11:56:39.195 [AWT-EventQueue-0] TRACE e.w.ssec.mcidasv.PersistenceManager - writeTo: Transferred -2147467 Kbytes 11:56:39.196 [AWT-EventQueue-0] TRACE e.w.ssec.mcidasv.PersistenceManager - writeTo: num read: 100000 11:56:39.196 [AWT-EventQueue-0] TRACE e.w.ssec.mcidasv.PersistenceManager - writeTo: Transferred -2147367 Kbytes

The negative total returned by the method is considered a failure by callers.
Long story short (twisted pun, sorry), cumulative count needed to be a long. Luckily the ripple to other code was not bad.

public static long writeTo(InputStream from, OutputStream to,
                          Object loadId, long length)
        throws IOException {
    from = new BufferedInputStream(from, 100000);
    byte[] content = new byte[100000];
    long    total   = 0;
    while (true) {
        int howMany = from.read(content);
        if (howMany <= 0) {
            break;
        }
        total += howMany;
        if ( !JobManager.getManager().canContinue(loadId)) {
            return -1;
        }
        String msg;
        if (length > 0) {
            msg = "Transferred " + (total / 1000) + "/" + (length / 1000)
                  + " Kbytes ";
        } else {
            msg = "Transferred " + ((total) / 1000) + " Kbytes";
        }

        JobManager.getManager().setDialogLabel2(loadId, msg);

        to.write(content, 0, howMany);
    }
    return total;
}