Use Apache commons io utils for more robust array copying

lintool / warcbase

Warcbase is an open-source platform for managing analyzing web archives

http://warcbase.org/

161 stars 47 forks source link

Use Apache commons io utils for more robust array copying #261

Closed zackwang closed 7 years ago

zackwang commented 7 years ago

OpenJDK's implementation of io utilities sucks. It will raise a java.lang.OutOfMemoryError: Requested array size exceeds VM limit exception when copying array larger than 2G. This will affect large warc files. org.apache.commons.io provides more robust io utilities.