search

borisbrodski / sevenzipjbinding

7-Zip-JBinding
http://sevenzipjbind.sourceforge.net/
Other
176 stars 50 forks source link

OutOfMemoryError while constructing ByteArrayStream #21

Open bhuvim1 opened 5 years ago

bhuvim1 commented 5 years ago

Creating ByteArrayStream(net.sf.sevenzipjbinding.util.ByteArrayStream) from InputStream(java.io.InputStream) is throwing OutOfMemoryError: Java heap space Exception for larger files. As I checked the code, byte array with file size is getting created. Is there any other way to convert InputStream to IInStream?

borisbrodski commented 5 years ago

Thank you for the report. But it is the case of "It's not a bug, it's a feture" :) Sorry.

You see, the ByteArrayStream was explicitly designed for the small files, that should completely fit into the memory.

Converting InputStream into IInStream is unfortunately not as simple as one my wish. The core problem here is, that 7-Zip engine relies heavily on the SEEK-operation. On the other hand, the main idea of the InputStream is, that you read the data consequently, byte after byte. Implementing SEEK forward operation on an sequential stream is not particularly performant, whether implementing SEEK back is impossible.

Consider, that the table of content is often located at the end on an archive. So you seek to the last bytes of the archive, read the table of content, and then seek back to the beginning to start the extraction.

So you are better off to use new RandomAccessFileInStream(randomAccessFile) from the example snippets: http://sevenzipjbind.sourceforge.net/first_steps.html

bhuvim1 commented 5 years ago

So you are better off to use new RandomAccessFileInStream(randomAccessFile) from the example snippets: http://sevenzipjbind.sourceforge.net/first_steps.html

Here, I don't have the file in my hand. I've only InputStream. I don't think it's possible to use RandomAccessFileInStream with InputStream. If at all it's possible, redirect me to the examples.

borisbrodski commented 5 years ago

You are right, you can use RandomAccessFileInStream only with a files. But may be you can implement IInStream interface yourself? What is the source of your data? If it is for example HTTP url, you can implement seek operation requesting from the server different parts of the remote file.