search

marschall / memoryfilesystem

An in memory implementation of a JSR-203 file system
284 stars 36 forks source link

Files bigger than 16MBs not yet supported #71

Open freezing opened 8 years ago

freezing commented 8 years ago

Although the exception says: java.lang.AssertionError: files bigger than 16GB not yet supported; According to my tests, files bigger than 16MB are not supported.

According to the following lines: https://github.com/marschall/memoryfilesystem/blob/master/src/main/java/com/github/marschall/memoryfilesystem/MemoryInode.java#L32 https://github.com/marschall/memoryfilesystem/blob/master/src/main/java/com/github/marschall/memoryfilesystem/MemoryInode.java#L356

IndirectBlocks is a matrix of size BLOCK_SIZE * BLOCK_SIZE which is 4k * 4k = 16MB.

Also, I think there should be two constants, NUMBER_OF_BLOCKS * BLOCK_SIZE. It's more readable.

Can it be changed to support a file of size at least a couple of Gigs? The exception message should be changed as well: https://github.com/marschall/memoryfilesystem/blob/master/src/main/java/com/github/marschall/memoryfilesystem/MemoryInode.java#L360

marschall commented 8 years ago

Regarding the additional constant and exception message, these are good points and I fixed them in d6f915ad00d3001d114bd69ba632ca771a93fca7.

Now for the maximum file size how big do you really need? Remember this will all be heap memory. The quick fix is to simply up the limits but this will cause really large objects in the heap. The proper solution would be to implement double indirect addressing.

freezing commented 8 years ago

Thanks.

I think 1GB should be more than enough at the moment. I understand that it will be heap memory, but for now the quick fix will work just fine.

For 1GB it should take 300k of blocks to be initialized. Each of them will be empty unless something is written to the file. It should be fine since not all of the files will be that large.

develar commented 4 years ago

Any news?

marschall commented 4 years ago

@develar no new so far, is this a deal breaker for you?

develar commented 4 years ago

Library is widely used in tests, but some tests create large files. So, will be great if limit will be raised (at least, 512 MB).

On Sun, 26 Jul 2020, 14:56 Philippe Marschall, notifications@github.com wrote:

@develar https://github.com/develar no new so far, is this a deal breaker for you?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/marschall/memoryfilesystem/issues/71#issuecomment-663984659, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACVTXUGB76IZMQFDSTGMDLR5QRZFANCNFSM4B2FIRCQ .