icsharpcode / AvalonEdit

The WPF-based text editor component used in SharpDevelop
http://avalonedit.net/
MIT License
1.85k stars 469 forks source link

Open files larger than available RAM? #382

Open sopgenorth opened 1 year ago

sopgenorth commented 1 year ago

Is it possible to use AvalonEdit with files that are larger than the available RAM? For example, a setup where only a buffered portion of a file is kept in memory, and updated as the user scrolls.

Are there any examples or pointers available on how to achieve this kind of setup?

damian-666 commented 1 year ago

Don' t know if this will help, but see https://www.codeguru.com/csharp/memory-mapped-files-c-sharp/ Do you mean using this sort of thing? Im curious how easy it would be to just integrate to the existing editors input , but with the disparate immutable partial clones and such for threading and parsing performance, i just saw discussed.. mabye not. also useful examples of usecases are in image processing for large resolution files say Archiectural plots which would require 24Gig at 600 dpi 1 bit. They take a tiling or an pipeline approach and im not sure its trival to just change the stream from a file to a memory map file. Other example is here https://docs.julialang.org/en/v1/stdlib/Mmap/

sopgenorth commented 1 year ago

Yeah, I think some sort of memory mapped file could work. Might also be able to use FileStream.ReadBytes with the available offset to do a similar thing.

My thoughts were that the strategy used would want to buffer some number of bytes above and below what's currently viewable.

The "interface" for Rope (https://github.com/icsharpcode/AvalonEdit/blob/master/ICSharpCode.AvalonEdit/Utils/Rope.cs) used by the TextDocument seems like it's not too far off from being able to accomodate something like this, as long as the users of it don't just always jump to getting the full text at once...

sopgenorth commented 1 year ago

There's a plugin for Notepad++ that adds a similar kind of large file support too: https://github.com/superolmo/BigFiles