save large file is extremely slow, network mounted PanFS drive

[umnyaga]

I have a txt file about 26.5 Mb, 124000 lines. I selected first 7550 lines, deleted them and saved the file. Took about minute or two. I even thought it froze, but not, later it displayed progress dialog and later it finally saved the file. I tried the same procedure with PS Pad and Notepad2 and they used less then a second to save.

[Alexey-T]

ops. indeed! on saving, even ASCII file, I do for each line

    SWide:= Item^.Line + cLineEndStrings[ItemEnd];     
...
      cEncUTF8:
        begin
          SAnsi:= UTF8Encode(SWide);
          AStream.WriteBuffer(SAnsi[1], Length(SAnsi));
        end;

• concatenation of 2 strs
• converting ASCII to Unicode SWide and later back to ascii SAnsi

[umnyaga]

@Alexey-T , it was just plain ASCII file.

[Alexey-T]

made some work, notes--

• Sublime saves 300M log file within 4 sec, while it reads this file for 10 sec, so saving time (4 sec < 10 sec) is almost fake , ie Sublime doesn't save the entire file, it skips some parts
• after I optimized the stuff I found, above, saving did not become faster!

currently I cannot speed up it. seems my saving time is almost OK? saving of 300M log takes about 15-20 sec on my old PC (SSD drive)

file about 26.5 Mb, 124000 lines. I selected first 7550 lines, deleted them and saved the file. Took about minute or two.

cannot repeat such long time with 25M file.

[Alexey-T]

@umnyaga Do you have the issue on slow HDD? can you see it on SSD drives?

[umnyaga]

@Alexey-T it was network mounted PanFS drive. I do not know what is actual physical drive was there.

[Alexey-T]

i will search for how-to, "how to install PanFS drive on Ubuntu" and if I'll find the docs, i will dig this more.

[umnyaga]

Decided to reproduce the issue. Saving to local ssd takes about a second, for the network file system about a minute.

[Alexey-T]

I did search about PanFS and ffound this https://www.panasas.com/products/panfs/ this is commercial software, right? then I cannot use it. i don't want to pirate it. tell me steps how to install PanFS drive in my Win10 PC.

[Alexey-T]

@umnyaga Can you make the experiment - test if this issue present in Lazarus IDE? (its source code editor - just paste some big text there.) you can easily install Lazarus in portable way - with 2 clicks! fpcUpDeluxe tool - https://github.com/LongDirtyAnimAlf/fpcupdeluxe/releases

if you will see repro in Laz, next step is posting the bugreport, https://gitlab.com/freepascal.org/lazarus/lazarus/-/issues

[umnyaga]

Yes, panfs is commercial software. I do not know how to install it, I just use that network mounted file system at work, our system group manages it.

On Mon, Aug 29, 2022 at 10:13 AM Alexey T. @.***> wrote:

I did search about PanFS and ffound this https://www.panasas.com/products/panfs/ this is commercial software, right? then I cannot use it. i don't want to pirate it. tell me steps how to install PanFS drive in my Win10 PC.

— Reply to this email directly, view it on GitHub https://github.com/Alexey-T/CudaText/issues/4289#issuecomment-1230366122, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALQWW2YLS6GW4YN5QXDPRKTV3TAPPANCNFSM57BHUDAA . You are receiving this because you were mentioned.Message ID: @.***>

[Alexey-T]

@umnyaga Do you see the same issue with

a) Lazarus IDE (it has internal editor - you can load there big file, and call 'save as...') b) Double Commander internal editor (in Options dialog, activate internal editor for F4)

If yes, let's report to https://gitlab.com/freepascal.org/lazarus/lazarus/-/issues

[Alexey-T]

cannot replicate. so i want to close the issue, sorry. if @veksha can install the needed software and replicate it, we can talk more.