Strange behavior of ReadLn.

[coffeegreg]

I tried to use mORMot2 in my project and encountered some rather unexpected behavior. Let's consider following very simple test code :

program project4;

uses
  mormot.core.data;

const
  MY_FILE = 'myfile.txt';  //Simple file contains one character, e.g. "1"

var
  MyFileContent: string;
  MyFilePath: string;
  F: Text;

begin
  MyFilePath:='D:\Project\Lazarus\Project4';
  Assign(F,MyFilePath+'\'+MY_FILE);
  Reset(F);
  ReadLn(F,MyFileContent);
  Close(F);
  Writeln(MyFileContent);
end.

Test case 1

OS: Win64 FPC64 3.2.2 (regardless of optimization level) mORMot2 (last commit c68d240)

The application "stops" at the readln line and waits for keyboard input instead of reading data from the file. (!?) 1) If we try remove mormot.core.data from uses everything works as it should. 2) If we leave mormot.core.data and change first two lines from :

  MyFilePath:='D:\Project\Lazarus\Project4';
  Assign(F,MyFilePath+'\'+MY_FILE);

to

  MyFilePath:='D:\Project\Lazarus\Project4';
  MyFilePath:=MyFilePath+'\'+MY_FILE;        
  Assign(F,MyFilePath);  

everything works fine too. Strange behaviour. Isn't it?

Test case 2:

OS: Win64 (VBox VM) FPC64 3.3.1 (commit fbec954d 01.09.2023) (optimization level: no optimization or -O1) mORMot2 (last commit c68d240)

Same behavior as test case 1.

Test case 3:

OS: Win64 (VBox VM) FPC64 3.3.1 (commit fbec954d 01.09.2023) (optimization level: -O2, -O3, -O4) mORMot2 (last commit c68d240)

Everything works fine (as expected) regardless of mormot.core.data in uses.

Test case 4:

OS: Win64 (VBox VM) FPC32 3.3.1 (commit fbec954d 01.09.2023) (regardless of optimization level) mORMot2 (last commit c68d240)

Everything works fine (as expected) regardless of mormot.core.data in uses.

Test case 5:

OS: Win64 (VBox VM) FPC64 3.3.1 (commit fbec954d 01.09.2023) (regardless of optimization level) mORMot2 (old commit 4940d574 11.02.2022)

Everything works fine (as expected) regardless of mormot.core.data in uses.

I am not sure if the described problem is directly related to mORMot2 . It looks more like a problem with FPC64. Nevertheless, last test case prompted me to add this issue.

[synopse]

I am not able to reproduce it here.

It may come from the FPC RTL which is patched by mormot.core.rtti.pas. So the problem is not about Readln but about string concatenation.

1) what is the value of the MyFilePath variable? 2) try to define NOPATCHRTL conditional and see what happens. 3) what is the value of DefaultSystemCodePage in your system? 4) try with Win32 as target and not Win64.

Please try to publish something in the forum for further discussion.

[synopse]

I guess I understand the problem.

The assign() method is defined as such: Procedure Assign(out t:Text;const s:rawbytestring);

And with FPC, RawByteString is not a plain string.

So if you write Assign(F,MyFilePath+'\'+MY_FILE); then it will try to compute the concatenation into RawByteString (code page 65535), and mess up the code pages.

But if you write

  MyFilePath:=MyFilePath+'\'+MY_FILE;        
  Assign(F,MyFilePath);  

then it will properly compute the string concatenation into MyFilePath, then assign it to a RawByteString.

IMHO this is more like a FPC RTL bug/Limitation. In Delphi, RawByteString is handled as a "neutral" code page, and the Delphi RTL tries no conversion during the concatenation. In FPC, RawByteString has its own code page, and the FPC RTL tries to make conversions during the concatenation.

It is very disapointing, but it sounds like a FPC RTL bug to me. If you use shortstring or UnicodeString instead of plain string my guess is that your code will work as expected. They should have defined Assign() with 'AnsiString' or 'String' and not 'RawByteString'.

[coffeegreg]

Thank you for your quick reply.

1) To reproduce this strange behavior ... 2) No change with {$define NOPATCHRTL} 3) 65001 4) Yes, but this does not actually solve the problem described, it just workaround it.

Yes, You're absolutely right. This looks like concatenation problem and indeed using shortstring solved it. But I still don't know how to explain the Test case5 with an older version of mORMot2 where the described problem did not occur. :thinking: Any suggestions?