Access violation

[GoogleCodeExporter]

Code:

assert(true, 'z');

Output:

Syntax Error: Access violation at address 004EF13B in module 'dws.exe'. Read of 
address 0000000B

Original issue reported on code.google.com by kazantse...@mail.ru on 13 Nov 2013 at 10:17

[GoogleCodeExporter]

also this code:

procedure test; deprecated 'z';
begin
end;

Original comment by kazantse...@mail.ru on 13 Nov 2013 at 11:49

[GoogleCodeExporter]

I can't reproduce your issues. Which versions are you using? (DWScript & Delphi)

Original comment by zar...@gmail.com on 14 Nov 2013 at 7:13

• Changed state: NeedMoreInfo

[GoogleCodeExporter]

Delphi 2010. rev.2327

Original comment by kazantse...@mail.ru on 14 Nov 2013 at 9:01

[GoogleCodeExporter]

Error at dwsTokenizer (line 1524). trns is nill.

Original comment by kazantse...@mail.ru on 14 Nov 2013 at 9:10

[GoogleCodeExporter]

I don't have D2010, but the error location is surprising.

Would you have a sample/simple project that triggers the issue?

Original comment by zar...@gmail.com on 14 Nov 2013 at 9:45

[GoogleCodeExporter]

Error is reproduced when {$HIGHCHARUNICODE ON}

Original comment by kazantse...@mail.ru on 14 Nov 2013 at 1:39

[GoogleCodeExporter]

Tried forcing {$HIGHCHARUNICODE ON} in dws.inc, still no issue here in XE.

Can you reproduce it in a simple/sample project? f.i. in the dwsRunner demo and 
then attach your project with all settings?

Original comment by zar...@gmail.com on 14 Nov 2013 at 1:55

[GoogleCodeExporter]

Reproduced on Delphi XE too.

Original comment by kazantse...@mail.ru on 14 Nov 2013 at 4:30

Attachments:

• dws_issue_448.zip

[GoogleCodeExporter]

You pre-compiled executable gives me the error, but when compiling it here, 
release or debug, I don't have no error...

Also when I compile my executable size is 1048 kB, but your pre-compiled 
executable is 1061 kB, so there has to be some difference in the code you're 
compiling, the Delphi version / RTL or some compile option outside of the dproj 
reach...

What's your XE version? I've got 15.0.3953.35171, ie. with Update 1 and IDE Fix 
Pack XE 5.3 installed.

Original comment by zar...@gmail.com on 15 Nov 2013 at 10:32

[GoogleCodeExporter]

My executable (size: 1048K - 1 073 152 bytes) was compiled on clean 
installation of Delphi 2010 Update 5 (14.0.3615.26342) without any installed 
plugins and experts and on clean Windows XP SP3 (it's separate virtual machine 
for test purposes).

Also i compile this on clean installation of Delphi XE Update 1 
(15.0.3953.35171) without any installed plugins and experts and on clean 
installation of Windows XP SP3 (it's other virtual machine for test purposes). 
I have executable of size: ~1058K - 1 083 904 bytes. Bug is reproduced.

Original comment by kazantse...@mail.ru on 15 Nov 2013 at 2:38

Attachments:

• dws_issue_448_xe.zip

[GoogleCodeExporter]

Your exe comes at 1059 kB, so it's still smaller than what I see here by a 
small amount with XE. I've tried without IDE Fix Pack, it makes no difference.

Apart from stray files or DCUs, the only explanation could be related to 
compiler settings not part of your dproj, but at the IDE level (Value from 
"base").

When opening your dproj, I see those options:
http://imgur.com/YtRXVpb

Are those you see on your side? if not could you post yours?

Original comment by zar...@gmail.com on 15 Nov 2013 at 3:05

[GoogleCodeExporter]

I have the same options. May be this is not important, but In my case 
DWS-packets is not installed in Delphi.

Original comment by kazantse...@mail.ru on 15 Nov 2013 at 3:18

Attachments:

• xe_compiler_options.png

[GoogleCodeExporter]

Installing packages shouldn't have any effect.
I see you're on XP, but I don't see why that should influence the Delphi 
compiler, but maybe it does?

Two last things to try:
- zip of the files I have here in case there was some oddity with the SVN:  
http://www.delphitools.info/test/Source.zip
- if that fails could you attach the details MAP file you get for XE with the 
options you posted? to help figure out where the delta in size comes from

Original comment by zar...@gmail.com on 15 Nov 2013 at 3:30

[GoogleCodeExporter]

I compiled example with your sources. Executable size is reduced on 2Kb. Bug is 
reproduced.

Original comment by kazantse...@mail.ru on 15 Nov 2013 at 3:49

Attachments:

• dws_issue_448_xe_source.zip

[GoogleCodeExporter]

Our two map files begin to diverge in dwsUtils between line 2677 and line 2713, 
but that's where the compiler stored the file path for an Assert message.

Could you attach the map with assertions off?

Original comment by zar...@gmail.com on 15 Nov 2013 at 4:07

[GoogleCodeExporter]

Original comment by kazantse...@mail.ru on 15 Nov 2013 at 4:13

Attachments:

• dws_issue_448_xe_source_without_assertions.zip

[GoogleCodeExporter]

Map files perfectly match now, and exe length as well.

However your binary differs from mine for a few constants (not sure which just 
yet), where you have a series of (in hexa) FF 03 00 00 00 ... and I have FF FF 
FF FF ...

I suspect this could be some issue with your local codepage and a set, this is 
the only reason why HIGHCHARUNICODE could affect you but not me.

Could you try in dwsPascalTokenizer to change cANYCHAR to  [#0..#127] ?

Original comment by zar...@gmail.com on 15 Nov 2013 at 4:37

[GoogleCodeExporter]

This modification solve problem.

Help about $HIGHCHARUNICODE:

 {$HIGHCHARUNICODE ON}
   A := #$80; // Ord(A) = $3F ('?')
   W := #$80; // Ord(W) = $80

   A := #200; // Ord(A) depends on default ANSI codepage; for example
              //   = $45 for Win1251 codepage ('E')
              //   = $C8 for Win1252 codepage ('È');
   W := #200; // Ord(W) = 200 ($C8) 

Original comment by kazantse...@mail.ru on 15 Nov 2013 at 4:47

[GoogleCodeExporter]

For curiosity's sake, what code page is your XP machine running under?

The #255 was a leftover from the pre-Unicode version, I guess it got through as 
long as it mapped to anything outside ASCII range, when converting to Unicode, 
the tables and ranges were limited to #0..#127. Anyway thanks for the time, 
I'll probably blog about it, it's an interesting leftover bug from hell of the 
Unicode conversion (which was done in 2009)

Original comment by zar...@gmail.com on 16 Nov 2013 at 12:46

• Changed state: Started

[GoogleCodeExporter]

Cyrillic 1251. :) Thank for your work Eric.

Original comment by kazantse...@mail.ru on 16 Nov 2013 at 2:17

[GoogleCodeExporter]

Hmm, there is still something I don't exactly understand, in 1251, character 
255 maps to 'я', ie. Unicode $44F, since that's above 255, and only the 0..127 
range is used in the set, it should have worked, just wasted a few hundred bits 
in the set.

So apparently there has to be some bug in the compiler?
With HIGHCHARUNICODE ON, what character does get displayed for #255 in your 
case? the question mark "?" (ie. some conversion failure) or "F" (ie. the 
compiler keeps only the $4F)

Original comment by zar...@gmail.com on 18 Nov 2013 at 7:39

[GoogleCodeExporter]

The [#0..#255] is silently interpreted as [#0..'?']...

Original comment by zar...@gmail.com on 18 Nov 2013 at 8:08

• Changed state: Fixed

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

This not compiler bug. This such mechanism of unicode codepoints conversion to 
AnsiChar. Help:

-----------------------------------------------------------------
The {$HIGHCHARUNICODE ON} directive controls the behavior of characters #$80 
... #$FF (#128 ... #255). 

When HIGHCHARUNICODE is OFF: 

All decimal #xxx n-digit literals are parsed as AnsiChar. 
All hexadecimal #$xx 2-digit literals are parsed as AnsiChar. 
All hexadecimal #$xxxx 4-digit literals are parsed as WideChar. 
When HIGHCHARUNICODE is ON: 

All literals are parsed as WideChar. 
-----------------------------------------------------------------

Hence, when directive is ON, for code #255 compiler logic a next:

1. Consider #255 as Unicode codepoint - #$00FF;
2. Get AnsiChar for this codepoint by used default (current) code page - 
TEncoding.Default.GetBytes(#$00FF)[0]; (for code page 1251 is 'y' - 121 ($79));

Hence set of [#0 .. #255] will be represented as [#0 .. 'y'].

Original comment by kazantse...@mail.ru on 18 Nov 2013 at 9:17

[GoogleCodeExporter]

To me the bug is that the "set" syntax then accepts an Unicode character 
without error, and silently convert it to Ansi.
If you pass a Unicode character literal directly, you get a range error, but 
with HIGHCHARUNICODE, you don't get any error, just a silent cast.

I can't construe a scenario in which the current compiler behavior would be 
desirable.

Original comment by zar...@gmail.com on 18 Nov 2013 at 9:38

[GoogleCodeExporter]

Compiler frankly generate warning like:
[DCC Warning] Project1.dpr(11): W1061 Narrowing given WideChar constant 
(#$00FF) to AnsiChar lost information

This is sooner bug of my attentiveness :)

Original comment by kazantse...@mail.ru on 18 Nov 2013 at 9:52

[GoogleCodeExporter]

I don't get anything here, this warning should really be tied to the conversion 
itself, rather than its outcome, if it had been, I would have spotted the issue 
long ago. :(

Original comment by zar...@gmail.com on 18 Nov 2013 at 9:54