Windows: Invalid argument error

tavmem commented 2 years ago

MINGW64 ~/kona
$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a"; 4 5)

  1: "file"
("a"
 4 5)

  "file" 1: ("a";4 5)
Invalid argument error
"file" 1: data
       ^

tavmem commented 2 years ago

There is a problem opening the file the 2nd time when using the parameter ("a";4 5) If you add these 2 lines to the function _1d_write

$ git diff src/0.c
diff --git a/src/0.c b/src/0.c
index 03c4bb0..2e83889 100644
--- a/src/0.c
+++ b/src/0.c
@@ -576,7 +576,9 @@ Z K _1d_write(K x,K y,I dosync) {

   //Largely copy-pasted from 6:dyadic
   I f=open(e,O_RDWR|O_CREAT|O_TRUNC,07777);
+  O("e:%s\n",e);
   free(e);
+  O("f:%lld\n",f);
   P(f<0,SE)

Then you get f<0 and satisfy P(f<0,SE)

$ ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a";4 5)
e:file.l
f:3

  1: "file"
("a"
 4 5)

  "file" 1: ("a";4 5)
e:file.l
f:-1
Invalid argument error
"file" 1: ("a";4 5)
       ^

Note that if you use the parameter ("a";4) it works OK

$ ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a";4)
e:file.l
f:3

  1: "file"
("a";4)

  "file" 1: ("a";4)
e:file.l
f:3

tavmem commented 2 years ago

It looks like the problem is is caused by the "read", as two successive "writes" works fine

$ ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a";4 5)
e:file.l
f:3

  "file" 1: ("a";4 5)
e:file.l
f:3

tavmem commented 2 years ago

The problem is that the "read" is modifying the file. It looks like this after just the "write"

$ ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a";4 5)
e:file.l
f:3
  \\
$ xxd file.l
00000000: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0200 0000 0000 0000  ................
00000020: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000030: 0300 0000 0000 0000 6100 0000 0000 0000  ........a.......
00000040: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000050: ffff ffff ffff ffff 0200 0000 0000 0000  ................
00000060: 0400 0000 0000 0000 0500 0000 0000 0000  ................

It looks like this after a "write" and a subsequent "read"

$ ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a";4 5)
e:file.l
f:3
  1: "file"
("a"
 4 5)
  \\
$ xxd file.l
00000000: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0200 0000 0000 0000  ................
00000020: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000030: 0300 0000 0000 0000 6100 0000 0000 0000  ........a.......
00000040: fdff ffff ffff ffff 0006 1c00 0000 0000  ................
00000050: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000060: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000070: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000080: 0000 0000 0000 0000 0000 0000 0000 0000  ................

tavmem commented 2 years ago

However, not setting the file as RDONLY seems purposeful. Note these lines in the function _1m in file src/0.c

  //These mmap arguments are present in Arthur's code. WRITE+PRIVATE lets reference count be modified without affecting file
  if(MAP_FAILED==(v=mmap(0,s,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_NORESERVE,f,0)))R SE;

tavmem commented 2 years ago

Well ... this is good news:

$ ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a";4 5)
  \\

$ ./k
kona      \ for help. \\ to exit.

  1: "file"
("a"
 4 5)
  \\
$ xxd file.l
00000000: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0200 0000 0000 0000  ................
00000020: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000030: 0300 0000 0000 0000 6100 0000 0000 0000  ........a.......
00000040: fdff ffff ffff ffff 4006 1c00 0000 0000  ........@.......
00000050: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000060: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000070: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000080: 0000 0000 0000 0000 0000 0000 0000 0000  ................
$

The above shows that the problem created by the "read" ( 1: "file") is not dependent upon the immediate prior execution of the "write" ( "file" 1: ("a";4 5) ). They are independent of each other.

tavmem commented 1 year ago

Actually, the 2 commands are not independent at all. The problem only occurs (on "write") when you execute the "read" and then the "write" in the same session. In this example I do the "write" and the "read" in one session, and then successfully do the "write" in a new session.

$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a"; 4 5)
  1: "file"
("a"
 4 5)
  \\
$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a"; 4 5)

In this example, I do the "write" in one session, then have problems doing a "read" and then "write" in the next session

$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a"; 4 5)
  \\
$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  1: "file"
("a"
 4 5)
  "file" 1: ("a"; 4 5)
Invalid argument error
"file" 1: ("a"; 4 5)
       ^
>

So, it looks like there are 2 problems:

the "read" modifies the file
the "read" creates a condition where a subsequent "write" fails (if done in the same session).

tavmem commented 1 year ago

We know that there appears to be no problem in this case

$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a";4)
  1: "file"
("a";4)
  "file" 1: ("a";4)

However, we can create the very same problem here if we use 2 separate kona sessions on the same Windows machine Session 1:

$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a";4 5)
  1: "file"
("a"
 4 5)

Session 2:

$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a";4)
Invalid argument error
"file" 1: ("a";4)
       ^

However, if you don't do the "read" in session 1, then you can do the "write" in session 1 and then again in session 2. Conclusion: The "read" command

is modifying the file
does not seem to be releasing the file for a subsequent "write" in either session.

tavmem commented 1 year ago

But 1: "file"modifies the file seemingly based on the content of the file. Here, there is no file modification

$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a";4)
  \\
$ xxd file.l
00000000: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0200 0000 0000 0000  ................
00000020: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000030: 0300 0000 0000 0000 6100 0000 0000 0000  ........a.......
00000040: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000050: 0100 0000 0000 0000 0400 0000 0000 0000  ................
$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  1: "file"
("a";4)
  \\
$ xxd file.l
00000000: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0200 0000 0000 0000  ................
00000020: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000030: 0300 0000 0000 0000 6100 0000 0000 0000  ........a.......
00000040: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000050: 0100 0000 0000 0000 0400 0000 0000 0000  ................

Here , there is:

$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a";4 5)
  \\
$ xxd file.l
00000000: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0200 0000 0000 0000  ................
00000020: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000030: 0300 0000 0000 0000 6100 0000 0000 0000  ........a.......
00000040: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000050: ffff ffff ffff ffff 0200 0000 0000 0000  ................
00000060: 0400 0000 0000 0000 0500 0000 0000 0000  ................
$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  1: "file"
("a"
 4 5)
  \\
$ xxd file.l
00000000: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0200 0000 0000 0000  ................
00000020: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000030: 0300 0000 0000 0000 6100 0000 0000 0000  ........a.......
00000040: fdff ffff ffff ffff 4006 1800 0000 0000  ........@.......
00000050: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000060: 0000 0000 0000 0000 0000 0000 0000 0000  ................

tavmem commented 1 year ago

This issue is another manifestation of problems using mmap in a non-Linux enviromment. From issue #615: According to the conclusion reached in this article:

Conclusion:

The mmap() is a powerful system call. 
This function should not be used when there are portability issues because 
this function is only supported by the Linux environment.

We need to find a workaround for mmap in Windows (and, overall, reduce the use of mmap in Kona).

In this issue, the problematic use of mmap is in the function _1m_r in the file src/0.c

This also appears relevant. (from #615)

tavmem commented 1 year ago

This is interesting: When mmap is used in the function Z K _1m_r(I f,V fixed, V v,V aft,I*b) in the file src/0.c: if(MAP_FAILED==(u=mmap(0,size,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_NORESERVE,f,offset))){R SE;} there is no subsequent call to munmap(u,size) which, on the face of it, is a problem.

However, there are immediate steps to track the damage: mMap+=length; mUsed+=length;if(mUsed>mMax)mMax=mUsed;

This indicates awareness that this was a problem, i.e., something that had to be tracked and revisited later.

tavmem commented 1 year ago

My first assessment was that eliminating the second mmap in function _1m_r might fix the Windows problem. It turn out this is done rather easily

diff --git a/src/0.c b/src/0.c
index 24a2753..6bf4fcf 100644
--- a/src/0.c
+++ b/src/0.c
@@ -543,7 +543,8 @@ Z K _1m_r(I f,V fixed, V v,V aft,I*b) {   //File descriptor, moving * into mmap,
     if(fstat(f, &buf))R SE;
     I size = buf.st_size;

-    if(MAP_FAILED==(u=mmap(0,size,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_NORESERVE,f,offset))){R SE;}
+    //if(MAP_FAILED==(u=mmap(0,size,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_NORESERVE,f,offset))){R SE;}
+    u=fixed;

     mMap+=length;
     mUsed+=length;if(mUsed>mMax)mMax=mUsed;

but then we get

$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a";4 5)
  1: "file"
Segmentation fault

To fix the segfault, we need to eliminate the munmap in the function _1m

diff --git a/src/0.c b/src/0.c
index 24a2753..f2d67e9 100644
--- a/src/0.c
+++ b/src/0.c
@@ -508,7 +508,7 @@ K _1m(K x) {    //Keeps binary files mapped
   I b=0;
   K z = _1m_r(f,v,v,v+s,&b);
   r=close(f); if(r)R FE;
-  r=munmap(v,s); if(r)R UE;
+  //r=munmap(v,s); if(r)R UE;
   R z;
 }

@@ -543,7 +543,8 @@ Z K _1m_r(I f,V fixed, V v,V aft,I*b) {   //File descriptor, moving * into mmap,
     if(fstat(f, &buf))R SE;
     I size = buf.st_size;

-    if(MAP_FAILED==(u=mmap(0,size,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_NORESERVE,f,offset))){R SE;}
+    //if(MAP_FAILED==(u=mmap(0,size,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_NORESERVE,f,offset))){R SE;}
+    u=fixed;

     mMap+=length;
     mUsed+=length;if(mUsed>mMax)mMax=mUsed;

But then, we are back where we started

$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a";4 5)
  1: "file"
("a"
 4 5)
  "file" 1: ("a";4 5)
Invalid argument error
"file" 1: ("a";4 5)
       ^
>

bakul commented 1 year ago

May be you can create an API for reading/writing code and data files? It can be implemented differently on different platforms (ifdefed, or implemented in platform specific files).

tavmem commented 1 year ago

Thanks! That is an excellent suggestion.

On another tack, though I realize it may be a digression -> I'm also interested in why the ```munmap```` needs to be eliminated.

tavmem commented 1 year ago

As it might be a good idea to remove unnecessary mmap calls on all platforms, I tried the simple changes in Linux

$ git diff
diff --git a/src/0.c b/src/0.c
index 24a2753..f2d67e9 100644
--- a/src/0.c
+++ b/src/0.c
@@ -508,7 +508,7 @@ K _1m(K x) {    //Keeps binary files mapped
   I b=0;
   K z = _1m_r(f,v,v,v+s,&b);
   r=close(f); if(r)R FE;
-  r=munmap(v,s); if(r)R UE;
+  //r=munmap(v,s); if(r)R UE;
   R z;
 }

@@ -543,7 +543,8 @@ Z K _1m_r(I f,V fixed, V v,V aft,I*b) {   //File descriptor, moving * into mmap,
     if(fstat(f, &buf))R SE;
     I size = buf.st_size;

-    if(MAP_FAILED==(u=mmap(0,size,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_NORESERVE,f,offset))){R SE;}
+    //if(MAP_FAILED==(u=mmap(0,size,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_NORESERVE,f,offset))){R SE;}
+    u=fixed;

     mMap+=length;
     mUsed+=length;if(mUsed>mMax)mMax=mUsed;

(Just like in Windows, I needed to remove the munmap call to prevent a segfault.) I got a surprising result, which alternates between correct and incorrect results

$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a"; 4 5)
  1: "file"
("a"
 4 5)

  "file" 1: ("a"; 4 5)
  1: "file"
()

  "file" 1: ("a"; 4 5)
  1: "file"
("a"
 4 5)

  "file" 1: ("a"; 4 5)
  1: "file"
()

tavmem commented 1 year ago

There is a core dump with the segfault, and according to geeks for geeks

Core Dump/Segmentation fault is a specific kind of error caused by accessing memory that 
“does not belong to you.” 

    When a piece of code tries to do read and write operation in a read only location in memory or 
    freed block of memory, it is known as core dump.

    It is an error indicating memory corruption.

Interesting that memory mmaped only 4 lines earlier in the function _1m no longer "belongs" to that function.

  if(MAP_FAILED==(v=mmap(0,s,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_NORESERVE,f,0)))R SE;

  //TODO: verify that the file is valid K data. For -1,-2,-3 types (at least) you can avoid scanning the whole thing and check size
  I b=0;
  K z = _1m_r(f,v,v,v+s,&b);
  r=close(f); if(r)R FE;
  r=munmap(v,s); if(r)R UE;

tavmem commented 1 year ago

We alternately get correct & incorrect results because of how the file gets written out

$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a"; 4 5)
  1: "file"
("a"
 4 5)
  \\

$ xxd file.K
00000000: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0200 0000 0000 0000  ................
00000020: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000030: 0300 0000 0000 0000 6100 0000 0000 0000  ........a.......
00000040: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000050: ffff ffff ffff ffff 0200 0000 0000 0000  ................
00000060: 0400 0000 0000 0000 0500 0000 0000 0000  ................
$ 
$ 
$ 
$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a"; 4 5)
  1: "file"
("a"
 4 5)
  "file" 1: ("a"; 4 5)
  1: "file"
()
  \\

$ xxd file.K
00000000: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000020: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000030: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000040: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000050: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000060: 0000 0000 0000 0000 0000 0000 0000 0000  ................

Still don't know why we can't munmap the mapping.

tavmem commented 1 year ago

I was experiencing a problem with rlwrap in Linux. It was the same problem described here. I contacted Hans Lub and he

said it was probably due to a change in readline going from 8.1 to 8.2
recommended putting set enable-bracketed-paste off in my .inputrc

That fixed the problem.

It also fixed the problem concerning the alternate correct and incorrect results in Linux in this issue.

The remaining mystery in Linux is why the munmap causes Segmentation fault (core dumped) when you remove the mmap in the function _1m_r by replacing it with u=fixed;

bakul commented 1 year ago

May be you should create a separate issue for linux with a specific test? I don't see a crash under linux but I do see alternate reads not returning the correct object under linux. This works fine under FreeBSD.

Under Ubuntu:

$ ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a"; 4 5);1: "file"
("a"
 4 5)
   "file" 1: ("a"; 4 5);1: "file"
()
   "file" 1: ("a"; 4 5);1: "file"
("a"
 4 5)
   "file" 1: ("a"; 4 5);1: "file"
()

Under FreeBSD:


$ ./k
kona      \ for help. \\ to exit.

   "file" 1: ("a"; 4 5);1: "file"
("a"
 4 5)
   "file" 1: ("a"; 4 5);1: "file"
("a"
 4 5)
   "file" 1: ("a"; 4 5);1: "file"
("a"
 4 5)
   "file" 1: ("a"; 4 5);1: "file"
("a"
 4 5)```

tavmem commented 1 year ago

Yes, I agree. It should be a separate issue ... I will create one. However, I should probably first check Windows and OSX to re-verify that the alternate read problem does not occur there.

Strangely, this alternate read problem disappeared for me using Fedora after I implemented the rlwrap fix in Fedora. I tried it several times and got the correct results.

However, after restarting my Fedora machine today, the alternate read problem is back. I can't explain why.

tavmem commented 1 year ago

Well ... of course ... we can't tell whether if it's a problem in Windows, because we can't get that far

$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a"; 4 5);1: "file"
("a"
 4 5)
  "file" 1: ("a"; 4 5);1: "file"
Invalid argument error
"file" 1: ("a"; 4 5);1: "file"
at execution instance 1 of "1:"

>

I don't have access to my OSX system till sometime next week, so I will proceed to creating a new issue.

tavmem commented 1 year ago

After fixing the Linux "alternate read" problem (issue #633), the original problem for this issue is now fixed in Windows:

$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a";4 5)
  1: "file"
("a"
 4 5)
  "file" 1: ("a";4 5)
  1: "file"
("a"
 4 5)
  "file" 1: ("a";4 5)
  1: "file"
("a"
 4 5)
  "file" 1: ("a";4 5)
  1: "file"
("a"
 4 5)

Before that fix, the Linux "alternate read" problem could not even be tested in Windows (as we couldn't get that far). It now also works fine in Windows:

$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a"; 4 5);1: "file"
("a"
 4 5)
  "file" 1: ("a"; 4 5);1: "file"
("a"
 4 5)
  "file" 1: ("a"; 4 5);1: "file"
("a"
 4 5)
  "file" 1: ("a"; 4 5);1: "file"
("a"
 4 5)

However, a "write" followed by 2 "reads" fails in Windows (but works in Linux)

$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a"; 4 5)
  1: "file"
("a"
 4 5)
  1: "file"
nonce error
>

This is because, as demonstrated earlier ... the first "read" modifies the file (apparently creating gibberish). It's quite possible that this happens because Windows does not fully support "mmap".

tavmem commented 1 year ago

It's probably a good idea to attempt to determine whether k3.2 in Windows uses "mmap" in this case.

tavmem commented 1 year ago

I was hoping that "strace" would work on Windows for checking to see if "mmap" is used in k3.2 in Windows. It does report some activity but not within k. I will probably try "Process Monitor" next

C:\k3.2>strace k
--- Process 25212 created
--- Process 25212 loaded C:\Windows\System32\ntdll.dll at 00007ff9426c0000
--- Process 25212 loaded C:\Windows\SysWOW64\ntdll.dll at 0000000077c30000
--- Process 25212 loaded C:\Windows\System32\wow64.dll at 00007ff941850000
--- Process 25212 loaded C:\Windows\System32\wow64base.dll at 00007ff940c40000
--- Process 25212 loaded C:\Windows\System32\wow64win.dll at 00007ff941cc0000
--- Process 25212 loaded C:\Windows\System32\wow64con.dll at 00007ff9410c0000
--- Process 25212 loaded C:\Windows\System32\wow64cpu.dll at 0000000077c20000
--- Process 25212 loaded C:\Windows\SysWOW64\kernel32.dll at 0000000076d80000
--- Process 25212 loaded C:\Windows\SysWOW64\KernelBase.dll at 0000000076b20000
--- Process 25212 thread 25256 created
--- Process 25212 loaded C:\Windows\SysWOW64\msvcrt.dll at 0000000077800000
--- Process 25212 thread 24020 created
--- Process 25212 loaded C:\k3.2\k20.dll at 0000000010000000
--- Process 25212 loaded C:\Windows\SysWOW64\ws2_32.dll at 00000000760d0000
--- Process 25212 loaded C:\Windows\SysWOW64\rpcrt4.dll at 0000000076e70000
--- Process 25212 thread 23760 created
--- Process 25212 loaded C:\Windows\SysWOW64\user32.dll at 0000000075db0000
--- Process 25212 loaded C:\Windows\SysWOW64\win32u.dll at 0000000076140000
--- Process 25212 loaded C:\Windows\SysWOW64\gdi32.dll at 0000000077970000
--- Process 25212 loaded C:\Windows\SysWOW64\gdi32full.dll at 0000000075fe0000
--- Process 25212 loaded C:\Windows\SysWOW64\msvcp_win.dll at 00000000776c0000
--- Process 25212 loaded C:\Windows\SysWOW64\ucrtbase.dll at 0000000077a00000
--- Process 25212 loaded C:\Windows\SysWOW64\comdlg32.dll at 0000000077740000
--- Process 25212 loaded C:\Windows\SysWOW64\combase.dll at 0000000075b00000
--- Process 25212 loaded C:\Windows\SysWOW64\SHCore.dll at 0000000076160000
--- Process 25212 loaded C:\Windows\SysWOW64\shlwapi.dll at 0000000075f90000
--- Process 25212 loaded C:\Windows\SysWOW64\shell32.dll at 00000000764a0000
--- Process 25212 loaded C:\Windows\WinSxS\x86_microsoft.windows.common-controls_6595b64144ccf1df_5.82.22000.1_none_6ec7c6847ea94424\comctl32.dll at 000000006e9b0000
--- Process 25212 loaded C:\Windows\SysWOW64\advapi32.dll at 0000000077100000
--- Process 25212 loaded C:\Windows\SysWOW64\sechost.dll at 0000000076fa0000
--- Process 25212 loaded C:\Windows\SysWOW64\oleaut32.dll at 00000000778d0000
--- Process 25212, exception 4000001f at 0000000077ce6ce1
--- Process 25212 loaded C:\Windows\SysWOW64\imm32.dll at 0000000075f60000
--- Process 25212 loaded C:\Windows\SysWOW64\uxtheme.dll at 000000006e520000
--- Process 25212 loaded C:\Windows\SysWOW64\msctf.dll at 0000000077020000
--- Process 25212 loaded C:\Windows\SysWOW64\NapiNSP.dll at 000000006f6a0000
--- Process 25212 loaded C:\Windows\SysWOW64\pnrpnsp.dll at 000000006f680000
--- Process 25212 loaded C:\Windows\SysWOW64\mswsock.dll at 00000000755a0000
--- Process 25212 loaded C:\Windows\SysWOW64\dnsapi.dll at 000000006f5d0000
--- Process 25212 loaded C:\Windows\SysWOW64\IPHLPAPI.DLL at 00000000755f0000
--- Process 25212 loaded C:\Windows\SysWOW64\nsi.dll at 0000000077c10000
--- Process 25212 loaded C:\Windows\SysWOW64\winrnr.dll at 0000000074130000
--- Process 25212 loaded C:\Windows\SysWOW64\wshbth.dll at 000000006f5b0000
--- Process 25212 loaded C:\Windows\SysWOW64\nlansp_c.dll at 000000006f590000
--- Process 25212 loaded C:\Windows\SysWOW64\FWPUCLNT.DLL at 000000006f2f0000
--- Process 25212 loaded C:\Windows\SysWOW64\rasadhlp.dll at 000000006f350000
--- Process 25212 thread 4056 created
K 3.2 2005-06-25 Copyright (C) 1993-2004 Kx Systems
WIN32 8CPU 3904MB desktop-fvkenu9.hsd1.pa.comcast.net

  "file" 1: ("a"; 4 5)
  1: "file"
("a"
 4 5)
  "file" 1: ("a"; 4 5)
  --- Process 25212 thread 11596 created
--- Process 25212 thread 14124 created
--- Process 25212 thread 25256 exited with status 0x0
--- Process 25212 thread 24020 exited with status 0x0
--- Process 25212 thread 23760 exited with status 0x0
--- Process 25212 thread 11596 exited with status 0x0
--- Process 25212 thread 14124 exited with status 0x0
--- Process 25212 thread 25312 created
--- Process 25212 thread 25312 exited with status 0x0
--- Process 25212 thread 25096 created
--- Process 25212 thread 2136 created

tavmem commented 1 year ago

When using k3.2, Process Monitor reports that the command "file" 1: ("a"; 4 5) results in 9 operations (when file.l does not already exist):

Operation           Path              Result
ReadFile            c:\k3.2\k20.dll   SUCCESS
CreateFile          c:\k3.2           SUCCESS
QueryDirectory      c:\k3.2\file.l    NO SUCH FILE
CloseFile           c:\k3.2           SUCCESS
CreateFile          c:\k3.2\file.l    SUCCESS
WriteFile           c:\k3.2\file.l    SUCCESS
FlushBuffersFile    c:\k3.2\file.l    SUCCESS
WriteFile           c:\k3.2\file.l    SUCCESS
CloseFile           c:\k3.2\file.l    SUCCESS

Note, there is no mention of file mapping.

When using kona, Process Monitor reports that the command "file" 1: ("a"; 4 5) results in 26 operations (when file.l does not already exist):

Operation                      Path                                Result
CreateFile                     C:\kona\file.l                      NAME NOT FOUND
ReadFile                       C:\Windows\System32\msvcrt.dll      SUCCESS
ReadFile                       C:\Windows\System32\msvcrt.dll      SUCCESS
ReadFile                       C:\Windows\System32\msvcrt.dll      SUCCESS
CreateFile                     C:\kona\file.l                      SUCCESS
ReadFile                       C:\$Secure\$SDH\$INDEX_ALLOCATION   SUCCESS
QueryStandardInformationFile   C:\kona\file.l                      SUCCESS
QueryStandardInformationFile   C:\kona\file.l                      SUCCESS
QueryInformationVolume         C:\kona\file.l                      SUCCESS
QueryAllInformationFile        C:\kona\file.l                      BUFFER OVERFLOW
CreateFile                     C:\                                 SUCCESS
QueryNameInformationFile       C:\                                 SUCCESS
QueryNameInformationVolume     C:\                                 SUCCESS
CloseFile                      C:\                                 SUCCESS
CreateFile                     C:\                                 SUCCESS
QuerySizeInformationVolume     C:\                                 SUCCESS
CloseFile                      C:\                                 SUCCESS
SetEndOfFileInformationFile    C:\kona\file.l                      SUCCESS
SetAllocationInformationFile   C:\kona\file.l                      SUCCESS
CreateFileMapping              C:\kona\file.l                      FILE LOCKED WITH WRITERS
QueryStandardInformationFile   C:\kona\file.l                      SUCCESS
CreateFileMapping              C:\kona\file.l                      SUCCESS
ReadFile                       C:\Windows\System32\msvcrt.dll      SUCCESS
ReadFile                       C:\Windows\System32\msvcrt.dll      SUCCESS
CloseFile                      C:\kona\file.l                      SUCCESS
ReadFile                       C:\kona\file.l                      SUCCESS

NOTE: The CreateFileMapping operation is performed twice.

tavmem commented 1 year ago

When using k3.2, and file.l already exists Process Monitor reports that the command "file" 1: ("a"; 4 5) results in 52 operations. The CreateFileMapping operation is executed 8 times.

Operation | Path | Result
-- | -- | --
ReadFile | C:\k3.2\k20.dll | SUCCESS
CreateFile | C:\k3.2 | SUCCESS
QueryDirectory | C:\k3.2\file.l | SUCCESS
ReadFile | C:\Windows\SysWOW64\msvcrt.dll | SUCCESS
ReadFile | C:\Windows\SysWOW64\msvcrt.dll | SUCCESS
CreateFile | C:\Windows\SysWOW64\tzres.dll | SUCCESS
QueryBasicInformationFile | C:\Windows\SysWOW64\tzres.dll | SUCCESS
CloseFile | C:\Windows\SysWOW64\tzres.dll | SUCCESS
CreateFile | C:\Windows\SysWOW64\tzres.dll | SUCCESS
CreateFileMapping | C:\Windows\SysWOW64\tzres.dll | FILE LOCKED WITH ONLY READERS
QueryStandardInformationFile | C:\Windows\SysWOW64\tzres.dll | SUCCESS
ReadFile | C:\Windows\SysWOW64\tzres.dll | SUCCESS
ReadFile | C:\Windows\SysWOW64\tzres.dll | SUCCESS
CreateFileMapping | C:\Windows\SysWOW64\tzres.dll | SUCCESS
CloseFile | C:\Windows\SysWOW64\tzres.dll | SUCCESS
ReadFile | C:\Windows\SysWOW64\tzres.dll | SUCCESS
RegOpenKey | HKLM\Software\WOW6432Node\Microsoft\LanguageOverlay\OverlayPackages\en-US | REPARSE
RegOpenKey | HKLM\SOFTWARE\Microsoft\LanguageOverlay\OverlayPackages\en-US | NAME NOT FOUND
CreateFile | C:\Windows\SysWOW64\en-US\tzres.dll.mui | NAME NOT FOUND
CreateFile | C:\Windows\System32\en-US\tzres.dll.mui | SUCCESS
CreateFileMapping | C:\Windows\System32\en-US\tzres.dll.mui | FILE LOCKED WITH ONLY READERS
QueryStandardInformationFile | C:\Windows\System32\en-US\tzres.dll.mui | SUCCESS
CreateFileMapping | C:\Windows\System32\en-US\tzres.dll.mui | SUCCESS
CloseFile | C:\Windows\System32\en-US\tzres.dll.mui | SUCCESS
CreateFile | C:\Windows\SysWOW64\tzres.dll | SUCCESS
QueryBasicInformationFile | C:\Windows\SysWOW64\tzres.dll | SUCCESS
CloseFile | C:\Windows\SysWOW64\tzres.dll | SUCCESS
CreateFile | C:\Windows\SysWOW64\tzres.dll | SUCCESS
CreateFileMapping | C:\Windows\SysWOW64\tzres.dll | FILE LOCKED WITH ONLY READERS
CreateFileMapping | C:\Windows\SysWOW64\tzres.dll | SUCCESS
CloseFile | C:\Windows\SysWOW64\tzres.dll | SUCCESS
RegOpenKey | HKLM\Software\WOW6432Node\Microsoft\LanguageOverlay\OverlayPackages\en-US | REPARSE
RegOpenKey | HKLM\SOFTWARE\Microsoft\LanguageOverlay\OverlayPackages\en-US | NAME NOT FOUND
CreateFile | C:\Windows\SysWOW64\en-US\tzres.dll.mui | NAME NOT FOUND
CreateFile | C:\Windows\System32\en-US\tzres.dll.mui | SUCCESS
CreateFileMapping | C:\Windows\System32\en-US\tzres.dll.mui | FILE LOCKED WITH ONLY READERS
QueryStandardInformationFile | C:\Windows\System32\en-US\tzres.dll.mui | SUCCESS
CreateFileMapping | C:\Windows\System32\en-US\tzres.dll.mui | SUCCESS
CloseFile | C:\Windows\System32\en-US\tzres.dll.mui | SUCCESS
CloseFile | C:\k3.2 | SUCCESS
ReadFile | C:\Windows\SysWOW64\msvcrt.dll | SUCCESS
CreateFile | C:\k3.2\file.l | SUCCESS
QueryAttributeTagFile | C:\k3.2\file.l | SUCCESS
SetDispositionInformationEx | C:\k3.2\file.l | SUCCESS
CloseFile | C:\k3.2\file.l | SUCCESS
CreateFile | C:\k3.2\file.l | SUCCESS
WriteFile | C:\k3.2\file.l | SUCCESS
ReadFile | C:\Windows\System32\wow64.dll | SUCCESS
FlushBuffersFile | C:\k3.2\file.l | SUCCESS
WriteFile | C:\k3.2\file.l | SUCCESS
CloseFile | C:\k3.2\file.l | SUCCESS

tavmem commented 1 year ago

When using kona and file.l already exists Process Monitor reports that the command "file" 1: ("a"; 4 5) results in only 27 operations. The CreateFileMapping operation is only executed 2 times

SetDispositionInformationEx     C:\kona\file.l  SUCCESS
CloseFile       C:\kona\file.l  SUCCESS
ReadFile        C:\Windows\System32\msvcrt.dll  SUCCESS
ReadFile        C:\Windows\System32\msvcrt.dll  SUCCESS
CreateFile      C:\kona\file.l  SUCCESS
QueryStandardInformationFile    C:\kona\file.l  SUCCESS
QueryStandardInformationFile    C:\kona\file.l  SUCCESS
QueryInformationVolume  C:\kona\file.l  SUCCESS
QueryAllInformationFile C:\kona\file.l  BUFFER OVERFLOW
CreateFile      C:\     SUCCESS
QueryNameInformationFile        C:\     SUCCESS
QueryInformationVolume  C:\     SUCCESS
CloseFile       C:\     SUCCESS
CreateFile      C:\     SUCCESS
QuerySizeInformationVolume      C:\     SUCCESS
CloseFile       C:\     SUCCESS
SetEndOfFileInformationFile     C:\kona\file.l  SUCCESS
SetAllocationInformationFile    C:\kona\file.l  SUCCESS
CreateFileMapping       C:\kona\file.l  FILE LOCKED WITH WRITERS
QueryStandardInformationFile    C:\kona\file.l  SUCCESS
CreateFileMapping       C:\kona\file.l  SUCCESS
ReadFile        C:\Windows\System32\msvcrt.dll  SUCCESS
ReadFile        C:\Windows\System32\msvcrt.dll  SUCCESS
CloseFile       C:\kona\file.l  SUCCESS
ReadFile        C:\kona\file.l  SUCCESS

tavmem commented 1 year ago

The problem (as we probably already knew) is that the "read" modifies the file

C:\kona>k
kona      \ for help. \\ to exit.

  "file" 1: ("a"; 4 5)
  \\

C:\kona>xxd file.l
00000000: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0200 0000 0000 0000  ................
00000020: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000030: 0300 0000 0000 0000 6100 0000 0000 0000  ........a.......
00000040: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000050: ffff ffff ffff ffff 0200 0000 0000 0000  ................
00000060: 0400 0000 0000 0000 0500 0000 0000 0000  ................

C:\kona>k
kona      \ for help. \\ to exit.

  1: "file"
("a"
 4 5)
  \\

C:\kona>xxd file.l
00000000: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0200 0000 0000 0000  ................
00000020: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000030: 0300 0000 0000 0000 6100 0000 0000 0000  ........a.......
00000040: fdff ffff ffff ffff 4006 d900 0000 0000  ........@.......
00000050: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000060: 0000 0000 0000 0000 0000 0000 0000 0000  ................

tavmem commented 1 year ago

The file gets corrupted. The questions are: "When?", "Where?", "How?". If we make the following change:

$ git diff
diff --git a/src/0.c b/src/0.c
index cfcd49f..0b60bee 100644
--- a/src/0.c
+++ b/src/0.c
@@ -507,6 +507,7 @@ K _1m(K x) {    //Keeps binary files mapped
   //TODO: verify that the file is valid K data. For -1,-2,-3 types (at least) you can avoid scanning the whole thing and check size
   I b=0;
   K z = _1m_r(f,v,v,v+s,&b);
+  I*xx=(I*)(v+104); O("*xx:%lld\n",*xx);
   r=close(f); if(r)R FE;
   r=munmap(v,s); if(r)R UE;
   R z;
$

we get a display of byte #104 of the file mapping (which should be "5"). The display occurred after the mapping is complete and just before the file is "closed" and "unmapped".

$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a";4 5)
  \xxd file.l
00000000: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0200 0000 0000 0000  ................
00000020: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000030: 0300 0000 0000 0000 6100 0000 0000 0000  ........a.......
00000040: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000050: ffff ffff ffff ffff 0200 0000 0000 0000  ................
00000060: 0400 0000 0000 0000 0500 0000 0000 0000  ................
\\
$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  1: "file"
*xx:5
("a"
 4 5)
  \xxd file.l
00000000: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0200 0000 0000 0000  ................
00000020: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000030: 0300 0000 0000 0000 6100 0000 0000 0000  ........a.......
00000040: fdff ffff ffff ffff 4006 1800 0000 0000  ........@.......
00000050: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000060: 0000 0000 0000 0000 0000 0000 0000 0000  ................
\\

This shows that the file mapping is OK, right up to the point that the file is "closed" and "unmapped". Maybe the corruption occurs during the unmapping process?

tavmem commented 1 year ago

"file.l" gets corrupted (in Windows) if you don't assign the result to a global variable:

$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a";3 4);  1: "file"
("a"
 3 4)
  \xxd file.l
00000000: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0200 0000 0000 0000  ................
00000020: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000030: 0300 0000 0000 0000 6100 0000 0000 0000  ........a.......
00000040: fdff ffff ffff ffff 0006 d900 0000 0000  ................
00000050: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000060: 0000 0000 0000 0000 0000 0000 0000 0000  ................
\\

There is no corruption if you do assign the result to a global varialble

$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a";3 4);  b: 1: "file"
  \xxd file.l
00000000: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0200 0000 0000 0000  ................
00000020: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000030: 0300 0000 0000 0000 6100 0000 0000 0000  ........a.......
00000040: fdff ffff ffff ffff 0601 0000 0000 0000  ................
00000050: ffff ffff ffff ffff 0200 0000 0000 0000  ................
00000060: 0300 0000 0000 0000 0400 0000 0000 0000  ................
\\

You can eliminate the file corruption if you make this change (which, at this point, has unknown side effects)

$ git diff
diff --git a/src/kc.c b/src/kc.c
index e761432..e6ccc0d 100644
--- a/src/kc.c
+++ b/src/kc.c
@@ -278,7 +278,7 @@ I line(FILE*f, S*a, I*n, PDA*p)     //just starting or just executed: *a=*n=*p=0
   #endif
  next:
   if(o && fam && !feci)show(k);
-  cd(k);
+  //cd(k);
  cleanup:
   if(fCheck && (strlen(s)==0 || s[strlen(s)-1]<0)) exit(0);
   S ptr=0;
$

you get

$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a";3 4);  1: "file"
("a"
 3 4)
  \xxd file.l
00000000: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0200 0000 0000 0000  ................
00000020: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000030: 0300 0000 0000 0000 6100 0000 0000 0000  ........a.......
00000040: fdff ffff ffff ffff 0601 0000 0000 0000  ................
00000050: ffff ffff ffff ffff 0200 0000 0000 0000  ................
00000060: 0300 0000 0000 0000 0400 0000 0000 0000  ................
\\

tavmem commented 1 year ago

"file.l" gets corrupted (in Windows) with this command sequence

  "file" 1: ("a"; 3 4); 1: "file"

The corruption does not occur (in Windows) with this command sequence

  "file" 1: ("a"; 3 4); b: 1: "file"

which creates the global variable "b".

What happens if "b" is subsequently deleted (removed, expunged) after being created? Is there a k command to delete a global variable? I couldn't find such a command in the k reference manual.

bakul commented 1 year ago

There is no way to delete a global variable. But lack of corruption when you assign the result to a variable points to the result of 1: “file” being used after it being freed. Assigning to b increments it’s recount and prevents it from being freed.

tavmem commented 1 year ago

Thank you for confirming that there is no way to delete a global value!

Regarding the file corruption ... consider

$ git diff
diff --git a/src/kc.c b/src/kc.c
index e761432..02459aa 100644
--- a/src/kc.c
+++ b/src/kc.c
@@ -278,7 +278,9 @@ I line(FILE*f, S*a, I*n, PDA*p)     //just starting or just executed: *a=*n=*p=0
   #endif
  next:
   if(o && fam && !feci)show(k);
+  system("xxd 'file.l'");
   cd(k);
+  system("xxd 'file.l'");
  cleanup:
   if(fCheck && (strlen(s)==0 || s[strlen(s)-1]<0)) exit(0);
   S ptr=0;
$

The contents of "file.l" will be displayed twice, once immediately before cd(k), and once immediately after.

$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a";3 4); 1: "file"
00000000: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0200 0000 0000 0000  ................
00000020: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000030: 0300 0000 0000 0000 6100 0000 0000 0000  ........a.......
00000040: fdff ffff ffff ffff 0601 0000 0000 0000  ................
00000050: ffff ffff ffff ffff 0200 0000 0000 0000  ................
00000060: 0300 0000 0000 0000 0400 0000 0000 0000  ................
00000000: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0200 0000 0000 0000  ................
00000020: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000030: 0300 0000 0000 0000 6100 0000 0000 0000  ........a.......
00000040: fdff ffff ffff ffff 4006 1c00 0000 0000  ........@.......
00000050: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000060: 0000 0000 0000 0000 0000 0000 0000 0000  ................
("a"
 3 4)
  "file" 1: ("a";3 4); b: 1: "file"
00000000: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0200 0000 0000 0000  ................
00000020: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000030: 0300 0000 0000 0000 6100 0000 0000 0000  ........a.......
00000040: fdff ffff ffff ffff 0602 0000 0000 0000  ................
00000050: ffff ffff ffff ffff 0200 0000 0000 0000  ................
00000060: 0300 0000 0000 0000 0400 0000 0000 0000  ................
00000000: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0200 0000 0000 0000  ................
00000020: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000030: 0300 0000 0000 0000 6100 0000 0000 0000  ........a.......
00000040: fdff ffff ffff ffff 0601 0000 0000 0000  ................
00000050: ffff ffff ffff ffff 0200 0000 0000 0000  ................
00000060: 0300 0000 0000 0000 0400 0000 0000 0000  ................
  \\

Note that when a global variable is not used to store the result, the file gets corrupted. It appears that the cd(k) command corrupts the file.

Note that the "unmap" on the flle was performed significantly earlier. It appears that the "unmap" was not effective.

tavmem commented 1 year ago

Note that whether or not the file gets corrupted depends on the content of the data Using the same technique as above to display the file before and after cd(k):

$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a";"b");  1: "file"
00000000: fdff ffff ffff ffff 0601 0000 0000 0000  ................
00000010: fdff ffff ffff ffff 0200 0000 0000 0000  ................
00000020: 6162 0000 0000 0000                      ab......
00000000: fdff ffff ffff ffff 0600 0000 0000 0000  ................
00000010: fdff ffff ffff ffff 0200 0000 0000 0000  ................
00000020: 6162 0000 0000 0000                      ab......
"ab"
  "file" 1: ("c";"d");  1: "file"
00000000: fdff ffff ffff ffff 0601 0000 0000 0000  ................
00000010: fdff ffff ffff ffff 0200 0000 0000 0000  ................
00000020: 6364 0000 0000 0000                      cd......
00000000: fdff ffff ffff ffff 0600 0000 0000 0000  ................
00000010: fdff ffff ffff ffff 0200 0000 0000 0000  ................
00000020: 6364 0000 0000 0000                      cd......
"cd"

we get no corruption.

tavmem commented 1 year ago

Corruption in this case:

$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  "file" 1: ("ab";"cd");  1: "file"
00000000: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0200 0000 0000 0000  ................
00000020: fdff ffff ffff ffff 0601 0000 0000 0000  ................
00000030: fdff ffff ffff ffff 0200 0000 0000 0000  ................
00000040: 6162 0000 0000 0000 fdff ffff ffff ffff  ab..............
00000050: 0601 0000 0000 0000 fdff ffff ffff ffff  ................
00000060: 0200 0000 0000 0000 6364 0000 0000 0000  ........cd......
00000000: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0200 0000 0000 0000  ................
00000020: fdff ffff ffff ffff 5000 8e00 0000 0000  ........P.......
00000030: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000040: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000050: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000060: 0000 0000 0000 0000 0000 0000 0000 0000  ................
("ab"
 "cd")

tavmem commented 1 year ago

This does NOT corrupt: "file" 1: ("a";"c"); 1: "file" This does: "file" 1: ("a";"cd"); 1: "file" This does: "file" 1: ("ab";"c"); 1: "file" This does NOT "file" 1: ("ab"); 1: "file"

bakul commented 1 year ago

What happens if you put a ; at the end to avoid printing the value?

tavmem commented 1 year ago

Corruption

$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a";"cd")
  \\
$ xxd file.l
00000000: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0200 0000 0000 0000  ................
00000020: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000030: 0300 0000 0000 0000 6100 0000 0000 0000  ........a.......
00000040: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000050: fdff ffff ffff ffff 0200 0000 0000 0000  ................
00000060: 6364 0000 0000 0000                      cd......
$
$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  "file" 1: ("a";"cd")
  1: "file";
  \\
$ xxd file.l
00000000: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0200 0000 0000 0000  ................
00000020: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000030: 0300 0000 0000 0000 6100 0000 0000 0000  ........a.......
00000040: fdff ffff ffff ffff 8007 1c00 0000 0000  ................
00000050: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000060: 0000 0000 0000 0000                      ........
$

tavmem commented 1 year ago

This change fixes the problem in Windows

$ git diff
diff --git a/src/0.c b/src/0.c
index cfcd49f..19786c5 100644
--- a/src/0.c
+++ b/src/0.c
@@ -551,7 +551,9 @@ Z K _1m_r(I f,V fixed, V v,V aft,I*b) {   //File descriptor, moving * into mmap,
     z=(K)(((V)u+mod)-3*sizeof(I)); //3*sizeof(I) for c,t,n

     //ref count should be reset to 1 after mapping
+    #ifndef WIN32
     mrc((K)z,1);
+    #endif
     //if(1<=t || 3<=t){dd(z->n)} // ???
   }

$ rlwrap -n ./k
kona      \ for help. \\ to exit.

  "file" 1: ("ab";"cd")
  1: "file"
("ab"
 "cd")
  \\
$ xxd file.l
00000000: fdff ffff ffff ffff 0100 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0200 0000 0000 0000  ................
00000020: fdff ffff ffff ffff 5000 8e00 0000 0000  ........P.......
00000030: fdff ffff ffff ffff 0200 0000 0000 0000  ................
00000040: 6162 0000 0000 0000 fdff ffff ffff ffff  ab..............
00000050: 0000 0000 0000 0000 fdff ffff ffff ffff  ................
00000060: 0200 0000 0000 0000 6364 0000 0000 0000  ........cd......
$

kevinlawler / kona

Windows: Invalid argument error #628