Character arrays not null-terminated

[tingtron]

Version: J902 beta, J901, J807. Tested in 64-bit J on Windows, but may be in 32-bit. Possibly related to https://code.jsoftware.com/wiki/Guides/DLLs/Calling_DLLs#J807_Incompatibilities

Steps to reproduce

• Copy/paste into a new script in Edit:

  load'general/misc/format'             NB. this just to reproduce more surely
  T=: '123456'
  T=: }:T                               NB. sometimes reproduced without this line
  a.i.memr (symdat<'T'),0,(1+#T),JCHAR  NB. size+1 chars - last expected = 0

• Select all lines and press Ctrl+E
• Expected: 49 50 51 52 53 0
• Observed: 49 50 51 52 53 54, or other non-zero trailing character

Impact

Many third-party DLLs expect null-terminated C strings. A workaround to append 0{.a is an inconvenience and inefficiency. More specifically, the most straightforward and efficient method to pass a string parameter is using cd declaration with both > (no parameter wrapping) and &c (no copy). Then when passing a J noun to a &c parameter, we effectively pass a pointer to its data (as shown in the example).

Note: In previous J versions it was guaranteed that string arrays were null-terminated. So before J807, *c was used as no-copy parameter (which is &c now) and it always worked.

[bilam]

This is not bug. It is the intended behavior. when calling dll via 15!:0, string parameters (c) are still guaranteed null-terminated. For passing memory pointer ( or x) directly, you need to be careful.

[tingtron]

when calling dll via 15!:0, string parameters (_c) are still guaranteed null-terminated.

The original issue is updated to be more specific. In particular, the null-terminated strings are expected to work in the > ... &c use case.

It would be good to understand the reason why since J807 strings are no longer null-terminated?

[bilam]

the browser email client had eaten the "star" in my message, when calling dll via 15!:0, string parameters ("star"c) are still guaranteed null-terminated. For passing memory pointer ("star" or x) directly, you need to be careful.

Avoid using &c "amp"c in 15!:0 call unless you know what you are doing. see documentation for detail.

Substrings are no longer null-terminated otherwise each substring must be a value copy of the original, see the example you had quoted, null terminated would overwrite the original noun.

[bilam]

correction. when new T is realized, new memory space is allocated independent of that for the old T. setting trailing null will not overwrite the original which had been released, but still trailing null is not mandatory therefore not initialized.

[tingtron]

still trailing null is not mandatory therefore not initialized

That is the main question: Why null is not added to the end of (realized) string, compared to earlier versions of J?

Even the binary/hex representation of string contains terminating null:

   3!:3 '12345678'
e300000000000000
0200000000000000  T = JCHAR
0800000000000000  N
0100000000000000  R
0800000000000000  0{S
3132333435363738  <= 8 bytes of the string
0000000000000000  <= the terminating null

[bilam]

For efficiency. padding trailing null in 3!:1/3 are added so results are predictable. eg (3!:3 '1234567')-:(3!:3 }:'12345678')

[HenryHRich]

In general strings CANNOT be terminated, because they may be virtual blocks which are part of other blocks. Termination is added to strings when they are passed outside of JE