ipc send (1+)

[kevinlawler]

ipc should be able to send type-7-1 verbs. this is related to binary encoding of derived verbs

[tavmem]

Seems to me that this was a planned enhancement to kona (beyond what existed in k3.2). I cannot test this on my copy of k3.2 as the demo version does not include ipc.

[tavmem]

Maybe I'm missing something, but this seems to work now (and that I can close this issue). The following is a transcript of what currently happens (h is the communication handle): server:

./k -i 8200

client:

./k
  3: `"127.0.0.1", 8200         /client responds with handle h
  h 3: "a:5"

server:

  a
5

client:

  h 3: "f:1+"

server:

  f
1+
  f 2
3

[kevinlawler]

Missing bit: example shows sending a string. Something like the following is valid:

h 3: ({1+x}; 22)

Curly brace functions will send. Derived verbs like (1+) will fail because there is no encoding.

Whether this is necessary functionality here is arguable.

[tavmem]

We may be worse off than the original issue description contends ... Currently, It seems that neither of these forms works in Kona: ( {1+x}; 22 ) ( (1+); 22 ) On Linux and on Windows, they each appear to fail. server:

./k -i 8200

client:

./k
  3: `"127.0.0.1", 8200         /client responds with handle h
  h 3: "a:5"

server:

  a
5

client:

  h 3: ( {1+x}; 22 )

There is no apparent result on the server. However, if you hit the "Enter" key on the server, you get the message "nonce error". I get the same behavior for

  h 3: ( (1+); 22 )

Page 188 of the reference manual does list 2 specific examples that should work:

h 3: "b:2+3"
h 3: ( `c; (); :; 5 )

The 1st (string) example works fine. The 2nd example does not. Again, there is no apparent result on the server. Hitting the "Enter" key on the sever produces the message "nonce error".
I will look into this further.

[tavmem]

This issue was opened on May 1, 2011. I did a checkout of the code as it existed on April 30, 2011 and tried

h 3:  "a:2+3"
h 3: ( {1+x}; 22 )
h 3: ( `b; (); :; 5 )

The 1st and 3rd cases work fine. The 2nd case does nothing on the server. There is no message "nonce error". The KTREE on the server has entries for a and forb.

[tavmem]

Quote from page 188 of the reference manual: "Even though all such messages will be executed in the other process, the only effective ones are those that set a value, such as "a:2+3" or ( `a; (); :; 5 ) ."

It may be possible that (when using the Kona code of April 30, 2011) all three cases listed in the prior comment are being executed in the server session, as there is no "nonce error" produced on the server for the 2nd message.

In addition, it may be possible that the message h 3: ( (1+); 22 ) is being executed as well. There is no "nonce error" on the server (as there is when using the current Kona code). There is no other error message in the client or in the server sessions.

In any case, whether either of these messages work h 3: ( {1+x}; 22 ) h 3: ( (1+); 22 ) seems irrelevant. Neither message sets a value, and is not considered "effective".

My current plan is to get the message format h 3: ( `a; (); :; 5 ) to work again (as it does when using the April 30, 2011 code). The message produces a "nonce error" in the server when using the current Kona code.

If there is some other 2-item list or 3-item list format that is "effective" (sets a value), and should be implemented, please let me know.

[kevinlawler]

It's quite likely I got 3: confused with 4:

On Feb 12, 2015, at 6:42 AM, Tom Szczesny notifications@github.com wrote:

Quote from page 188 of the reference manual: "Even though all such messages will be executed in the other process, the only effective ones are those that set a value, such as "a:2+3" or ( `a; (); :; 5 ) ."

It may be possible that (when using the Kona code of April 30, 2011) all three cases listed in the prior comment are being executed in the server session, as there is no "nonce error" produced on the server for the 2nd message. In addition, it may be possible that the message h 3: ( (1+); 22 ) is being executed as well. There is no "nonce error" on the server (as there is when using the current Kona code). There is no other error message in the client or in the server sessions.

In any case, whether either of these messages work h 3: ( {1+x}; 22 ) h 3: ( (1+); 22 ) seems irrelevant. Neither message sets a value, and is not considered "effective".

My current plan is to get the message format h 3: ( `a; (); :; 5 ) to work again (as it does when using the April 30, 2011 code). The message produces a "nonce error" in the server when using the current Kona code.

If there is some other 2-item list or 3-item list format that is "effective" (sets a value), that should be implemented, please let me know.

— Reply to this email directly or view it on GitHub.

[kevinlawler]

(Note however that such an operation is still perfectly well defined. You could set a global async via "{.a:1}". An omission in the manual is perhaps for pertinence.)

On Feb 12, 2015, at 9:43 AM, Kevin Lawler kevin.lawler@gmail.com wrote:

It's quite likely I got 3: confused with 4:

On Feb 12, 2015, at 6:42 AM, Tom Szczesny notifications@github.com wrote:

Quote from page 188 of the reference manual: "Even though all such messages will be executed in the other process, the only effective ones are those that set a value, such as "a:2+3" or ( `a; (); :; 5 ) ."

It may be possible that (when using the Kona code of April 30, 2011) all three cases listed in the prior comment are being executed in the server session, as there is no "nonce error" produced on the server for the 2nd message. In addition, it may be possible that the message h 3: ( (1+); 22 ) is being executed as well. There is no "nonce error" on the server (as there is when using the current Kona code). There is no other error message in the client or in the server sessions.

In any case, whether either of these messages work h 3: ( {1+x}; 22 ) h 3: ( (1+); 22 ) seems irrelevant. Neither message sets a value, and is not considered "effective".

My current plan is to get the message format h 3: ( `a; (); :; 5 ) to work again (as it does when using the April 30, 2011 code). The message produces a "nonce error" in the server when using the current Kona code.

If there is some other 2-item list or 3-item list format that is "effective" (sets a value), that should be implemented, please let me know.

— Reply to this email directly or view it on GitHub.

[tavmem]

To summarize the scope of this issue:

1 arg format: h 3: "a:2+3" works in 4/30/11 code, works in current Kona. On server a is 5

2 arg formats are irrelevant, as they do not set a value.

4 arg formats: h 3: ( `b; (); :; 5 ) works in 4/30/11 code, fails in current Kona. Should work. On the server b should be 5

h 3: ( `c; (); :; {x+1} ) fails in both. Should work. On the server c should be {x+1}

h 3: ( `c; (); :; 1+ ) fails in both. Should work. On the server c should be 1+

3 arg formats: h 3: "c: 0 1 2" h 3: ( `c; (); {x+1} ) fails in both. Should work. On the server c should be 1 2 3

h 3: "c: 0 1 2" h 3: ( `c; (); 1+ ) fails in both. Should work. On the server c should be 1 2 3

[tavmem]

h 3: ( `b; (); :; 5 ) now works.

[tavmem]

In k3.2 and k2.8

  .[ `b; (); +; 5 ]
type error
.[ `b; (); +; 5 ]
^
> 

since the 3rd arg needs to be ":"

If a Kona client session sends the message h 3: ( `b; (); +; 5 ) it will cause a "type error" on the server. The client does not know this, and may not be able to remotely clear the error prompt on the server.

I considered that a problem, and added code so that the client tests to see if the 3rd arg is "colon". If not, raise a "syntax error" on the client, and allow the client to send a corrected mesage.

However, the other parts of the message get sent, and since one part is missing, a "rank error" is now raised on the server (which the client does not know about, and is currently unable to clear remotely).

I see at least 5 potential solutions: (1) Allow sending bad messages. Errors are raised on the server. The client is unaware. (2) Allow the sending of bad messages but somehow inform the client of errors raised on the server. Errors on the server need to be cleared manually at the server. (3) Leave the current code. A syntax error is raised on the client. An incomplete message is sent. A "rank error" is raised on the server. The error on the server needs to be cleared manually at the server, not remotely by the client. (4) Do all syntax checking of a message on the client before any part of the message is sent, and raise a "syntax error" on the client before sending anything. Prevent the sending of a bad message. No error is created on the server. (5) Since the only valid syntax for the 3rd arg is "colon", send "colon" and raise a message on the client to alert that a replacement was made for the 3rd arg. The syntax should be valid, and will probably set a value, but it may not have the result that the client intended.

Options (1), (3) and (5) are the easiest to implement, but may not be the best way to go. There are probably additional options that I have not considered. Comments?

[tavmem]

Two other options to consider: (6) Maybe the server can determine that the "rank error" was caused by getting only 3 args from a client using a 4-arg format. Knowing that the client has already raised a syntax error, only print a warning at the server, but don't raise a "rank error" and an error prompt. (7) Although we can't tell from the "lite" versions of k2.8 and k3.2 what the full versions would have done, we do thave the "full" version of 32-bit KDB+. Check out what 32-bit KDB+ does under similar conditions and do the same in Kona.

[tavmem]

Next step: Find out what 32-bit KDB+ does in the case of a client attempting to send a message that will clearly raise an error condition on the server. In particular: (a) is the client session stopped from sending a "bad" message? (b) if the "bad" message is sent, is the client session aware that the server has raised an error?

[tavmem]

One more option: (8) Have the client session send the full "bad" message, but print a message at the client stating that an error was probably raised at the server session, just so the client is aware.

[tavmem]

Based on cursory testing, it seems that q tests the syntax of a message in the client session before sending anything to the server. If an error is found, the error is raised in the client session. No error is raised on the server since no message is sent. However, syntactically correct messages can still cause errors.

There is a warning on page 318 of the 2008 printing of "q For Mortals" by Jeffrey Borror: "Exercise caution when sending entities to a remote server. A trivial mistake could place the server into a non-reponding state."

Sample q sessions server:

tom@xps ~ $ q/l32/q -p 5042

client:

tom@xps ~ $ q/l32/q
q)h:hopen `::5042
q)f:{x*x}
q)h (f;2)
4
q)h (f;2;3)
'rank
q)

No error was raised in the server session. Apparently, no message was sent by the client.

[bakul]

IMHO only syntactically valid message should be sent. Runtime errors have to be caught during runtime so there's nothing you can do about that on the sending side. But it does mean you should be able to report them back to the sending side. To regain control in case you sent an infinite loop you need an ability to send ^C too.

On Feb 21, 2015, at 9:10 AM, Tom Szczesny notifications@github.com wrote:

Based on cursory testing, it seems that q tests the syntax of a message in the client session before sending anything to the server. If an error is found, the error is raised in the client session. No error is raised on the server since no message is sent. However, syntactically correct messages can still cause errors.

There is a warning on page 318 of the 2008 printing of "q For Mortals" by Jeffrey Borror: "Exercise caution when sending entities to a remote server. A trivial mistake could place the server into a non-reponding state."

Sample q sessions server:

tom@xps ~ $ q/l32/q -p 5042 client:

tom@xps ~ $ q/l32/q q)h:hopen `::5042 q)f:{x*x} q)h (f;2) 4 q)h (f;2;3) 'rank q) No error was raised in the server session. Apparently, no message was sent by the client.

— Reply to this email directly or view it on GitHub.

[tavmem]

Actually, it is not absolutely clear whether a message was sent, or not. Consider the following client session: client:

q)h "{x*x}[2]"
4
q)h "{x*x}[2;3]"
'rank
q)

The message is simply a character string to be executed on the server. Is the syntax checked on the client or on the server? However, it is clear that the error is reported only in the client session.

[tavmem]

The following demonstrates that maybe q does send syntactically "invalid" messages to the server. (On the other hand, maybe the client breaks up a character stream into separate commands and sends each command ... one at a time ... only if the command is "valid".) client:

q)h "b:1; b:b+{x*x}[2;3]"
'rank
q)

server:

q)b
1
q)

Again, there is no error raised at the server.

[tavmem]

I agree that it is best to send only syntactically valid messages. However, h 3: ( b; (); :; 5 ) currently works. h 3: (b; (); +; 5) and similar "invalid" 4-arg messages do raise a syntax error in the client.

To me, features that don't work at all yet should get higher priority, and attention first: h 3: ( c; (); :; {x+1} ) h 3: (c; (); :; 1+ ) h 3: "c: 0 1 2"; h 3: ( c; (); {x+1} ) h 3: "c: 0 1 2"; h 3: (c; (); 1+ )

So, next step: h 3: ( `c; (); :; {x+1} )

[tavmem]

Bakul wrote:

IMHO only syntactically valid message should be sent. Runtime errors have to be caught during runtime so there's nothing you can do about that on the sending side. But it does mean you should be able to report them back to the sending side. To regain control in case you sent an infinite loop you need an ability to send ^C too.

Observations on how some aspects of this is handled in q Server:

   q -p 5042

Client:

  q
  h: hopen `::5042
  h "b:2"

Server:

  b
2

Client:

  h "c:b"

Server:

  c
2

Client:

  x:3
  h "y:x"
'x

Server:

  y
'y

Firstly, 'x seems to be q shorthand for "value error on x". h "y:x" is a syntactically valid message, but x only exists on the client, so the server cannot execute this command, and an error should occur on the server. However, the error message is reported on the client, with no error report on the server. The server seems unaffected, with no need to recover or reset the server.

[bakul]

Likely the server sends back errors over the same connection. In effect the server is loading from a FIFO instead of a file & the reverse direction is connected to stderr.

On Apr 9, 2015, at 9:05 AM, Tom Szczesny notifications@github.com wrote:

Bakul wrote:

IMHO only syntactically valid message should be sent. Runtime errors have to be caught during runtime so there's nothing you can do about that on the sending side. But it does mean you should be able to report them back to the sending side. To regain control in case you sent an infinite loop you need an ability to send ^C too.

Observations on how some aspects of this is handled in q Server:

q -p 5042 Client:

q h: hopen `::5042 h "b:2" Server:

b 2 Client:

h "c:b" Server:

c 2 Client:

x:3 h "y:x" 'x Server:

y 'y Firstly, 'x seems to be q shorthand for "value error on x". h "y:x" is a syntactically valid message, but x only exists on the client, so the server cannot execute this command, and an error should occur on the server. However, the error message is reported on the client, with no error report on the server. The server seems unaffected, with no need to recover or reset the server.

— Reply to this email directly or view it on GitHub.