REPL: fix the process of reading data from the server.

[Hanaasagi]

Fixes the bug introduced by https://github.com/erg-lang/erg/pull/420

This is a brief description.

• Introduce a small data protocol, where the first two bytes represent the length of the data. So the client knows how many bytes to read.

Changes proposed in this PR:

-

@mtshiba

[mtshiba]

As you point out, the data format should be well-defined. We have defined the length section, but there is a confusion between instructions and data.

Here is a format, I just came up with. This shall be a common protocol for both client (Erg DummyVM) and server (repl_server.py).

format ::= length ' [' instruction '] ' data
instruction ::= 'print' | 'load' | 'exit' | 'initialize' | 'exception' | 'unknown'

• print: Send from server to client. Informs the client to print data.
• load: Send from client to server. Informs the REPL server that the executable .pyc file has been written out and is ready for evaluation.
• exception: Send from server to client. Represents an exception.
• initialize: Send from server to client. Tells the code generator to initialize due to an error.
• exit: Informs that the connection is to be / should be terminated.
• unknown: Informs that it is not a supported instruction.

quit and closed are removed. data can be empty.

[Hanaasagi]

Perhaps we should put the instruction section at the beginning, because some instructions, like load, are not followed by data.

-------------------------------
| ins    | size    | data 
-------------------------------
| 1 byte | 2 bytes | n bytes
-------------------------------

And the instruction can be represented directly in bytes, without the need for a format like [Exception].

enum Inst {
    Print = 0x01,
    Load = 0x02,
    // ...
}

[mtshiba]

That looks better :)

[Hanaasagi]

Maybe I can finish today.

在 2023年4月23日星期日，Shunsuke Shibayama @.***> 写道：

That looks better :)

— Reply to this email directly, view it on GitHub https://github.com/erg-lang/erg/pull/421#issuecomment-1518692722, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACILBG7JJLBSHAXXY2XB57LXCP6IJANCNFSM6AAAAAAXH3XU5M . You are receiving this because you authored the thread.Message ID: @.***>

[Hanaasagi]

A simple test for socket read and write.

import itertools
import random
import string

with open("./src/scripts/repl_server.py") as f:
    code = f.readlines()

code.insert(0, "__PORT__ = 9000\n")
code = itertools.takewhile(lambda l: not l.startswith("# DummyVM"), code)

exec("".join(code))

class MockSocket:
    def __init__(self):
        self.data = bytearray()
        self.cursor = 0

    def send(self, data):
        self.data.extend(data)

    def recv(self, _bufsize):
        if self.cursor >= len(self.data):
            raise Exception()
        # return one byte everytime
        data = bytes(self.data[self.cursor : self.cursor + 1])
        self.cursor += 1
        return data

corr_data = "".join(random.choices(string.ascii_uppercase + string.digits, k=2048))
s = MessageStream(MockSocket())
s.send_msg(INST.PRINT, corr_data)
inst, recv_data = s.recv_msg()

assert inst == INST.PRINT
assert recv_data == corr_data

[mtshiba]

A simple test for socket read and write.

import itertools
import random
import string

...

I'll make a few edits and add this one to the test suite, ok?

[Hanaasagi]

Sure, no problem. Thank you for your review and edits.

[mtshiba]

Thank you so much!