TCP Sockets

[reijne]

Is your feature request related to a problem? Please describe. Currently there exists no (standard) method of utilising TCP sockets in Rascal, at least not that I can find.

Describe the solution you'd like Take a look at my proposed idea of implementation and tell me how it can be done better, or if there already exist an alternative methodology for achieving TCP sockets please show me the way.

Describe alternatives you've considered The solution proposal is based on the following structure: JAVA::Client & Server RASCAL::ClientController & ServerController Network.zip

Additional context I tried to create a singular controller but was unable to get the blocking server waiting for connection to go through such that the following statement initialising the client would be executed. A possible solution for this would be utilising threads but I was unable to do so.

[DavyLandman]

We have a build in webserver: util::Webserver that can get you quite far. Was there a specific use case in mind for this feature request?

But generic features for an abritraty TCP server/client application runs into two issues:

• we have no way to represent native bytes
• rascal is a single-threaded model and we have no async features (yet)

I would also like this, but there has to change quite a bit to make it nicer to use.

With regards to your proposed solution, the server implementation blocks for the first client to get connected, and then will not handle new clients. (that's the problem with single-threaded code of rascal, we have no concept of forking new threads/ frames of execution).

[reijne]

The use case is connecting Rascal to Unity, or better said, linking Rascal to Java which will then connect to a .NET socket within a running Unity application.

This way I can utilise Rascal on a DSL and send the information to the Unity application that will be started up using Java.

The unity application version used for this will have a blocking server on startup, which will be no issue since im starting it from java, whereafter I send the information it is waiting for using the client immediately.

[DavyLandman]

Ah cool, yeah for that a single connection looks fine, I would advice continuing like you are doing. But offering this as a general thing will be hard.

[reijne]

Yeah I noticed with not being able to have some fun multithreading. Is there not a way to circumvent this by having multiple Java threads that somehow all feed into Rascal at the same entry point with some ThreadID to distinguish them?

[jurgenvinju]

The Rascal interpreter is really not thread-safe or even thread-friendly, but if you get a function as a parameter from a java builtin and call it, that function will make sure to lock the interpreter automatically. So this means you can call callbacks from Java from any thread, but you will not get the benefit of parallel execution.

So let's say you write:

java void startSocketConnection(value(value) reply);

then in Java it would look like:

public void startSocketConnection(IFunction reply) {
    ... perhaps code that starts a new Thread and runs:
    {
      while (true) {
         input = receive()
         output = reply.call(input); // this would lock the interpreter that called startSocketConnection
         send(output);
      }
    }   
}

There is a chance of deadlocks here and I'm not sure how to avoid it.

[reijne]

Hmm interesting, thanks for the replies :)

[reijne]

Would it be fair to say: benefit, an interface for TCP socket connection both Server and Client side, with a pitfall of being limited to having one consecutive connection at a time?

[jurgenvinju]

Sure, but multiple connections are possible it's just that they would be handled concurrently but not in parallel due to the interpreter lock. I think a socket library would be a very valuable addition to the standard library.

I wonder how you would parametrize the wire protocol. For a webserver it's http but a generic sockets can pass any data. What's the plan there? For example, it would be cool if we could use json-rpc on top or some other protocol, binary Rascal values even. Or are we going for a fixed protocol for now?

On Tue, 6 Jul 2021 at 22:57, snipy123 @.***> wrote:

Would it be fair to say: benefit, an interface for TCP socket connection both Server and Client side, with a pitfall of being limited to having one consecutive connection at a time?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/usethesource/rascal/issues/1516#issuecomment-875076056, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAPF5F24FANTT3H7H6Q45D3TWNU3DANCNFSM474UYDEQ .

-- Jurgen Vinju http://jurgen.vinju.org CWI SWAT TU Eindhoven MDSE Swat.engineering BV

[reijne]

Indeed having multiple connections, but handling one at a time. Could still be useful I think.

As for the protocol I was heading in the json-rpc direction.

And the Rascal Binary value; does that mean sending arbitrary Rascal Values over the Socket much like ValueIO does writing to File?

[jurgenvinju]

yes just like File IO. The underlying vallang API supports streaming binary values based on Protocol Buffers.

On Wed, Jul 7, 2021 at 1:23 AM snipy123 @.***> wrote:

Indeed having multiple connections, but handling one at a time. Could still be useful I think.

As for the protocol I was heading in the json-rpc direction.

And the Rascal Binary value; does that mean sending arbitrary Rascal Values over the Socket much like ValueIO does writing to File?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/usethesource/rascal/issues/1516#issuecomment-875147566, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAPF5F2SBQKN3IKOINAENU3TWOF57ANCNFSM474UYDEQ .

-- Jurgen Vinju http://jurgen.vinju.org CWI SWAT TU Eindhoven MDSE Swat.engineering BV

[jurgenvinju]

JSON-RPC sounds good. So we first have a JSON-RPC specific Socket library. Later we could generalize to other protocols, or simply offer other protocols next to this.

[jurgenvinju]

@reijne I was wondering about your results.. did you manage to implement a JSON-RPC connection?

[reijne]

Yes sir, indeed I managed to implement a JSON-RPC connection. Rascal calls upon a Java Client to create a TCP connection. The Java Client looks as follows:

package Network;

import java.net.*;
import java.io.*;

import io.usethesource.vallang.IInteger;
import io.usethesource.vallang.IString;
import io.usethesource.vallang.IValueFactory;

// Insipiration:: https://www.baeldung.com/a-guide-to-java-sockets
public class Client {
    private final IValueFactory values;
    private Socket clientSocket;
    private PrintWriter out;
    private BufferedReader in;

    // Initialise the Client with the valuefactory (link to Rascal).
    public Client(IValueFactory values){
        this.values = values;
    }

    // Start the client by connecting to the ip+port.
    public void  startClient(IString ip, IInteger port) throws IOException {
        String ipString = ip.getValue();
        int portInteger = port.intValue();
        clientSocket = new Socket(ipString, portInteger);
        out = new PrintWriter(clientSocket.getOutputStream(), true);
        in = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));        
    }

    // Send a message to the connected server and return the response.
    public IString sendMessage(IString msg) throws IOException {
        String msgString = msg.getValue();
        out.println(msgString);
        return values.string("");
    }

    // Stop the client by closing the streams and connection.
    public void stopClient() throws IOException {
        in.close();
        out.close();
        clientSocket.close();           
    }
}

Then within Rascal Remote Process Calls can be made using a basic contract and the sendMessage function:

module Network::ClientController

import IO;
import Exception;
import lang::json::IO;

import Network::ServerController;

@javaClass{Network.Client}
public java int startClient(str ip, int port);
@javaClass{Network.Client}
public java str sendMessage(str msg);
@javaClass{Network.Client}
public java void stopClient();

private bool isConnected = false;
private str localhost = "127.0.0.1";
private int showeyPort = 529;
private int sceneyPort = 530;

@doc {
    Close the Unity Server Socket using close:true
}
void closeServer() {
    sendMessage(remoteCall("", "", true));
}

@doc {
    Create a Remote Process Call to send over the Socket, using JSON rpc format.
}
str remoteCall(str method, str param, bool close) {
    rpc = ("method": method,
              "param": param,
              "close": close);
    return asJSON(rpc);
}

NOTE, I have not fully** implemented the server side of the connection fully within Rascal, instead I constructed it in Unity. The basic setup again consists of the Rascal module linked to Java class.

The Network package can be found here

[jurgenvinju]

Thanks for sharing this @reijne. We'll study it and try to see if we can generalize it for inclusion in the library. It would be great if your code were open-source, for example BSD-2 licensed. Please don't put GPL or L-GPL on it, because then we can't use it.

[reijne]

I've updated the license from MIT to BSD-2, which should allow you to use it in any way you see fit.

[jurgenvinju]

excellent! for licenses like this to "work" in the USA, every file needs to have a license header unfortunately. I usually put them in an @license{ } tag on the module in Rascal.