search

asynkron / protoactor-go

Proto Actor - Ultra fast distributed actors for Go, C# and Java/Kotlin
http://proto.actor
Apache License 2.0
5.09k stars 523 forks source link

Server based on protoactor with address 0.0.0.0 doesn't work #833

Open stdstring opened 1 year ago

stdstring commented 1 year ago

Describe the bug I made server and client apps with help of protoactor library as the transport. When server has real address (e.g. 127.0.0.1), then all work well. When server has 0.0.0.0 address, then interaction between server and client doesn't work.

To Reproduce I have the following definition with messages for protobuf:

syntax = "proto3";
option go_package = "./messages";
package messages;

message RequestTypeA {
    string Data = 1;
}

message ResponseTypeA {
    string Data = 1;
}

I have the following definition of server app:

package main

import (
    "fmt"
    console "github.com/asynkron/goconsole"
    "github.com/asynkron/protoactor-go/actor"
    "github.com/asynkron/protoactor-go/remote"
    "os"
    "protoActorRemote/messages"
)

const host = "127.0.0.1"
//const host = "0.0.0.0"
const port = 23333
const name = "ProtoactorProblemExample"

func main() {
    system := actor.NewActorSystem()
    config := remote.Configure(host, port)
    remoter := remote.NewRemote(system, config)
    remoter.Start()
    props := actor.PropsFromFunc(func(context actor.Context) {
        switch msg := context.Message().(type) {
        case *actor.Started:
            fmt.Printf("Started server with address \"%v\"\n", context.Self().Address)
        case *actor.Stopping:
            fmt.Printf("Stopping server with address \"%v\"\n", context.Self().Address)
        case *actor.Stopped:
            fmt.Printf("Stopped server with address \"%v\"\n", context.Self().Address)
        case *actor.Restarting:
            fmt.Printf("Restarting server with address \"%v\"\n", context.Self().Address)
        case *messages.RequestTypeA:
            fmt.Printf("RequestTypeA with data \"%v\" from client with address \"%v\" (server address = \"%v\")\n", msg.Data, context.Sender().Address, context.Self().Address)
            context.Send(context.Sender(), &messages.ResponseTypeA{Data: msg.Data + ".resultA"})
        default:
            fmt.Printf("Other message %v\n", msg)
        }
    })
    _, spawnError := system.Root.SpawnNamed(props, name)
    if spawnError != nil {
        fmt.Printf("Spawn error \"%v\"\n", spawnError)
        os.Exit(-1)
    }
    fmt.Println("Server started")
    _, _ = console.ReadLine()
}

I have the following definition of client app:

package main

import (
    "fmt"
    console "github.com/asynkron/goconsole"
    "github.com/asynkron/protoactor-go/actor"
    "github.com/asynkron/protoactor-go/remote"
    "os"
    "protoActorRemote/messages"
    "time"
)

const host = "127.0.0.1"
const port = 34444
const serverAddress = "127.0.0.1:23333"
const serverName = "ProtoactorProblemExample"

func main() {
    system := actor.NewActorSystem()
    config := remote.Configure(host, port)
    remoter := remote.NewRemote(system, config)
    remoter.Start()
    serverPID := actor.NewPID(serverAddress, serverName)
    response, requestError := system.Root.RequestFuture(serverPID, &messages.RequestTypeA{Data: "IDDQD"}, 1*time.Second).Result()
    if requestError != nil {
        fmt.Printf("Request error \"%v\"\n", requestError)
        os.Exit(-1)
    }
    fmt.Printf("Response \"%v\"\n", response)
    _, _ = console.ReadLine()
}

Let's server has 127.0.0.1 address (defined in host const in the server app definition). When we start server first and after that - client, then we have the following output on server:

2023/03/22 19:44:56 INFO  [REMOTE]      Starting remote with address address="127.0.0.1:23333" 
2023/03/22 19:44:56 INFO  [REMOTE]      Started Activator
2023/03/22 19:44:56 INFO  [REMOTE]      Started EndpointManager
2023/03/22 19:44:56 INFO  [REMOTE]      Starting Proto.Actor server address="127.0.0.1:23333"  
Server started
Started server with address "127.0.0.1:23333"
2023/03/22 19:45:20 DEBUG [REMOTE]      EndpointReader received connect request message=server_connection:{SystemId:"CxAjoFiLPXHXCXp5yoUPJW"  Address:"127.0.0.1:34444"} 
RequestTypeA with data "IDDQD" from client with address "127.0.0.1:34444" (server address = "127.0.0.1:23333")
2023/03/22 19:45:20 DEBUG [REMOTE]      EndpointSupervisor spawning EndpointWriter and EndpointWatcher address="127.0.0.1:34444"
2023/03/22 19:45:20 INFO  [REMOTE]      Started EndpointWriter. connecting address="127.0.0.1:34444"
2023/03/22 19:45:20 INFO  [REMOTE]      Started EndpointWatcher address="127.0.0.1:34444"
2023/03/22 19:45:20 DEBUG [REMOTE]      Received connect response fromAddress="127.0.0.1:34444"
2023/03/22 19:45:20 INFO  [REMOTE]      EndpointWriter connected address="127.0.0.1:34444" cost=3.5336ms
2023/03/22 19:45:23 INFO  [REMOTE]      EndpointReader failed to read error="rpc error: code = Canceled desc = context canceled" 
2023/03/22 19:45:23 DEBUG [REMOTE]      EndpointReader removed active endpoint from endpointManager 
2023/03/22 19:45:23 ERROR [REMOTE]      EndpointWriter lost connection address="127.0.0.1:34444" error="rpc error: code = Unavailable desc = error reading from server: read tcp 127.0.0.1:58812->127.0.0.1:34444: wsarecv: An existi
ng connection was forcibly closed by the remote host."
2023/03/22 19:45:23 DEBUG [REMOTE]      EndpointManager received endpoint terminated event, removing endpoint message=&{127.0.0.1:34444}
2023/03/22 19:45:23 DEBUG [REMOTE]      Sending EndpointTerminatedEvent to EndpointWatcher ans EndpointWriter address="127.0.0.1:34444"
2023/03/22 19:45:23 DEBUG [REMOTE]      Handling array wrapped terminate event address="127.0.0.1:34444" msg=&{127.0.0.1:34444}
2023/03/22 19:45:23 INFO  [REMOTE]      EndpointWatcher handling terminated address="127.0.0.1:34444" watched=0
2023/03/22 19:45:23 DEBUG [REMOTE]      EndpointWriter stopped address="127.0.0.1:34444"
2023/03/22 19:45:23 INFO  [REMOTE]      EndpointWriter closing client connection address="127.0.0.1:34444"

and on client:

2023/03/22 19:50:57 INFO  [REMOTE]  Starting remote with address address="127.0.0.1:34444" 
2023/03/22 19:50:57 INFO  [REMOTE]  Started Activator 
2023/03/22 19:50:57 INFO  [REMOTE]  Started EndpointManager 
2023/03/22 19:50:57 INFO  [REMOTE]  Starting Proto.Actor server address="127.0.0.1:34444" 
2023/03/22 19:50:57 DEBUG [REMOTE]  EndpointSupervisor spawning EndpointWriter and EndpointWatcher address="127.0.0.1:23333" 
2023/03/22 19:50:57 INFO  [REMOTE]  Started EndpointWriter. connecting address="127.0.0.1:23333" 
2023/03/22 19:50:57 INFO  [REMOTE]  Started EndpointWatcher address="127.0.0.1:23333" 
2023/03/22 19:50:57 DEBUG [REMOTE]  Received connect response fromAddress="127.0.0.1:23333" 
2023/03/22 19:50:57 INFO  [REMOTE]  EndpointWriter connected address="127.0.0.1:23333" cost=2.8485ms 
2023/03/22 19:50:57 DEBUG [REMOTE]  EndpointReader received connect request message=server_connection:{SystemId:"FabFGb96KT8i4gSdog9VX7"  Address:"127.0.0.1:23333"} 
Response "Data:"IDDQD.resultA""

Let's server has 0.0.0.0 address (defined in host const in the server app definition). When we start server first and after that - client, then we have the following output on server:

2023/03/22 19:53:07 INFO  [REMOTE]      Starting remote with address address="[::]:23333"
2023/03/22 19:53:07 INFO  [REMOTE]      Started Activator
2023/03/22 19:53:07 INFO  [REMOTE]      Started EndpointManager
2023/03/22 19:53:07 INFO  [REMOTE]      Starting Proto.Actor server address="[::]:23333"
Server started
Started server with address "[::]:23333"
2023/03/22 19:53:18 DEBUG [REMOTE]      EndpointReader received connect request message=server_connection:{SystemId:"JgBn2wcP3TNW4HwP9Gm9vg"  Address:"127.0.0.1:34444"} 
2023/03/22 19:53:18 DEBUG [REMOTE]      EndpointSupervisor spawning EndpointWriter and EndpointWatcher address="127.0.0.1:23333"
2023/03/22 19:53:18 INFO  [REMOTE]      Started EndpointWatcher address="127.0.0.1:23333"
2023/03/22 19:53:18 INFO  [REMOTE]      Started EndpointWriter. connecting address="127.0.0.1:23333"
2023/03/22 19:53:18 DEBUG [REMOTE]      EndpointReader received connect request message=server_connection:{SystemId:"5UKtdU7DkjnAovWJ65fsvQ"  Address:"[::]:23333"}      
2023/03/22 19:53:18 DEBUG [REMOTE]      Received connect response fromAddress="127.0.0.1:23333"
2023/03/22 19:53:18 INFO  [REMOTE]      EndpointWriter connected address="127.0.0.1:23333" cost=3.2992ms
2023/03/22 19:53:19 INFO  [REMOTE]      EndpointReader failed to read error="rpc error: code = Canceled desc = context canceled" 
2023/03/22 19:53:19 DEBUG [REMOTE]      EndpointReader removed active endpoint from endpointManager

and on client:

2023/03/22 19:54:09 INFO  [REMOTE]  Starting remote with address address="127.0.0.1:34444" 
2023/03/22 19:54:09 INFO  [REMOTE]  Started Activator 
2023/03/22 19:54:09 INFO  [REMOTE]  Started EndpointManager 
2023/03/22 19:54:09 INFO  [REMOTE]  Starting Proto.Actor server address="127.0.0.1:34444" 
2023/03/22 19:54:09 DEBUG [REMOTE]  EndpointSupervisor spawning EndpointWriter and EndpointWatcher address="127.0.0.1:23333" 
2023/03/22 19:54:09 INFO  [REMOTE]  Started EndpointWatcher address="127.0.0.1:23333" 
2023/03/22 19:54:09 INFO  [REMOTE]  Started EndpointWriter. connecting address="127.0.0.1:23333" 
2023/03/22 19:54:09 DEBUG [REMOTE]  Received connect response fromAddress="127.0.0.1:23333" 
2023/03/22 19:54:09 INFO  [REMOTE]  EndpointWriter connected address="127.0.0.1:23333" cost=4.1111ms 
Request error "future: timeout"

Anyone can see the following:

  1. When server has 127.0.0.1 address, interaction between server and client has place: client sends RequestTypeA message on server and receives ResponseTypeA message from server.
  2. When server has 0.0.0.0 address, interaction between server and client doesn't have place: anyone can see the following error on client - Request error "future: timeout"
  3. When server has 127.0.0.1 address, anyone can see only one connect request: 2023/03/22 19:45:20 DEBUG [REMOTE] EndpointReader received connect request message=server_connection:{SystemId:"CxAjoFiLPXHXCXp5yoUPJW" Address:"127.0.0.1:34444"}
  4. When server has 0.0.0.0 address, anyone can see two connect requests: 2023/03/22 19:53:18 DEBUG [REMOTE] EndpointReader received connect request message=server_connection:{SystemId:"JgBn2wcP3TNW4HwP9Gm9vg" Address:"127.0.0.1:34444"} and 2023/03/22 19:53:18 DEBUG [REMOTE] EndpointReader received connect request message=server_connection:{SystemId:"5UKtdU7DkjnAovWJ65fsvQ" Address:"[::]:23333"}

Expected behavior Server must work correctly and equally for both case of addresses - real (e.g., 127.0.0.1) and 0.0.0.0. Moreover, I made the same example on .NET with help of protoactor.NET library and this example work correctly for both case of addresses - real (e.g., 127.0.0.1) and 0.0.0.0.

Screenshots None

Additional context None

rogeralsing commented 1 year ago

For 0.0.0.0, you need to also set an "AdvertisedHost", so that remote systems know what to connect to. as you cannot remotely connect to 0.0.0.0.

stdstring commented 1 year ago

I know about AdvertisedHost property, but i can't use it because my server is situated inside the Docker container (and server doesn't know anything about Docker container address). Moreover, the same example on .NET (whiсh used protoactor.NET library) works correctly (when server has 0.0.0.0 address) without usage of AdvertisedHost property. If you look on server output (when server has 0.0.0.0 address), you can see, that server receives connect request from client (EndpointReader received connect request message=server_connection:{SystemId:"JgBn2wcP3TNW4HwP9Gm9vg" Address:"127.0.0.1:34444"}). But after that server receives another connect request from itself (EndpointReader received connect request message=server_connection:{SystemId:"5UKtdU7DkjnAovWJ65fsvQ" Address:"[::]:23333"}). It seems, that this is reason of this issue.

stdstring commented 1 year ago

@rogeralsing I made example with AdvertisedHost as you advised me:

I have the following definition of server app:

package main

import (
    "fmt"
    console "github.com/asynkron/goconsole"
    "github.com/asynkron/protoactor-go/actor"
    "github.com/asynkron/protoactor-go/remote"
    "os"
    "protoActorRemote/messages"
)

const host = "0.0.0.0"
const port = 23333
const name = "ProtoactorProblemExample"

func main() {
    system := actor.NewActorSystem()
    config := remote.Configure(host, port, remote.WithAdvertisedHost("localhost:23333"))
    remoter := remote.NewRemote(system, config)
    remoter.Start()
    props := actor.PropsFromFunc(func(context actor.Context) {
        switch msg := context.Message().(type) {
        case *actor.Started:
            fmt.Printf("Started server with address \"%v\"\n", context.Self().Address)
        case *actor.Stopping:
            fmt.Printf("Stopping server with address \"%v\"\n", context.Self().Address)
        case *actor.Stopped:
            fmt.Printf("Stopped server with address \"%v\"\n", context.Self().Address)
        case *actor.Restarting:
            fmt.Printf("Restarting server with address \"%v\"\n", context.Self().Address)
        case *messages.RequestTypeA:
            fmt.Printf("RequestTypeA with data \"%v\" from client with address \"%v\" (server address = \"%v\")\n", msg.Data, context.Sender().Address, context.Self().Address)
            context.Send(context.Sender(), &messages.ResponseTypeA{Data: msg.Data + ".resultA"})
        default:
            fmt.Printf("Other message %v\n", msg)
        }
    })
    _, spawnError := system.Root.SpawnNamed(props, name)
    if spawnError != nil {
        fmt.Printf("Spawn error \"%v\"\n", spawnError)
        os.Exit(-1)
    }
    fmt.Println("Server started")
    _, _ = console.ReadLine()
}

When I start server first and after that - client, then I have the following output on server:

2023/03/23 16:44:29 INFO  [REMOTE]      Starting remote with address address="localhost:22333"
2023/03/23 16:44:29 INFO  [REMOTE]      Started Activator
Server started
2023/03/23 16:44:29 INFO  [REMOTE]      Started EndpointManager
2023/03/23 16:44:29 INFO  [REMOTE]      Starting Proto.Actor server address="localhost:22333"
Started server with address "localhost:22333"
2023/03/23 16:44:34 DEBUG [REMOTE]      EndpointReader received connect request message=server_connection:{SystemId:"8Y7ty56Y37qEU5GgQLtUpn"  Address:"127.0.0.1:34443"} 
2023/03/23 16:44:34 DEBUG [REMOTE]      EndpointSupervisor spawning EndpointWriter and EndpointWatcher address="127.0.0.1:22333"
2023/03/23 16:44:34 INFO  [REMOTE]      Started EndpointWriter. connecting address="127.0.0.1:22333"
2023/03/23 16:44:34 INFO  [REMOTE]      Started EndpointWatcher address="127.0.0.1:22333"
2023/03/23 16:44:34 DEBUG [REMOTE]      EndpointReader received connect request message=server_connection:{SystemId:"Q2eebY9vXBwFkhUzS4ddWz"  Address:"localhost:22333"} 
2023/03/23 16:44:34 DEBUG [REMOTE]      Received connect response fromAddress="127.0.0.1:22333"
2023/03/23 16:44:34 INFO  [REMOTE]      EndpointWriter connected address="127.0.0.1:22333" cost=3.7093ms
2023/03/23 16:44:35 INFO  [REMOTE]      EndpointReader failed to read error="rpc error: code = Canceled desc = context canceled" 
2023/03/23 16:44:35 DEBUG [REMOTE]      EndpointReader removed active endpoint from endpointManager

You can see, that i have the similar output with 2 connect requests:

  1. from client: EndpointReader received connect request message=server_connection:{SystemId:"8Y7ty56Y37qEU5GgQLtUpn" Address:"127.0.0.1:34443"}
  2. from server itself: EndpointReader received connect request message=server_connection:{SystemId:"Q2eebY9vXBwFkhUzS4ddWz" Address:"localhost:22333"}

As the result, interaction between client and server doesn't work. So, usage of AdvertisedHost property doesn't help me for this issue.

stdstring commented 1 year ago

@rogeralsing has comments?