Drive the accepted socket from a separate thread

[pinver]

Is there a way to do that, possibly with std.concurrency?

I've tried using a standard ala C way to setup a listening socket, transferring the socket FD via concurrency.send and instantiating a new std.io.net.socket.Socket(fd) and everything seems to work.

I can't concurrency.send the std.io.net.socket.Socket` as the compiler complains about Aliases to mutable thread-local data

[schveiguy]

In order for that to work, you'd need to have assurances that the original thread cannot mutate the data. This is why std.concurrency is rejecting it.

In order to do it correctly, you should cast it to shared, send it over the connection, then cast it back. It's on you to make sure you don't access it in the original thread (or if you do, that the other thread is no longer accessing it).

[pinver]

Thanks Martin, the problem is that the Socket is not copyable:

~/dlang/dmd-2.092.0/osx/bin/dmd -c -o- -i=. -i=std.io -Isrc -debug src/fieldmanager.d 
 /Users/pinver/dlang/dmd-2.092.0/osx/bin/../../src/phobos/std/concurrency.d(556): Error: struct std.io.net.socket.Socket is not copyable because it is annotated with @disable
/Users/pinver/dlang/dmd-2.092.0/osx/bin/../../src/phobos/std/concurrency.d(464): Error: template instance std.concurrency._spawn!(void function(shared(Socket)), shared(Socket)) error instantiating
src/fieldmanager.d(214):        instantiated from here: spawn!(void function(shared(Socket)), shared(Socket))
/Users/pinver/dlang/dmd-2.092.0/osx/bin/../../src/phobos/std/concurrency.d(464): Error: struct std.io.net.socket.Socket is not copyable because it is annotated with @disable

and I think there's no way to obtain the raw file descriptor, to instantiate a new Socket in the spawned thread

[schveiguy]

Thanks Martin

Steve actually ;) Martin doesn't have too much time to work on this.

It looks like you may have to refCount this to pass it through. This does make sense, because the destructor will close it.

There isn't a way to obtain the raw descriptor, true. But you have to be careful, because if the Socket goes out of scope, it's going to close the descriptor anyway. I'm not sure it makes a lot of sense to allow access to the descriptor because on some drivers (i.e. async that hasn't been written yet), it would be bad to access via the descriptor. Still, it might be useful to have a destructive extraction of the descriptor.

It would be ideal if std.concurrency allowed passing unique data (i.e. noncopyable) through, but for now you probably have to refCount it.

[pinver]

Steve actually ;) Martin doesn't have too much time to work on this.

Sorry Steve :-)

It looks like you may have to refCount this to pass it through. This does make sense, because the destructor will close it.

Using a shared RefCounted works, but I'm unable to cast away the shared once received in the threaded function:

src/fieldmanager.d(404): Error: non-shared inout method std.typecons.RefCounted!(Socket, cast(RefCountedAutoInitialize)0).RefCounted.refCountedPayload is not callable using a shared mutable object
src/fieldmanager.d(404):        Consider adding shared to std.typecons.RefCounted!(Socket, cast(RefCountedAutoInitialize)0).RefCounted.refCountedPayload

There isn't a way to obtain the raw descriptor, true. But you have to be careful, because if the Socket goes out of scope, it's going to close the descriptor anyway. I'm not sure it makes a lot of sense to allow access to the descriptor because on some drivers (i.e. async that hasn't been written yet), it would be bad to access via the descriptor. Still, it might be useful to have a destructive extraction of the descriptor.

Agreed

It would be ideal if std.concurrency allowed passing unique data (i.e. noncopyable) through, but for now you probably have to refCount it.

I'm really in a rush, but still I haven't found a sound way to do that ...

[schveiguy]

Using a shared RefCounted works, but I'm unable to cast away the shared once received in the threaded function

This is what I meant. If it doesn't work, I'll have to look into it (don't have time today):

receive(
  (shared(RefCounted!Socket) sock) {
     auto usableSock = cast() sock; // cast away shared
     // now use usableSock
  });

You don't need to remove the RefCounted wrapper, it should still get destroyed when the last reference is removed.

[pinver]

You are right, that's works. Previously I've tried with:

 auto usableSock = cast(RefCounted!Socket) sock;

src/sidesock.d(22): Error: none of the overloads of refCountedPayload are callable using a shared mutable object, candidates are:
/Users/pinver/dlang/dmd-2.092.0/osx/bin/../../src/phobos/std/typecons.d(6508):        std.typecons.RefCounted!(Socket, cast(RefCountedAutoInitialize)1).RefCounted.refCountedPayload()
/Users/pinver/dlang/dmd-2.092.0/osx/bin/../../src/phobos/std/typecons.d(6500):        std.typecons.RefCounted!(Socket, cast(RefCountedAutoInitialize)1).RefCounted.refCountedPayload()

I'm not sure why ...

UPDATE:

auto uniqueSocket = cast(shared(RefCounted!Socket))(refCounted!Socket(client.move));
spawn(&clientHandler, uniqueSocke);

/Users/pinver/dlang/dmd-2.092.0/osx/bin/../../src/phobos/std/concurrency.d(545): Error: variable `std.concurrency._spawn!(void function(shared(RefCounted!(Socket, cast(RefCountedAutoInitialize)1))), shared(RefCounted!(Socket, cast(RefCountedAutoInitialize)1)))._spawn._param_2` has scoped destruction, cannot build closure

while

auto tid = spawn(&clientHandler);
send(tid, uniqueSocket);

/Users/pinver/dlang/dmd-2.092.0/osx/bin/../../src/phobos/std/variant.d(681): Error: non-shared method `std.typecons.RefCounted!(Socket, cast(RefCountedAutoInitialize)1).RefCounted.__postblit` is not callable using a `shared` object
/Users/pinver/dlang/dmd-2.092.0/osx/bin/../../src/phobos/std/variant.d(681):        Consider adding `shared` to std.typecons.RefCounted!(Socket, cast(RefCountedAutoInitialize)1).RefCounted.__postblit
/Users/pinver/dlang/dmd-2.092.0/osx/bin/../../src/phobos/std/variant.d(578): Error: non-shared method `std.typecons.RefCounted!(Socket, cast(RefCountedAutoInitialize)1).RefCounted.__postblit` is not callable using a `shared` object
/Users/pinver/dlang/dmd-2.092.0/osx/bin/../../src/phobos/std/variant.d(578):        Consider adding `shared` to std.typecons.RefCounted!(Socket, cast(RefCountedAutoInitialize)1).RefCounted.__postblit
/Users/pinver/dlang/dmd-2.092.0/osx/bin/../../src/phobos/std/variant.d(705): Error: template instance `std.variant.VariantN!32LU.VariantN.handler!(shared(RefCounted!(Socket, cast(RefCountedAutoInitialize)1)))` error instantiating
/Users/pinver/dlang/dmd-2.092.0/osx/bin/../../src/phobos/std/variant.d(603):        instantiated from here: `opAssign!(shared(RefCounted!(Socket, cast(RefCountedAutoInitialize)1)))`
/Users/pinver/dlang/dmd-2.092.0/osx/bin/../../src/phobos/std/concurrency.d(126):        instantiated from here: `__ctor!(shared(RefCounted!(Socket, cast(RefCountedAutoInitialize)1)))`
/Users/pinver/dlang/dmd-2.092.0/osx/bin/../../src/phobos/std/concurrency.d(660):        instantiated from here: `__ctor!(shared(RefCounted!(Socket, cast(RefCountedAutoInitialize)1)))`
/Users/pinver/dlang/dmd-2.092.0/osx/bin/../../src/phobos/std/concurrency.d(650):        ... (1 instantiations, -v to show) ...
/Users/pinver/dlang/dmd-2.092.0/osx/bin/../../src/phobos/std/concurrency.d(627):        instantiated from here: `_send!(shared(RefCounted!(Socket, cast(RefCountedAutoInitialize)1)))`
src/sidesock.d(58):        instantiated from here: `send!(shared(RefCounted!(Socket, cast(RefCountedAutoInitialize)1)))`

I'm going to burn the concurrency chapter of my TDPL as a sacrifice to the shared gods ...

[schveiguy]

You are right, that's works. Previously I've tried with:

What cast() does is remove all attributes, but keep the type. I'm thinking possibly what you did was cast to something you thought was the type without attributes, but the type was subtly different (like maybe the AutoInitialize flag was different).

has scoped destruction, cannot build closure

That's a new one I haven't seen. Probably what happens here is that the closure will not call the destructor of the object?

__postblitis not callable using ashared` object

Ugh... so much grief.

Let's try something different. Try wrapping the socket in an IOObject, so it can be easily passed and cast back and forth from shared. Don't forget to close it manually when you are done. This should at least get you by for now. But I need to come up with a good answer to this that doesn't involve allocating a class. I think a big share of the blame falls on std.concurrency, but maybe the community has some better answers.