Documentation and other feedback

[mulle-kybernetik-tv]

Some feedback on my first use of libdill, so these are all newbie issues 😌 :

It is not well explained how to use the int[2] channel you get from chmake

The examples don't help much and sometimes are even wrong like on chdone:

int ch[2];
chmake(ch);
chsend(ch, "ABC", 3, -1); // should be ch[ 1]
chdone(ch);                     // likely also ch[ 1] 

There is no plain "wait" methods for unbundled gos

It's a bit strange, that there is no documented way to run a few coroutines and wait for them to finish. At least I couldn't discern that from the documentation. I had to use bundle_go to be able to use bundle_wait.

It's not really clear if I can hclose a channel

Since both are handles it sounds like it would be OK. It seemed chdone( ch[1]) is what I want, but it isn't really explained what I gain/lose there.

Compile problems

It doesn't compile with -std=c99 because of the missing asm keyword. It did compile when I changed those to __asm. -std=gnu99 would work as well. Also Linux needs _POSIX_SOURCE defined to have sigjmp_buf defined, also fixed by -std=gnu99.

Why do coroutines need file descriptors ?

From my observations I could work with ~30000 coroutines, and then things went southwards. I attribute that to max file descriptors. I don't understand though, why a coroutine needs a file descriptor and that would have been nice to read in the documentation.

[sustrik]

That's useful. Thanks!

There is no plain "wait" methods for unbundled gos: unbundled go actually creates a bundle with a single coroutine inside. You can thus wait on the returned fd.

It's not really clear if I can hclose a channel: chdone basically writes EOF into the channel. hclose() deallocates it. This may be redundant. Something to think about in newer versions of libdill.

Why do coroutines need file descriptors: It's actually bundle descriptors. Anyway, what exactly do you mean by things going south? Running out of memory?

[mulle-kybernetik-tv]

wait: if there was a default bundle handle '0', you could bundle all unbundled gos together under them and then implement #define wait( x) bundle_wait( 0,x). As a user I'd like that more.

hclose: so would/should I hclose after chdone ?

descriptors: I looked into the chan.c sourcecode, and I didn't see any system calls off hand, so I kind of assumed running my tests that the problem is with the coroutines, which I didn't look into.

You can look at my test code, which has some comments. It basically creates a channel for each coroutine and then read/writes to it a few times. It hit the exact limit of 32693 coroutines, then bundle_go would fail.

One question I would like to ask is, can multiple coroutines wait on the same channel ? I plan to use coroutines for animation purposes, and I would like to wake them all up on screen refresh.

[sustrik]

wait: I prefer to be C-idiomatic, i.e. user has to handle cleanup manually.

hclose: Yes, you shoud.

I'll check the test code later on.

Yes, multiple coroutines can wait on a single channel. They are then served in the order they've started waiting in.

[mulle-kybernetik-tv]

wait: but that's what wait would do, clean up ? Now you can't really clean up because of the hidden bundle-per-coroutine magic. The only option is msleep, but the running time might be just guess-work.

[sustrik]

I don't follow. No, wait does not clean up, hclose does. Wait just waits till everybody's finished. When I was speaking about manual cleanup it meant the C-idiomatic way of doing it in pairs (malloc/free, socket/close etc.) Having a single file descriptor 0 that would contain all coroutines violates the principle.

[mulle-kybernetik-tv]

Its just my first days in libdill, so the chances of me overlooking something are large. Lets simplify my problem by contrasting libdill with pthreads

pthread_create( &thread, NULL, whatever); // create a thread running whatever
pthread_join( thread); // wait for completion

I know what to do for bundle_go, but what do I do for plain go ?

h = go( whatever);
wait( h);  // doesn't exist ? hclose would kill go, wouldn't it

Nevertheless having all unbundled coroutines part of a default bundle 0, would be like a unix process group to me, so not that unfamiliar.

[sustrik]

Ah, I see you problem. go() is just a shortcut for bundle()+bundle_go() so you can do this:

h = go(fn());
bundle_wait(h);

[mulle-kybernetik-tv]

OK now I understand it. If the go() documentation told you that go() is just a shortcut for bundle()+bundle_go() and otherwise refer to those pages, that would probably be better (at least for me). Now it appears as if go and bundle_go are two seperate concepts really.

[sustrik]

Fair enough. I'll add that wording.

[mulle-nat]

Coming back to this after a pause...

Yes, multiple coroutines can wait on a single channel. They are then served in the order they've started waiting in.

It turns out that the message is consumed by the first reader, so the next reader doesn't get it. A subsequent message is then consumed by the next reader. But this means the sender would have to know how many receivers there are and then then send that many messages to reach them all, which for my use case is too cumbersome/slow.

I am looking for something broadcast style, so that each reader gets the message. Actually the message itself isn't really that important to me, but the "waking up" part is.

[sustrik]

Broadcast is out of scope for libdill. You can build it on top though.

[mulle-nat]

The documentation for hquery mentions a second type parameter, but it is otherwise absent from synopsis and description.

[mulle-nat]

And another question does a yield after chsend guarantee a context switch ? In my tests a chsend alone does not necessarily context switch. If my assumption, that a yield will guarantee this though, is incorrect, then I am headed towards a dead-end...

[sustrik]

Why do you care about when exactly the context switch happens? Isn't it enough to know that it will happen eventually?

[mulle-nat]

As I wrote, I'd like to use coroutines for animation purposes.

Doesn't the word "eventually" and the currently observed behaviour requiring a yield for a deterministic switch suggest, that libdill can get stuck or laggy in a producer/consumer scenario without it ?

[sustrik]

Depends on what your coroutines are doing. If they are doing blocking operations such as reading writing to socket/channel or sleeping, then the context switching will happen transparently within those calls. If you are doing busy computation with no I/O involved then you should call yield() once in a while to give scheduler a chance to act.

[mulle-nat]

It had been my observation though, that in a simple producer/consumer scenario a chsend does not necessarily immediately wake up the consumer that is blocked in a chread. That only happened if I explicitly yielded.

Here is an example:

coroutine void   print( int ch)
{
   for(;;)
   {
      fprintf( stderr, "r");
      if( chrecv( ch, NULL, 0, -1) == -1)
      {
         switch( errno)
         {
         case EPIPE     :
         case ECANCELED : return;
         default        : perror("chrecv"); abort();
         }
      }

      fprintf( stderr, "*");
   }
}

int    main( int argc, char *argv[])
{
   int   ch[ 2];
   int   i;

   chmake( ch);

   go( print( ch[ 0]));

   for( i = 0; i < 6; i++)
   {
      fprintf( stderr, "s");
      chsend( ch[ 1], NULL, 0, -1);

      // yield();       // needed to get chsend to reliably send ?
   }
   fprintf( stderr, "c");
   chdone( ch[ 1]);
}

This prints rss*r*rss*r*rss*r*rc. So there actually two sends being done before the consumer is woken up twice. When I comment in the yield line it prints rs*rs*rs*rs*rs*rs*rc, which is the reliable back and forth I am looking for. "Eventually" isn't what I need here.

[sustrik]

The documentation for hquery mentions a second type parameter, but it is otherwise absent from synopsis and description.

Fixed in the head. Thanks for spotting that.