Endless loop?

[cdrage]

What would be the best way / practical way to have an endless loop of the pool? From the current looks of it / examples, the pool will automatically close when there are no new jobs.

I know that you can simply add:

for {
}

To the end since the pools are running as goroutines.

But shouldn't this behavior be by default? Or maybe I'm just too used to using Resque haha.

[albrow]

I'm sorry if this is not currently clear in the README. Once you start a pool by calling pool.Start, it will continue executing jobs ad infinitum. If there are no jobs currently available, it will keep polling until at least one job is available. The only reason a pool would stop is if there was some unexpected error or you manually call pool.Stop. Does that clear things up?

[albrow]

@cdrage the main worker loop is surprisingly concise and may be worth checking out.

// start starts a goroutine in which the worker will continuously
// execute jobs until the jobs channel is closed.
func (w *worker) start() {
    go func() {
        for job := range w.jobs {
            w.doJob(job)
        }
        w.wg.Done()
    }()
}

Here, w.jobs is a channel of jobs. You may already know this, but in go, when you apply range across a channel, it will keep receiving from the channel until the channel is closed. So the for loop will never exit unless the channel closes. The channel closes when you call pool.Stop or when there is an unexpected error.

[cdrage]

From your comments I added pool.Start and it doesn't seem to work.

package main                                                                                                                                                                                                                                                   

import (                                                                                                                                                                                                                                                       
  "fmt"                                                                                                                                                                                                                                                        
  "time"                                                                                                                                                                                                                                                       

  "github.com/albrow/jobs"                                                                                                                                                                                                                                     
)                                                                                                                                                                                                                                                              

var (                                                                                                                                                                                                                                                          
  sche *jobs.Type                                                                                                                                                                                                                                              
  // I made pool a top-level variable so that was can access it inside of the main func.                                                                                                                                                                       
  pool *jobs.Pool                                                                                                                                                                                                                                              
)                                                                                                                                                                                                                                                              

func initJob() (err error) {                                                                                                                                                                                                                                   
  sche, err = jobs.RegisterType("DEFAULT", 3, func(s string) (err error) {                                                                                                                                                                                     
    fmt.Println("exec at ", time.Now(), s)                                                                                                                                                                                                                     
    return                                                                                                                                                                                                                                                     
  })                                                                                                                                                                                                                                                           
  if err != nil {                                                                                                                                                                                                                                              
    fmt.Println(err)                                                                                                                                                                                                                                           
    return                                                                                                                                                                                                                                                     
  }                                                                                                                                                                                                                                                            
  return                                                                                                                                                                                                                                                       
}                                                                                                                                                                                                                                                              

func main() {                                                                                                                                                                                                                                                  
  err := initJob()                                                                                                                                                                                                                                             
  if err != nil {                                                                                                                                                                                                                                              
    fmt.Println(err)                                                                                                                                                                                                                                           
    return                                                                                                                                                                                                                                                     
  }                                                                                                                                                                                                                                                            
  pool, err := jobs.NewPool(nil)                                                                                                                                                                                                                               
  if err != nil {                                                                                                                                                                                                                                              
    // Handle err                                                                                                                                                                                                                                              
  }                                                                                                                                                                                                                                                            
  defer func() {                                                                                                                                                                                                                                               
    pool.Close()                                                                                                                                                                                                                                               
    if err := pool.Wait(); err != nil {                                                                                                                                                                                                                        
      // Handle err                                                                                                                                                                                                                                            
    }                                                                                                                                                                                                                                                          
  }()                                                                                                                                                                                                                                                          
  if err := pool.Start(); err != nil {                                                                                                                                                                                                                         
    // Handle err                                                                                                                                                                                                                                              
  }                                                                                                                                                                                                                                                            
}                                                                                                                                                                                                                                                              ```

Causes it to exit rather than stay active / online. Am I doing something wrong? Perhaps define the number of workers or something?

[albrow]

@cdrage Ahhh, I see what's going on.

pool.Start is non-blocking, so even though it continues to do stuff in the background, it returns immediately. Since pool.Start is the last statement in your main function, it means the main function reaches its end and calls anything that was deferred. In the deferred function, the pool is closed. So the effect is that the main function exits and the pool is closed without doing anything.

To fix this, you can simply prevent the main function from exiting. If you want to wait a certain amount of time before exiting, you can use time.Sleep. If you want the main function to never exit (unless you send SIGTERM or SIGKILL, of course) you can receive from a channel which you never send anything through. This would be the idiomatic way to wait forever in go:

func main() {
  // At the very end of the main function
  waitForever := make(chan struct{})
  <-waitForever
}

I'm sorry for the confusion. I think the example in the README could be improved.

[albrow]

Also, you generally should not do something like this in go:

for {}

It would have the same effect of waiting forever, but it would be a busy loop. The CPU would still be doing things and wasting cycles. If you use a channel instead, it's more efficient.

Apparently select {} would work too instead of a channel. Since there are no conditions inside the select statement, it will just block forever. See http://golang-examples.tumblr.com/post/43110284482/sleep-forever.

[cdrage]

Not a problem! Glad that I could get this clarified! I'll go ahead and make it idiomatic with your changes.

I've got to say, you're quite detailed in your responses and me being a Go newbie it's quite informative! :) so many thanks!