Scala implementation with Futures

[ochrons]

Here's the benchmark implemented using Scala Future

import scala.concurrent._
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration._

object Root extends App {
  def skynet(num: Int, size: Int, div: Int): Future[Long] = {
    if(size == 1)
      Future.successful(num.toLong)
    else {
      Future.sequence((0 until div).map(n => skynet(num + n*(size/div), size/div, div))).map(_.sum)
    }
  }
  val startTime = System.currentTimeMillis()
  val x = Await.result(skynet(0, 1000000, 10), 10.seconds)
  val diffMs = System.currentTimeMillis() - startTime
  println(s"Result: $x in $diffMs ms.")
}

Runs in 250ms on my i7 laptop. For comparison the Akka version takes about 8200ms and Go takes about 750ms.

[cskr]

Where is the concurrency here? Future.successful returns an already completed future and hence this entire program runs sequentially.

[ochrons]

Only the leaf nodes in the Future tree complete immediately with the value, as they should. If you run the program, it will fully use all available cores to perform the calculations.

[cskr]

Do Future.sequence or map spawn actors? If not, which operation in this code creates them?

I'm don't think just looking at the CPU usage is the right way to determine concurrency/parallelism as it may arise from JVM's activities like GC, rather than from the program itself.

[Daxten]

Actor's are no coroutines. Actor's are used for mutable state in concurrent / distributed applications. Futures are used for concurrent calculations (You can substitute Future.successful(n) with Future { n } if you think this is cheating, but it's equivalent to what you wrote in the go example as far as I can tell)

[ochrons]

Call to the Future map method is performed asynchronously.

[naveensky]

Future in Scala are similar to Async to .NET. Since your .NET code uses async feature, correct way to compare would be against Future implementation

[cskr]

@naveensky I don't think they're the same at all, due to how awaiting works. await in .Net doesn't block the thread, but Await.result in scala does.

[Daxten]

@tuxychandru

Await.result(f)

is the counterpart to

var result = AsyncContext.Run(MyAsyncMethod);

while

var result = await MyAsyncMethod

is the counterpart to

f map { result =>

}

languages work different and have different names for stuff (specially .NET) what you are trying to test here is creating 1000000 threads inside an execution context, which is what Future is (and not actors, additionaly actors are not a core concept of scala and are not part of the core library, or any of the other languages)

[naveensky]

@tuxychandru - await keyword is more or less same as Await in scala. Both wait for Task(.NET) or Future(scala) to complete before moving ahead.

If they are not used, in .NET the return type of calling method is Task while in Scala it is Future[T].

If you use them, in .NET you get a result of enclosing type.

[cskr]

@naveensky Their return type is identical, but their behavior is very different. See @Daxten's reply.

[cskr]

@Daxten Why isn't creating actors the same as creating threads in same execution context?

Also if I understand correctly isn't the way Futures are used here the same as submitting tasks to an ExecutorService in Java?

[Daxten]

An actor is a container for State, Behavior, a Mailbox, Children and a Supervisor Strategy. All of this is encapsulated behind an Actor Reference. Finally, this happens When an Actor Terminates. (see http://doc.akka.io/docs/akka/2.4.1/general/actors.html)

which is different to what a Thread is. the scala benchmark in this example is so slow, because actors are for something completly different then running short computation inside it and then destroying it. In a real world example actors get created and then run for a long period of time

[Daxten]

Also if I understand correctly isn't the way Futures are used here the same as submitting tasks to an ExecutorService in Java?

yes

[cskr]

Is there anyway to introduce a sleep in the leaf Futures, without blocking the underlying thread? That can demonstrate concurrency in the program, if it terminates in few seconds.

For example, I can throw in a time.Sleep(1 * time.Second) in the go version and it terminates within a few seconds. Of course, Thread.sleep blocks the underlying thread making this scala code run out of memory.

[ochrons]

With Future(num.toLong) execution time goes up from 250ms to 390ms.

[ochrons]

To "sleep" inside an async function, you should use a scheduler, for example java.util.concurrent.ScheduledThreadPoolExecutor

[sergey-scherbina]

@tuxychandru

object LearnIt extends App {
  import concurrent._, duration._
  import ExecutionContext.Implicits.global

  Future.sequence(Seq(
    Future.successful(Thread.sleep(1000)))).map {
    _ =>
      println("Is it?")
      System.exit(0)
  }

  while (true) println("See you?")
}

outputs:

See you? See you? See you? Is it? See you? See you? See you? See you? See you? See you? See you? See you?

[sergey-scherbina]

@tuxychandru There is fully sequential version

object SkynetSync extends App {

  def skynet(num: Int, size: Int, div: Int): Long =
    if (size > 1) (0 until div).map(i =>
      skynet(num + i * size / div, size / div, div)).sum
    else num

  def run(n: Int): Long = {
    val start = System.nanoTime()
    val x = skynet(0, 1000000, 10)
    val time = (System.nanoTime() - start) / 1000000
    println(s"$n. Result: $x in $time ms.")
    time
  }

  println(s"Best time ${(0 to 10) map (run) min} ms.")
}

and it works in ~50ms:

sbt (root)> runMain SkynetSync [info] Running SkynetSync

0. Result: 499999500000 in 94 ms.
1. Result: 499999500000 in 42 ms.
2. Result: 499999500000 in 50 ms.
3. Result: 499999500000 in 49 ms.
4. Result: 499999500000 in 46 ms.
5. Result: 499999500000 in 46 ms.
6. Result: 499999500000 in 48 ms.
7. Result: 499999500000 in 47 ms.
8. Result: 499999500000 in 47 ms.
9. Result: 499999500000 in 50 ms.
10. Result: 499999500000 in 48 ms. Best time 42 ms.

[sergey-scherbina]

@tuxychandru Try also this:

object LearnIt extends App {
  import concurrent._, duration._
  import ExecutionContext.Implicits.global

  Future(Thread.sleep(100)).map { _ =>
    println("Is it?")
    System.exit(0)
  }

  while (true) println("See you?")
}

it shows, of course: See you? See you? See you? See you? See you? See you?(... a lot of) See you? Is it?

[sergey-scherbina]

@tuxychandru There is Futures without successfuls

object Skynet extends App {
  import concurrent._, duration._
  import ExecutionContext.Implicits.global

  def skynet(num: Int, size: Int, div: Int): Future[Long] =
    if (size > 1) Future.sequence((0 until div) map (i =>
      skynet(num + i * size / div, size / div, div))).map(_.sum)
    else Future(num)

  def run(n: Int): Long = {
    val start = System.nanoTime()
    val x = Await.result(skynet(0, 1000000, 10), Duration.Inf)
    val time = (System.nanoTime() - start) / 1000000
    println(s"$n. Result: $x in $time ms.")
    time
  }

  println(s"Best time ${(0 to 10) map (run) min} ms.")
}

Results:

[info] Running skynet.SkynetAsync

0. Result: 499999500000 in 406 ms.
1. Result: 499999500000 in 319 ms.
2. Result: 499999500000 in 306 ms.
3. Result: 499999500000 in 313 ms.
4. Result: 499999500000 in 319 ms.
5. Result: 499999500000 in 302 ms.
6. Result: 499999500000 in 291 ms.
7. Result: 499999500000 in 310 ms.
8. Result: 499999500000 in 307 ms.
9. Result: 499999500000 in 308 ms.
10. Result: 499999500000 in 321 ms. Best time 291 ms.

Still as from 2x to 3x faster than Go.

[sergey-scherbina]

@tuxychandru See please http://docs.scala-lang.org/overviews/core/futures.html

[sergey-scherbina]

@tuxychandru Even with Future.sucessful it still works async:

object LearnIt extends App {
  import concurrent._, duration._
  import ExecutionContext.Implicits.global

  Future.successful(Thread.sleep(1000)).map { _ =>
    println("Is it?")
    System.exit(0)
  }

  while (true) println("See you?")
}

shows:

See you? See you? See you? See you? See you? See you? See you? See you? See you? See you? See you? See you? See you? See you? See you? Is it? See you? See you? See you? See you? See you? See you? See you? See you? See you? See you? See you? See you?

[sergey-scherbina]

@tuxychandru You are right that this doesn't work async

  Future.successful {
    Thread.sleep(100)
    println("Is it?")
    System.exit(0)
  }

  while (true)
    println("See you?")

and shows only one "Is it?"

But this still works async

 Future {
    Thread.sleep(1000)
    println("Is it?")
    System.exit(0)
  }

  while (true)
    println("See you?")

and shows a lot of "See you?" before dies.

So with successfuls it is optimized to do not spawn unnecessary asyncs. But for testing purposes you are right there is better to avoid successfuls.