atomist-attic / artifact-source

ArtifactSource abstraction for source code
GNU General Public License v3.0
2 stars 3 forks source link

Use Scala concurrency to improve performance #28

Closed alankstewart closed 7 years ago

cdupuis commented 7 years ago

How many threads is this creating? What happens if two archives get unpacked at the same time?

I think what this boils down to: What is the behaviour in a concurrent environment when calling Await.result(). What is the underlying thread pool?

alankstewart commented 7 years ago

According to the Scala docs (http://docs.scala-lang.org/overviews/core/futures.html),

the ExecutionContext.global sets the parallelism level of its underlying fork-join pool to the amount of available processors.

I experimented with varying thread pool sizes but using the default seemed to give the best performance over multiple runs. I am still writing more tests and tweaking the code. I've set up a test which unpacks 3 archives at the same time. No issues

ddgenome commented 7 years ago

Why was the timeout decreased to 15 s? Have you tested with archives having lots of files and large size?

alankstewart commented 7 years ago

There was concern by Christian that four seconds was too long for an archive