search

coin-or / ADOL-C

A Package for Automatic Differentiation of Algorithms Written in C/C++
Other
146 stars 31 forks source link

ADOL-C thread safety #35

Open chrhansk opened 3 years ago

chrhansk commented 3 years ago

I am working on a project which involves the evaluation of several functions and their derivatives. At each step, the derivatives of several different functions have to be evaluated (rather than several Jacobian rows or Taylor coefficients).

I use ADOL-C to compute these derivatives and am generally pleased with the results. However, I would like to compute all required derivatives in parallel. Unfortunately, it seems that ADOL-C is not by default thread-save. In my case, OpenMP-based parallel evaluation is not an option.

The thread unsafety seems (at least in part) due to the fact that the tape data structures are stored as global variables:

https://github.com/coin-or/ADOL-C/blob/e1fe4761ba54042b37fdff4b03215120ff8671e1/ADOL-C/src/tape_handling.cpp#L337-L348

As far as I can tell, the manual does not mention thread-safety, so this is certainly not a bug. What I would like to have is a guarantee of the form

Evaluation sequences for different tapes are save to execute in parallel

where an evaluation sequence involves a taping step using trace_on(id), ...,trace_off() followed by any gradient(id,...) or more complicated sweeping sequences.

As far as I can tell, most of the actual work is conducted on the current tape and the set of all tapes is only touched very infrequently. So I think making the current tape thread_local and synchronizing the access to the other global variables would go a long way towards thread safety without sacrificing much in terms of performance.

I would be willing to try my hand at implementing this, provided you would merge such a feature.

awalther1 commented 3 years ago

To the best of our knowledge ADOL-C should be thread-safe, however this feature is definitely not widely tested. Therefore I would appreciate your effort if you would fix this and I will definitely merge such a contribution to the master branch. Furthermore, I will forward your message to somebody who is looking into the parallelization of ADOL-C

Best regards

Andrea Walther

mflehmig commented 3 years ago

Hi @chrhansk,

ADOL-C can handle OpenMP parallel primal functions as described in [1] and [2]. For that, an isolation of global and thread-local memory is enforced using OpenMP techniques.

What parallelization API will you use?

Best regards Martin

[1] Parallel derivative computation using ADOL-C, Andreas Kowarz, Andrea Walther, 2008 [2] Section Advanced algorithmic differentiation of OpenMP parallel programs in ADOL-C Manual

cgraeser commented 2 weeks ago

@chrhansk Have a look at #62 which introduced a threadsafe traceless forward mode.