search

coin-or / ADOL-C

A Package for Automatic Differentiation of Algorithms Written in C/C++
Other
146 stars 30 forks source link

Heap Buffer Overflow due to reuse of `signatures` Buffer #82

Open michaelvanstraten opened 2 days ago

michaelvanstraten commented 2 days ago

In the trace_on function, we invoke initNewTape https://github.com/coin-or/ADOL-C/blob/d5e3c2a3573ae6c5c92a882c13bc6f1714cae21f/ADOL-C/src/tape_handling.cpp#L800 which checks if the tag ID has been used previously https://github.com/coin-or/ADOL-C/blob/d5e3c2a3573ae6c5c92a882c13bc6f1714cae21f/ADOL-C/src/tape_handling.cpp#L204

If the tag ID is found to be already used and no one is currently using it https://github.com/coin-or/ADOL-C/blob/d5e3c2a3573ae6c5c92a882c13bc6f1714cae21f/ADOL-C/src/tape_handling.cpp#L206 the function calls initTapeInfos_keep https://github.com/coin-or/ADOL-C/blob/d5e3c2a3573ae6c5c92a882c13bc6f1714cae21f/ADOL-C/src/tape_handling.cpp#L224 which in turn reassigns the signatures buffer https://github.com/coin-or/ADOL-C/blob/d5e3c2a3573ae6c5c92a882c13bc6f1714cae21f/ADOL-C/src/tape_handling.cpp#L184

The signatures buffer is initialized during the first call to zos_pl_forward https://github.com/coin-or/ADOL-C/blob/d5e3c2a3573ae6c5c92a882c13bc6f1714cae21f/ADOL-C/src/uni5_for.c#L1102 with the number of switching variables determining its size.

The issue arises when the tag ID is reused, leading to the reuse of the signatures buffer without reallocation. If the number of switching variables increases during this process, it can cause a heap buffer overflow due to insufficient space in the reused buffer.

TimSiebert1 commented 2 days ago

See #81