search

stan-dev / math

The Stan Math Library is a C++ template library for automatic differentiation of any order using forward, reverse, and mixed modes. It includes a range of built-in functions for probabilistic modeling, linear algebra, and equation solving.
https://mc-stan.org
BSD 3-Clause "New" or "Revised" License
751 stars 188 forks source link

Stan performance tests fail when setting macro NDEBUG #1939

Open SteveBronder opened 4 years ago

SteveBronder commented 4 years ago

Description

Scrounging for optimizations wherever I can I noticed we did not set the NDEBUG macro flag which can have some nice teny benefits. However setting the NDEBUG macro at compile time causes the SIR test to fail. If we don't support NDEBUG being defined we should catch it and throw an error message. Otherwise we should figure out how that happens. @wds15 have you ever seen test failures for that before?

Example

git clone --recursive https://github.com/stan-dev/performance-tests-cmdstan
cd performance-tests-cmdstan

replace compare-git-hashes.py with the version here that changes the compiler flags on the second iteration.

./compare-git-hashes.sh "stat_comp_benchmarks/benchmarks/sir/ -j32 --runs 1" develop develop develop develop

Failure's are here everything is off by a good bit.

Expected Output

Tests pass

Current Version:

v3.2.0

wds15 commented 4 years ago

This is new to me and should be resolved.

syclik commented 3 years ago

@SteveBronder, I just ran it and it didn't reproduce.

Mind making it a little simpler for me to follow along? Can you commit the changes on a branch and I'll see if I can reproduce it locally?

SteveBronder commented 3 years ago

I'll retry this tmrw. Would be great if this went away magically. Dan what compiler and cpu are you using?

syclik commented 3 years ago

I'm on a MacBook Pro: $ clang++ --version Apple clang version 12.0.0 (clang-1200.0.26.2) Target: x86_64-apple-darwin19.6.0 Thread model: posix InstalledDir: /Library/Developer/CommandLineTools/usr/bin

2.9 GHz Quad-Core Intel Core i7

On Wed, Nov 25, 2020 at 12:45 AM Steve Bronder notifications@github.com wrote:

I'll retry this tmrw. Would be great if this went away magically. Dan what compiler and cpu are you using?

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/stan-dev/math/issues/1939#issuecomment-733480374, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADH6F5JXTS2BFA6CEYRPDDSRSKXTANCNFSM4N5UBEOA .

syclik commented 3 years ago

@SteveBronder, any updates?