search

alashworth / test-issue-import

0 stars 0 forks source link

O_STANC and file size with C++11 #111

Open alashworth opened 5 years ago

alashworth commented 5 years ago

Issue by bgoodri Tuesday Aug 23, 2016 at 06:05 GMT Originally opened as https://github.com/stan-dev/stan/issues/2025

Summary:

The binary size of stanc is too damn high with C++11

Description:

Binary sizes are currently unmanageable if using C++11 only but are manageable if we change O_STANC from 0 to 2.

Reproducible Steps:

In cmdstan, if I currently do

make clean-all
make examples/bernoulli/bernoulli

with CC=g++-6 -std=c++11, the sizes on disk are

This is with g++ 6.2, I think g++ <= 6.1 was worse.

In cmdstan, if I do

make clean-all
make examples/bernoulli/bernoulli

with O_STANC=2 in make/local, the sizes on disk are

which is manageable.

IIRC, we set O_STANC to 0 by default, because mingw's g++-4.6 for Windows had difficulty compiling stanc with conventional amounts of RAM. So, we conserved RAM and sacrificed disk space by compiling stanc without optimization. Nowdays, almost no one compiles rstan from source on Windows and those that do probably have a lot of RAM and are using mingw's g++-4.9 which comes with RTools33 and higher. So, we can probably get set O_STANC to 3 by default now.

Current Output:

Expected Output:

Additional Information:

Current Version:

v2.11.0

alashworth commented 5 years ago

Comment by ariddell Tuesday Aug 23, 2016 at 11:37 GMT

For reference, clang had trouble with stanc when using high optimization levels, https://github.com/stan-dev/pystan/pull/33

alashworth commented 5 years ago

Comment by bob-carpenter Tuesday Aug 23, 2016 at 12:09 GMT

No idea how to address this problem. Almost all the complexity is buried inside the Spirit Qi parser from Boost. My only thought is to start using trial and error to rewrite grammar rules with less nesting. Given there's no milestone for this, is this an issue that's blocking anything right now?

alashworth commented 5 years ago

Comment by ariddell Tuesday Aug 23, 2016 at 12:24 GMT

Binary size is a distribution issue for pystan. Do we know if -O2 breaks clang or if it's just -O3?

alashworth commented 5 years ago

Comment by bob-carpenter Tuesday Aug 23, 2016 at 14:29 GMT

Does this literally not work with clang C++11 or just create a huge executable?

alashworth commented 5 years ago

Comment by bgoodri Tuesday Aug 23, 2016 at 14:44 GMT

What is being blocked is being able to use / require C++11 generally due to the excessive binary size. But I think we are okay as far as R is concerned because R uses -O2 to build stanc in the rstan shared object. FWIW, cmdstan would be okay under C++11 if O_STANC were something other than 0. I defer to @ariddell on Python.

alashworth commented 5 years ago

Comment by syclik Tuesday Aug 23, 2016 at 15:07 GMT

I thought we weren't ready to go to C++11 yet. Did I mistreatment from our meeting?

On Aug 23, 2016, at 6:44 AM, bgoodri notifications@github.com wrote:

What is being blocked is being able to use / require C++11 generally due to the excessive binary size. But I think we are okay as far as R is concerned because R uses -O2 to build stanc in the rstan shared object. FWIW, cmdstan would be okay under C++11 if O_STANC were something other than 0. I defer to @ariddell on Python.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

alashworth commented 5 years ago

Comment by bob-carpenter Tuesday Aug 23, 2016 at 15:33 GMT

What's wrong with CmdStan with -O0?

alashworth commented 5 years ago

Comment by bgoodri Tuesday Aug 23, 2016 at 15:58 GMT

Consumes almost 200M. It is just that no one enforces size restrictions on cmdstan like they do for rstan or pystan.

On Aug 23, 2016 11:33 AM, "Bob Carpenter" notifications@github.com wrote:

What's wrong with CmdStan with -O0?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/stan-dev/stan/issues/2025#issuecomment-241772706, or mute the thread https://github.com/notifications/unsubscribe-auth/ADOrqijAYGG6tkttb4f089beZECJOTM6ks5qixLAgaJpZM4JqlRw .

alashworth commented 5 years ago

Comment by bob-carpenter Wednesday Aug 24, 2016 at 11:22 GMT

OK, and given the memory headroom I have, and that compiling stanc takes many seconds, I don't even notice the memory hit.

alashworth commented 5 years ago

Comment by ariddell Friday Sep 09, 2016 at 20:23 GMT

So the change from -O3 to -O0 was motivated by clang++ compatibility issues related to the parser. I don't know if anyone ever tested -O2. What I don't understand is why CI tests didn't detect that -O3 was a problem. Surely we were testing on OS X at that point.

alashworth commented 5 years ago

Comment by ariddell Friday Sep 09, 2016 at 20:26 GMT

In stan-users, https://groups.google.com/d/msg/stan-users/HP3YjPrBKO0/j5JPDY4JrMMJ @goedman offered a fix for this issue that involved C++11. Now that we're in that brave new world, perhaps we could use the fix?

alashworth commented 5 years ago

Comment by ariddell Friday Sep 09, 2016 at 20:35 GMT

@kcarnold on stan-dev/pystan#33 suggests trying -Os which is like -O2 but optimizes for binary size. It's supported by gcc and clang. (thanks @kcarnold!)