Open alashworth opened 5 years ago
Comment by bob-carpenter Friday Mar 02, 2018 at 20:36 GMT
That's puzzling. What platform are you using? That value is OK for I/O with my 64-bit Clang, and I can't imagine we're reading these in any other way.
int main() {
std::stringstream ss("1.5316e-322");
double x;
ss >> x;
std::cout << "x = " << x << std::endl;
}
Running it gives
$ clang++ foo.cpp
$ ./a.out
x = 1.5316e-322
P.S. This may wind up being a CmdStan issue if the problem's in the outside layer, or a Stan issue if it's in one of the underlying services.
Comment by michael-prange Saturday Mar 03, 2018 at 01:20 GMT
Bob,
I am using an iMac running the latest OS. I will be away from my computer until Monday, but is there a software test you can suggest for me to run at that time? Perhaps I can change some of the stansummary code?
Michael
Sent from my phone
On Mar 2, 2018, at 15:36, Bob Carpenter notifications@github.com wrote:
That's puzzling. What platform are you using? That value is OK for I/O with my 64-bit Clang, and I can't imagine we're reading these in any other way.
int main() { std::stringstream ss("1.5316e-322"); double x; ss >> x; std::cout << "x = " << x << std::endl; } Running it gives
$ clang++ foo.cpp $ ./a.out x = 1.5316e-322 P.S. This may wind up being a CmdStan issue if the problem's in the outside layer, or a Stan issue if it's in one of the underlying services.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.
Comment by bob-carpenter Sunday Mar 04, 2018 at 16:25 GMT
The thing to do is always to take the case that is failing and create a small, reproducible example as a failing unit test.
Then fix the code so that it passes the test.
Stan's a big project with a lot of dev process at this point. If you want to dive in, here's the place to start:
https://github.com/stan-dev/stan/wiki/Developer-process-overview
Comment by michael-prange Monday Mar 05, 2018 at 02:47 GMT
I think I've localized the problem. In line 316 of cmdstan-2.17.1/stan/src/stan/io/stan_csv_reader.hpp is the line
samples(row, col) = boost::lexical_cast
When I made a test program using this line, I get the same error. Test program: // test.cpp
int main() {
double x = boost::lexical_cast
I ran this on an iMac running OS 10.13.3 using the following g++ version: prange% g++ --version Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1 Apple LLVM version 9.0.0 (clang-900.0.39.2) Target: x86_64-apple-darwin17.4.0 Thread model: posix InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
Comment by bob-carpenter Monday Mar 05, 2018 at 04:33 GMT
Thanks for creating that minimial example and tracking down the source.
The problem seems to be that the Boost lexical_cast function can only deal with normal double values; 1e-322 is too small to be represented as normal number---here are the double-precision numerical limits.
The Boost doc only suggests using std::stringstream
for more control---as I showed above, that can handle the subnormal value (at least the clang++ on my Mac). I seem to remember using lexical_cast
to get more control for some reason!
Here's a useful StackOverflow discussion---looks like a lot of this is going to be implementation dependent in the subnormal range, so I'm not sure there's going to be a general fix. But we might be able to go back to stringstream-based parsing.
Comment by michael-prange Monday Mar 05, 2018 at 12:04 GMT
Bob,
Since this issue is holding up a project I’m working on, I’ll switch my local installation to use stringstream instead of lexical_cast as a quick fix. I’ll report back if this produces any subsequent issues. Thanks.
Michael
On Mar 4, 2018, at 11:33 PM, Bob Carpenter notifications@github.com wrote:
The problem seems to be that the Boost lexical_cast function http://www.boost.org/doc/libs/1_66_0/doc/html/boost_lexical_cast/synopsis.html#boost_lexical_cast.synopsis.lexical_cast can only deal with normal double values https://en.wikipedia.org/wiki/Normal_number_(computing); 1e-322 is too small to be represented as normal number---here are the double-precision sizes https://en.wikipedia.org/wiki/Double-precision_floating-point_format#Double-precision_examples.
The Boost doc only suggests using std::stringstream for more control. I seem to remember using lexical_cast to get more control.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/stan-dev/stan/issues/2480#issuecomment-370307233, or mute the thread https://github.com/notifications/unsubscribe-auth/AV4cYvi_yuIpOtBR9MMOlSF8ULcx4T3Nks5tbMAAgaJpZM4SZ3Df.
Comment by bob-carpenter Monday Mar 05, 2018 at 22:09 GMT
Yes, if you can, that should be fine. If it works OK, would you mind submitting a patch for Stan itself? Or sharing your mods so that I can submit the patch?
Comment by michael-prange Monday Mar 05, 2018 at 22:45 GMT
Bob,
I've run 1000's of cases today and the patch has worked flawlessly. I don't know how to submit a patch, so I appreciate you offer to submit it for me. Here it is:
On line 316 of cmdstan-2.17.1/stan/src/stan/io/stan_csv_reader.hpp, replace
samples(row, col) = boost::lexical_cast
I'm sure there are lots of other places where the lexical_cast should be replace to gain more stability, but I've only replaced this one instance in order to fix my problem. I'll leave the larger design issue to the stan-dev team. Thanks for all your help. You really dug me out of a hole.
Michael
Issue by michael-prange Friday Mar 02, 2018 at 13:01 GMT Originally opened as https://github.com/stan-dev/stan/issues/2480
Summary:
stansummary crashes when run on a csv file that contains 1.5316e-322 in the accept_stat__ column of the csv file.
Description:
[Copied from comment below] The problem seems to be that the Boost lexical_cast function can only deal with normal double values; 1.6e-322 is too small to be represented as normal number---here are the double-precision numerical limits.
The Boost doc only suggests using
std::stringstream
for more control.Reproducible Steps:
My cmdstan code runs without warnings or errors, but when I run stansummary on the resulting csv file I get the error:
In order to trace the source of the error, I proceeded to delete sample-output lines from the csv file until I found the following offending line:
Run stansummary of the following csv file. I edited it down to the offending sample-output line:
I added a second sampling-output line below the offending line so stansummary will work when 1.5316e-322 is changed to 1.5316e-3
Current Output:
Expected Output:
When the offending number in the sampling output is changed from 1.5316e-322 to 1.5316e-3, the output is
Current Version:
v2.17.1