IntelPython / sdc

Numba extension for compiling Pandas data frames, Intel® Scalable Dataframe Compiler
https://intelpython.github.io/sdc-doc/
BSD 2-Clause "Simplified" License
645 stars 61 forks source link

SAT-2978 - SDC Series binops corrupt operands if fill_value is used #891

Closed kozlov-alexey closed 3 years ago

kozlov-alexey commented 4 years ago

This PR solves multiple issues in implementation of Series arithmetic and comparison methods:

pep8speaks commented 4 years ago

Hello @kozlov-alexey! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers:

Comment last updated at 2020-07-27 14:09:22 UTC
kozlov-alexey commented 4 years ago
Performance results show that new impl behaves similar to the old one, e.g. for a case of adding two equal size series with none indexes: on master (df4d8d3459be7dff3) name nthreads type size median min max compile boxing
Series.add_series_same_size 1 Python 60000000 0.88369 0.882879 0.891595    
1 SDC 60000000 0.368971 0.368446 0.36941 0.439162 0.000547
2 SDC 60000000 0.273992 0.273375 0.299651 0.4844 0.000718
4 SDC 60000000 0.141599 0.141549 0.142013 0.471689 0.000618
8 SDC 60000000 0.073205 0.073156 0.073496 0.471056 0.000618
16 SDC 60000000 0.040385 0.040329 0.044085 0.473571 0.000611
with #891 name nthreads type size median min max compile boxing
Series.add_series_same_size 1 Python 60000000 1.170385 1.167961 1.171951    
1 SDC 60000000 0.349359 0.349062 0.350519 0.162805 0.000558
2 SDC 60000000 0.233585 0.23304 0.23491 0.232838 0.000659
4 SDC 60000000 0.122814 0.122562 0.122925 0.227218 0.000718
8 SDC 60000000 0.061027 0.060741 0.061105 0.207034 0.000677
16 SDC 60000000 0.032536 0.032402 0.033507 0.206282 0.000671

There's a small degradation in absolute values for series + scalar case, but scalability is not affected. For common case and it's equal indexes sub-case, exec times were improved (previously these test just hang). See https://jira.devtools.intel.com/browse/SAT-2978 for details.