Logic benchmarks fail for older commits.

moorepants commented 9 years ago

These are failing on an import from the compatibility module. Maybe you want to check this out for older versions.

[ 17.71%] · For sympy commit hash 232be474:
[ 17.71%] ·· Building for py2.7-fastcache-mpmath....
[ 17.71%] ·· Benchmarking py2.7-fastcache-mpmath
[ 17.72%] ··· Running large_exprs.TimeLargeExpressionOperations.time_cse                                                                                                                                    100.95ms
[ 17.73%] ··· Running logic.LogicSuite.time_dpll                                                                                                                                                              failed
[ 17.73%] ····· Traceback (most recent call last):
                  File "/home/moorepants/miniconda/lib/python2.7/site-packages/asv-0.1.1-py2.7.egg/asv/benchmark.py", line 677, in <module>
                    benchmark_dir, benchmark_id, quick=quick)
                  File "/home/moorepants/miniconda/lib/python2.7/site-packages/asv-0.1.1-py2.7.egg/asv/benchmark.py", line 382, in from_name
                    root, parts, os.path.basename(root))
                  File "/home/moorepants/miniconda/lib/python2.7/site-packages/asv-0.1.1-py2.7.egg/asv/benchmark.py", line 355, in find_on_filesystem
                    module = import_module(new_package)
                  File "/home/moorepants/src/sympy_benchmarks_bjodah/env/f5b82df50164b5d73101b16d6e2f197f/lib/python2.7/importlib/__init__.py", line 37, in import_module
                    __import__(name)
                  File "benchmarks/logic.py", line 3, in <module>
                    from sympy.core.compatibility import range
                ImportError: cannot import name range

[ 17.74%] ··· Running logic.LogicSuite.time_dpll2                                                                                                                                                             failed
[ 17.74%] ····· Traceback (most recent call last):
                  File "/home/moorepants/miniconda/lib/python2.7/site-packages/asv-0.1.1-py2.7.egg/asv/benchmark.py", line 677, in <module>
                    benchmark_dir, benchmark_id, quick=quick)
                  File "/home/moorepants/miniconda/lib/python2.7/site-packages/asv-0.1.1-py2.7.egg/asv/benchmark.py", line 382, in from_name
                    root, parts, os.path.basename(root))
                  File "/home/moorepants/miniconda/lib/python2.7/site-packages/asv-0.1.1-py2.7.egg/asv/benchmark.py", line 355, in find_on_filesystem
                    module = import_module(new_package)
                  File "/home/moorepants/src/sympy_benchmarks_bjodah/env/f5b82df50164b5d73101b16d6e2f197f/lib/python2.7/importlib/__init__.py", line 37, in import_module
                    __import__(name)
                  File "benchmarks/logic.py", line 3, in <module>
                    from sympy.core.compatibility import range
                ImportError: cannot import name range

[ 17.75%] ··· Running logic.LogicSuite.time_load_file                                                                                                                                                         failed
[ 17.75%] ····· Traceback (most recent call last):
                  File "/home/moorepants/miniconda/lib/python2.7/site-packages/asv-0.1.1-py2.7.egg/asv/benchmark.py", line 677, in <module>
                    benchmark_dir, benchmark_id, quick=quick)
                  File "/home/moorepants/miniconda/lib/python2.7/site-packages/asv-0.1.1-py2.7.egg/asv/benchmark.py", line 382, in from_name
                    root, parts, os.path.basename(root))
                  File "/home/moorepants/miniconda/lib/python2.7/site-packages/asv-0.1.1-py2.7.egg/asv/benchmark.py", line 355, in find_on_filesystem
                    module = import_module(new_package)
                  File "/home/moorepants/src/sympy_benchmarks_bjodah/env/f5b82df50164b5d73101b16d6e2f197f/lib/python2.7/importlib/__init__.py", line 37, in import_module
                    __import__(name)
                  File "benchmarks/logic.py", line 3, in <module>
                    from sympy.core.compatibility import range
                ImportError: cannot import name range

bjodah commented 9 years ago

Hmmm.. yes, maybe there is a way to indicate that a benchmark is only relevant after a certain commit? @debugger22 you wrote the logic benchmarks, what do you think?

moorepants commented 9 years ago

I'm not sure the best strategy for these as it is difficult to write a benchmark that can span the whole history.

sympy / sympy_benchmarks

Logic benchmarks fail for older commits. #22