Errors while setting up radis-benchmark

encrypted-soul commented 3 years ago

While trying to setup benchmarks, several errors are being encountered. The setup is isolated with a separate virtualenv with the latest conda version.

·· Error running /home/gaganaryan/Desktop/Radis/radis-benchmark/env/f329282d0d237249a049116b7f234e30/bin/python setup.py build (exit status 1)
   STDOUT -------->

   STDERR -------->
   Traceback (most recent call last):
     File "setup.py", line 49, in <module>
       import numpy 
   ModuleNotFoundError: No module named 'numpy'

   During handling of the above exception, another exception occurred:

   Traceback (most recent call last):
     File "setup.py", line 56, in <module>
       'matplotlib pandas')
   ImportError: Please install these librairies first (with Anaconda is strongly recommended) 
    >>> conda install numpy scipy matplotlib pandas

I tried uninstalling and then reinstalling all the 4 packages listed in the last line of the error. But this doesn't resolve the issue. The packages are installed (and was verified with conda list).

This error was not faced a few times when I had freshly reinstalled the entire Anaconda. But I faced a similar ModuleNotFoundError for psutil. Facing even now for the commits that do not crash with the first error.

·· Error running /home/gaganaryan/Desktop/Radis/radis-benchmark/env/f329282d0d237249a049116b7f234e30/bin/python /home/gaganaryan/anaconda3/envs/radis/lib/python3.8/site-packages/asv/benchmark.py discover /home/gaganaryan/Desktop/Radis/radis-benchmark/benchmarks /tmp/tmpvtl2_a7z/result.json (exit status 1)
   STDOUT -------->

   STDERR -------->
   Traceback (most recent call last):
     File "/home/gaganaryan/anaconda3/envs/radis/lib/python3.8/site-packages/asv/benchmark.py", line 1315, in <module>
       main()
     File "/home/gaganaryan/anaconda3/envs/radis/lib/python3.8/site-packages/asv/benchmark.py", line 1308, in main
       commands[mode](args)
     File "/home/gaganaryan/anaconda3/envs/radis/lib/python3.8/site-packages/asv/benchmark.py", line 1004, in main_discover
       list_benchmarks(benchmark_dir, fp)
     File "/home/gaganaryan/anaconda3/envs/radis/lib/python3.8/site-packages/asv/benchmark.py", line 989, in list_benchmarks
       for benchmark in disc_benchmarks(root):
     File "/home/gaganaryan/anaconda3/envs/radis/lib/python3.8/site-packages/asv/benchmark.py", line 887, in disc_benchmarks
       for module in disc_modules(root_name, ignore_import_errors=ignore_import_errors):
     File "/home/gaganaryan/anaconda3/envs/radis/lib/python3.8/site-packages/asv/benchmark.py", line 869, in disc_modules
       for item in disc_modules(name, ignore_import_errors=ignore_import_errors):
     File "/home/gaganaryan/anaconda3/envs/radis/lib/python3.8/site-packages/asv/benchmark.py", line 857, in disc_modules
       module = import_module(module_name)
     File "/home/gaganaryan/Desktop/Radis/radis-benchmark/env/f329282d0d237249a049116b7f234e30/lib/python3.6/importlib/__init__.py", line 126, in import_module
       return _bootstrap._gcd_import(name[level:], package, level)
     File "<frozen importlib._bootstrap>", line 994, in _gcd_import
     File "<frozen importlib._bootstrap>", line 971, in _find_and_load
     File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
     File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
     File "<frozen importlib._bootstrap_external>", line 678, in exec_module
     File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
     File "/home/gaganaryan/Desktop/Radis/radis-benchmark/benchmarks/benchmarks.py", line 9, in <module>
       from psutil import virtual_memory
   ModuleNotFoundError: No module named 'psutil'

@anandxkumar is there anything you would like to add to this?

erwanp commented 3 years ago

What are the commands you executed initially to build the Benchmark environment and run the benchmarks? Can you confirm this is a Linux distribution with Python 3.8? What's Asv version?

encrypted-soul commented 3 years ago

Python - 3.8.10 ASV - 0.4.2 Command used - Both asv run and asv run HASHFILE:tested_commit_hash.txt -e

erwanp commented 3 years ago

May have been fixed by 0911cd4568894b65e25d2756793c651a81e2988d

@anandxkumar @gagan-aryan have a try !

anandxkumar commented 3 years ago

Yes @erwanp, It worked! The HITEMP benchmarks were using CDSD_HTEMP databases, and to download those file we had a ftp link - ftp://ftp.iao.ru/pub/CDSD-HITEMP/, so it was a pain figuring it out loading this website! One way I figured out was using FileZilla to download the required CDSD-HITEMP files. Modified the paths and its working now.

One major issue: The benchmarks are taking a lot of time to get executed especially the HITEMP ones. Like every single benchmark within a class is taking around 5 minutes. So to run the whole benchmark for a particular commit is taking around 15-20 minutes which is way too much. For 5 commits it will take around 1.5 hrs which I believe is way too much. Is it normal? It was relatively quicker for 0.9.29 so I believe the previous versions were a lot slower than the recent version (probably due to LDM implementation)?

erwanp commented 3 years ago

Hello ! The test will be easier once https://github.com/radis/radis/pull/281 is done and CO2 HITEMP can be fetched automatically. Side note : CDSD-HITEMP and HITEMP-2010 for CO2 are the same lines; only a slightly different format ; we'll switch to HITEMP-2010 (or 2021 when released!) as it will be fetched automatically.

Performance wise : is the test itself very slow, or the whole benchmark process ?

The benchmark requires to create and install the environment, which takes a significant time. ASV mentions they can cache some build results; it might help (but the RADIS library itself does not have many compiled components at this point!).
some HITEMP tests were indeed quite slow. The CO2 HITEMP equilibrium spectrum on my machine (results published here) took 250s in initial version, 10s with the latest benchmark. That's comparable with what you got !

--

Side note : you can use

asv run develop^!

to run only the latest commit from the develop branch.

As you'll probably quickly get familiar with the ASV setup & commands, you can update the README of this project with tips, or start with the Wiki.

erwanp commented 3 years ago

Can you confirm it's fixed ? Close if so !

encrypted-soul commented 3 years ago

Yes, the issue is fixed. Closing it !!

radis / radis-benchmark

Errors while setting up radis-benchmark #7