improve qisjob speed 😅

omarcostahamido commented 3 years ago

dear @jwoehr, I've finished the speed tests of all the possible combinations of software and externals I've been working with (in combination and each on its own) and realized that the ones that involve qisjob are the slowest. I noticed that even just running qisjob on its own, i can never get execution time bellow 1.3 seconds: time qisjob -a --qasm_simulator -t 2000 onebit.qasm . If you think you could have an insight into why this may be, and that you can find a fix for it, I'd be happy to continue testing with a newer versions. Thank you again for this :)

jwoehr commented 3 years ago

I think it's probably the imports at the head of the file. I've tried moving them to decision branches but the code is pretty gnarly and conditionally expects certain variables to be defined, so I've been unsuccessful this morning in simply moving the imports to branches and testing. Will have to profile sections of the code and test where the problem really is.

omarcostahamido commented 3 years ago

For the record, there is no denying the great advantage potential qisjob offers, in allowing to communicate to different types of backends. But right now I am just interested in qisjob because of its interface, and will using only the IBM machines, in the exported python app. If we could make this simpler (perhaps slightly more limited version in this sense) and more performant, that would be really nice :ok_hand: Thank you!

jwoehr commented 3 years ago

Hmm, well, there's one theory shot to Hades ... I profiled the import stage of the auxiliary libraries and here's the output. Trying and failing to load the extra libraries takes 0.006 seconds!

(qiskit_dev_venv6) jax@varian:~/work/QISKit/qis_job$ qisjob --version
<frozen importlib._bootstrap>:219: UserWarning: QuantumInspire not installed.
<frozen importlib._bootstrap>:219: UserWarning: Rigetti Forest not installed.
<frozen importlib._bootstrap>:219: UserWarning: qiskit-jku-provider not installed.
         4540 function calls (4536 primitive calls) in 0.006 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      357    0.002    0.000    0.002    0.000 {built-in method posix.stat}
      324    0.001    0.000    0.001    0.000 /usr/lib/python3.8/posixpath.py:71(join)
        3    0.000    0.000    0.004    0.001 /usr/lib/python3.8/imp.py:254(find_module)
      324    0.000    0.000    0.002    0.000 /usr/lib/python3.8/genericpath.py:27(isfile)
       33    0.000    0.000    0.001    0.000 <frozen importlib._bootstrap_external>:1431(find_spec)
        3    0.000    0.000    0.004    0.001 /home/jax/work/QISKit/qiskit_dev_venv6/lib/python3.8/site-packages/pyximport/pyximport.py:249(find_module)
      165    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:62(_path_join)
      360    0.000    0.000    0.000    0.000 /usr/lib/python3.8/posixpath.py:41(_get_sep)
      165    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:64(<listcomp>)
      363    0.000    0.000    0.000    0.000 {method 'startswith' of 'str' objects}
       36    0.000    0.000    0.000    0.000 /usr/lib/python3.8/imp.py:105(get_suffixes)
      404    0.000    0.000    0.000    0.000 {built-in method builtins.isinstance}
        3    0.000    0.000    0.006    0.002 <frozen importlib._bootstrap>:890(_find_spec)
        3    0.000    0.000    0.001    0.000 <frozen importlib._bootstrap_external>:1302(_get_spec)
      327    0.000    0.000    0.000    0.000 {method 'endswith' of 'str' objects}
      165    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:222(_verbose_message)
      360    0.000    0.000    0.000    0.000 {built-in method posix.fspath}
      330    0.000    0.000    0.000    0.000 {method 'rstrip' of 'str' objects}
        3    0.000    0.000    0.000    0.000 {method 'write' of '_io.TextIOWrapper' objects}
      5/3    0.000    0.000    0.006    0.002 <frozen importlib._bootstrap>:986(_find_and_load)
       36    0.000    0.000    0.000    0.000 /usr/lib/python3.8/posixpath.py:60(isabs)
      165    0.000    0.000    0.000    0.000 {method 'join' of 'str' objects}
       36    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1265(_path_importer_cache)
       33    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:80(_path_stat)
       36    0.000    0.000    0.000    0.000 /usr/lib/python3.8/imp.py:109(<listcomp>)
        5    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:157(_get_module_lock)
       36    0.000    0.000    0.000    0.000 /usr/lib/python3.8/imp.py:107(<listcomp>)
       15    0.000    0.000    0.004    0.000 <frozen importlib._bootstrap>:881(_find_spec_legacy)
      5/3    0.000    0.000    0.006    0.002 <frozen importlib._bootstrap>:956(_find_and_load_unlocked)
        3    0.000    0.000    0.000    0.000 /usr/lib/python3.8/warnings.py:35(_formatwarnmsg_impl)
        5    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:78(acquire)
        5    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:103(release)
       24    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:867(__exit__)
       35    0.000    0.000    0.000    0.000 {built-in method builtins.hasattr}
       36    0.000    0.000    0.000    0.000 /usr/lib/python3.8/imp.py:108(<listcomp>)
       10    0.000    0.000    0.000    0.000 {built-in method _thread.get_ident}
        6    0.000    0.000    0.000    0.000 {method 'format' of 'str' objects}
       24    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:863(__enter__)
       38    0.000    0.000    0.000    0.000 {method 'rpartition' of 'str' objects}
        5    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:58(__init__)
        3    0.000    0.000    0.000    0.000 /usr/lib/python3.8/warnings.py:20(_showwarnmsg_impl)
       34    0.000    0.000    0.000    0.000 {built-in method _imp.acquire_lock}
        5    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:176(cb)
       34    0.000    0.000    0.000    0.000 {built-in method _imp.release_lock}
        3    0.000    0.000    0.000    0.000 /usr/lib/python3.8/warnings.py:403(__init__)
       33    0.000    0.000    0.000    0.000 {built-in method _stat.S_ISREG}
        5    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:147(__enter__)
        3    0.000    0.000    0.000    0.000 /usr/lib/python3.8/linecache.py:15(getline)
        2    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:1017(_handle_fromlist)
       33    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:40(_relax_case)
        2    0.000    0.000    0.004    0.002 {built-in method builtins.__import__}
        5    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:151(__exit__)
        6    0.000    0.000    0.000    0.000 {built-in method _imp.is_builtin}
        3    0.000    0.000    0.000    0.000 /home/jax/work/QISKit/qiskit_dev_venv6/lib/python3.8/site-packages/pkg_resources/extern/__init__.py:23(find_module)
        3    0.000    0.000    0.001    0.000 <frozen importlib._bootstrap_external>:1334(find_spec)
        3    0.000    0.000    0.000    0.000 /usr/lib/python3.8/warnings.py:96(_showwarnmsg)
        3    0.000    0.000    0.000    0.000 /usr/lib/python3.8/linecache.py:82(updatecache)
        3    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:725(find_spec)
        3    0.000    0.000    0.000    0.000 /usr/lib/python3.8/linecache.py:37(getlines)
        3    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:342(__init__)
        3    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:800(find_spec)
        3    0.000    0.000    0.000    0.000 /usr/lib/python3.8/warnings.py:117(_formatwarnmsg)
       10    0.000    0.000    0.000    0.000 {method 'get' of 'dict' objects}
        2    0.000    0.000    0.004    0.002 <frozen importlib._bootstrap>:211(_call_with_frames_removed)
        5    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:143(__init__)
       10    0.000    0.000    0.000    0.000 {built-in method _thread.allocate_lock}
        6    0.000    0.000    0.000    0.000 {built-in method _imp.is_frozen}
        3    0.000    0.000    0.000    0.000 {method 'split' of 'str' objects}
        3    0.000    0.000    0.000    0.000 /home/jax/work/QISKit/qiskit_dev_venv6/lib/python3.8/site-packages/pkg_resources/_vendor/six.py:184(find_module)
        3    0.000    0.000    0.000    0.000 /home/jax/work/QISKit/qiskit_dev_venv6/lib/python3.8/site-packages/six.py:184(find_module)
        3    0.000    0.000    0.000    0.000 /home/jax/work/QISKit/qiskit_dev_venv6/lib/python3.8/site-packages/urllib3/packages/six.py:180(find_module)
        3    0.000    0.000    0.000    0.000 {method 'partition' of 'str' objects}
        3    0.000    0.000    0.000    0.000 /home/jax/work/QISKit/qiskit_dev_venv6/lib/python3.8/site-packages/pyximport/pyximport.py:71(_debug)
        3    0.000    0.000    0.000    0.000 {built-in method builtins.len}

omarcostahamido commented 3 years ago

hum... curious and curiouser... For the record my simpleton tests where:

(on the terminal python env with qisjob): time qisjob -a --qasm_simulator -t 2000
- I get execution time of around 1.3 seconds

(on the terminal python env with qiskit): enter python mode by typing python and then

>>> import time; from qiskit import *
>>> qc = QuantumCircuit(1,1); qc.h(0); qc.measure(0,0)
>>> backend = Aer.get_backend('qasm_simulator')
>>> start_time = time.time(); print( execute(qc, backend, shots=2000).result().get_counts() ); print("--- %s seconds ---" % (time.time() - start_time));

I get execution time of around 0.007 seconds

jwoehr commented 3 years ago

Yes, well, you're not counting the time it takes to load Python. qisjob loads a compiled object (which probably in turn loads portions of the Python runtime) which takes more time than the actual run, I suspect. I'm pretty sure (near certain) that qisjob doesn't take any longer to run your code. It's just loading qisjob itself and any other dynamic objects that's the difference between using the utility and running a piece of Python code from an already-loading instance.

omarcostahamido commented 3 years ago

that is a good point. However, I just tried that:

creating a qiskit speed.py file with the same code as above and running it: python qiskit\ speed.py
- I get execution time of around 0.350 seconds

edit: ah! I got it wrong. I should have tested: time python qiskit\ speed.py.

now I get execution time of around 1.260 seconds

Dang it. How am I going to get around this corner now?

jwoehr commented 3 years ago

Well, you could rewrite qiskit in straight C. People have thought of doing that.

jwoehr commented 3 years ago

from qis_job import QisJob and use it. new QisJob(arg ...)

omarcostahamido commented 3 years ago

thanks, again, you for this. The speed problem that I am facing seems to be now a very case specific one, that isn't qisjob specifc. You can close this issue if you want.

jwoehr / qisjob

improve qisjob speed 😅 #16