Open v1a0 opened 3 years ago
# function call # total runtime/(records), speedup .4/.3b
crete_table(db) # 0.00332 sec/(1 table), 1x
insert_fats(db) # 0.21690 sec/(10_000 rec), 3x
insert_slow(db) # 0.92790 sec/(10_000 rec), 3x
insert_many_fast(db) # 0.04220 sec/(10_000 rec), 1.06x
insert_many_slow(db) # 0.04269 sec/(10_000 rec), 1.01x
select_all(db) # 0.03394 sec/(10_000 rec), 1.68x
select_where_1(db) # 0.02888 sec/(10_000 rec), 1.01x
select_where_2(db) # 0.02556 sec/(10_000 rec), 1.5x
P.S.: Lister is still a problem, but logger-issue fixed.
# # total runtime/(records) 0.2.0.0-rc3
# # 0.674s
crete_table(db) # 0.00333 sec/(1 table), 1x
insert_fats(db) # 0.20000 sec/(10_000 rec), 1x
insert_slow(db) # 0.69250 sec/(10_000 rec), 1x
insert_many_fast(db) # 0.03473 sec/(10_000 rec), 1x
insert_many_slow(db) # 0.03330 sec/(10_000 rec), 1.1x
select_all(db) # 0.00995 sec/(10_000 rec), 1.1x
select_where_1(db) # 0.00665 sec/(10_000 rec), 1.1x
select_where_2(db) # 0.00699 sec/(10_000 rec), 1.22x
0.2.0.0-rc3, SQLite
# # total runtime/(records) 0.2.0.0-rc3 # # 0.674s crete_table(db) # 0.00333 sec/(1 table), 1x insert_fats(db) # 0.20000 sec/(10_000 rec), 1x insert_slow(db) # 0.69250 sec/(10_000 rec), 1x insert_many_fast(db) # 0.03473 sec/(10_000 rec), 1x insert_many_slow(db) # 0.03330 sec/(10_000 rec), 1.1x select_all(db) # 0.00995 sec/(10_000 rec), 1.1x select_where_1(db) # 0.00665 sec/(10_000 rec), 1.1x select_where_2(db) # 0.00699 sec/(10_000 rec), 1.22x
Same for v0.2.0.0
tests/test_sqllex.py
{test_name}.prof
files in tests
directory after run# @unittest.skip("Turned off manually")
@unittest.skipUnless(importlib.util.find_spec('cProfile'), "Module cProfile not found")
@unittest.skipUnless(importlib.util.find_spec('pstats'), "Module pstats not found")
class TimeTestsSqllexSQLite(unittest.TestCase):
@save_prof
def test_create_table(self):
...
@save_prof
def test_insert_fast(self):
...
...
@save_prof
def test_select_where_2(self):
...
python -m unittest tests.test_sqllex.TimeTestsSqllexSQLite
To visualize result from .prof
files I recommend to use snakeviz
module
python -m pip install snakeviz
python -m snakeviz time_test_create_table.prof
python -m snakeviz <file_name>
Sqllex v0.1.10.3b
Converting
(tuples)
to[lists]
costs for select-like methods 55.8% of all time.This is an results for:
In my opinion it's too much time for such feature. Maybe we should use
numpy.arrays
inside sqllex's processing (instead of lists).Found this example and i think it might be the decision.
https://stackoverflow.com/questions/63409324/why-numpy-array-is-faster-than-list-in-python
P.S.: \<built-in method builtins.isinstance> takes 1/10 of all time
P.P.S:
sqllex v0.1.10.3a
tree/dev/tests/time-tests