search

ParaToolsInc / taucmdr

Performance engineering for the rest of us.
http://www.taucommander.com
Other
29 stars 11 forks source link

SQLite database backend and SQLite profile format #380

Closed nchaimov closed 3 years ago

nchaimov commented 3 years ago

This adds support for using SQLite as the database instead of TinyDB. TinyDB is still used as the default. When initializing a new project, the argument --backend can be given to use SQLite instead. For example,

tau init --backend sqlite

will create a new project with a SQLite database to store project data.

When operating on an existing project, SQLite will be used if a SQLite database exists in the .tau directory; otherwise, TinyDB will be used.

Additionally, SQLite can be used as the format for TAU profiles. This is done by setting the attribute profile of a Measurement to sqlite. If the SQLite profile format is used, TAU is built with SQLite support and the SQLite plugin is enabled at runtime by setting the TAU_PLUGINS environment variable.

codecov[bot] commented 3 years ago

Codecov Report

Merging #380 into unstable will decrease coverage by 0.13%. The diff coverage is 64.77%.

Impacted file tree graph

@@             Coverage Diff              @@
##           unstable     #380      +/-   ##
============================================
- Coverage     68.39%   68.25%   -0.14%     
============================================
  Files            92       96       +4     
  Lines          7995     8552     +557     
  Branches       1374     1465      +91     
============================================
+ Hits           5468     5837     +369     
- Misses         2030     2180     +150     
- Partials        497      535      +38
Flag Coverage Δ
#CI 68.25% <64.77%> (-0.14%) :arrow_down:
#unittests 68.25% <64.77%> (-0.14%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
packages/taucmdr/__init__.py 57.57% <ø> (ø)
packages/taucmdr/cf/software/installation.py 63.34% <0.00%> (+0.46%) :arrow_up:
packages/taucmdr/cf/software/ompt_installation.py 57.14% <0.00%> (-6.02%) :arrow_down:
packages/taucmdr/cf/storage/__init__.py 50.00% <0.00%> (-0.88%) :arrow_down:
packages/taucmdr/cf/storage/project.py 74.02% <ø> (ø)
packages/taucmdr/model/measurement.py 73.46% <ø> (ø)
packages/taucmdr/model/target.py 60.97% <ø> (ø)
packages/taucmdr/mvc/controller.py 80.61% <ø> (-0.09%) :arrow_down:
packages/taucmdr/cf/storage/sqlite3_project.py 43.66% <43.66%> (ø)
packages/taucmdr/cf/storage/local_file.py 65.23% <50.00%> (+1.07%) :arrow_up:
... and 17 more
github-actions[bot] commented 3 years ago

Pylint Output

Report

8660 statements analysed.

Statistics by type

type number old number difference %documented %badname
module 97 93 +4.00 100.00 0.00
class 126 116 +10.00 99.21 0.79
method 689 591 +98.00 98.69 0.87
function 117 116 +1.00 94.87 0.85

Raw metrics

type number % previous difference
code 10417 51.10 9804 +613.00
docstring 5318 26.09 4817 +501.00
comment 2980 14.62 2841 +139.00
empty 1669 8.19 1526 +143.00

Duplication

now previous difference
nb duplicated lines 0 0 =
percent duplicated lines 0.000 0.000 =

Messages by category

type number previous difference
convention 26 20 +6.00
refactor 16 15 +1.00
warning 25 20 +5.00
error 0 0 =

% errors / warnings by module

module error warning refactor convention
taucmdr.model.experiment 0.00 36.00 0.00 3.85
taucmdr.cf.storage.sqlite3_file 0.00 16.00 6.25 11.54
taucmdr.util 0.00 4.00 18.75 3.85
taucmdr.model.trial 0.00 4.00 12.50 0.00
taucmdr.model.project 0.00 4.00 12.50 0.00
taucmdr.model.target 0.00 4.00 6.25 0.00
taucmdr.cf.storage.local_file 0.00 4.00 6.25 0.00
taucmdr.cf.software.papi_installation 0.00 4.00 0.00 7.69
taucmdr.cf.storage.levels 0.00 4.00 0.00 3.85
taucmdr.cli.commands.trial.renumber 0.00 4.00 0.00 0.00
taucmdr.cli.commands.target.edit 0.00 4.00 0.00 0.00
taucmdr.cli.commands.target.create 0.00 4.00 0.00 0.00
taucmdr.cli.commands.target.copy 0.00 4.00 0.00 0.00
taucmdr.init 0.00 4.00 0.00 0.00
taucmdr.cf.software.tau_installation 0.00 0.00 18.75 26.92
taucmdr.cf.compiler.init 0.00 0.00 6.25 3.85
taucmdr.cli.commands.select 0.00 0.00 6.25 0.00
taucmdr.cli.init 0.00 0.00 6.25 0.00
taucmdr.mvc.model 0.00 0.00 0.00 7.69
taucmdr.mvc.controller 0.00 0.00 0.00 7.69
taucmdr.progress 0.00 0.00 0.00 3.85
taucmdr.model.application 0.00 0.00 0.00 3.85
taucmdr.error 0.00 0.00 0.00 3.85
taucmdr.cli.commands.experiment.edit 0.00 0.00 0.00 3.85
taucmdr.cli.arguments 0.00 0.00 0.00 3.85
taucmdr.cf.software.sqlite3_installation 0.00 0.00 0.00 3.85

Messages

message id occurrences
missing-docstring 16
arguments-differ 12
inconsistent-return-statements 10
invalid-name 4
useless-super-delegation 3
line-too-long 3
fixme 3
bad-continuation 3
too-many-nested-blocks 2
too-many-locals 2
logging-format-interpolation 2
unused-import 1
unused-argument 1
too-many-public-methods 1
redefined-builtin 1
raising-format-tuple 1
no-else-return 1
logging-not-lazy 1

Your code has been rated at 9.81/10 (previous run: 9.83/10, -0.03)

Per-file output (click to expand) 
************* Module taucmdr
W: 42, 4: Redefining built-in 'basestring' (redefined-builtin)
************* Module taucmdr.error
C: 76, 4: Missing method docstring (missing-docstring)
************* Module taucmdr.progress
C:178, 4: Missing method docstring (missing-docstring)
************* Module taucmdr.cf.software.sqlite3_installation
C: 51, 0: Line too long (121/120) (line-too-long)
************* Module taucmdr.cf.software.papi_installation
C: 49, 0: Wrong continued indentation (add 7 spaces).
         'http://fs.paratools.com/tau-mirror/papi-5.5.1.tar.gz']}
         ^      | (bad-continuation)
W: 69, 4: Parameters differ from overridden '_prepare_src' method (arguments-differ)
C: 77, 8: Variable name "cc" doesn't conform to u'[a-z_][a-z0-9_]{2,30}$' pattern (invalid-name)
************* Module taucmdr.util
R:143, 0: Either all return statements in a function should return an expression, or none of them should. (inconsistent-return-statements)
R:423, 0: Either all return statements in a function should return an expression, or none of them should. (inconsistent-return-statements)
W:465,18: Unused argument 'label' (unused-argument)
R:714, 4: Either all return statements in a function should return an expression, or none of them should. (inconsistent-return-statements)
C:734, 4: Missing function docstring (missing-docstring)
************* Module taucmdr.cf.storage.levels
C: 64, 0: Missing function docstring (missing-docstring)
W: 42, 0: Unused import os (unused-import)
************* Module taucmdr.cf.storage.local_file
W: 80, 4: Parameters differ from overridden 'write' method (arguments-differ)
R:227, 4: Either all return statements in a function should return an expression, or none of them should. (inconsistent-return-statements)
************* Module taucmdr.cf.compiler
R:300, 8: Too many nested blocks (6/5) (too-many-nested-blocks)
C:662, 4: Missing method docstring (missing-docstring)
************* Module taucmdr.mvc.controller
C: 54, 0: Missing function docstring (missing-docstring)
C: 55, 4: Missing function docstring (missing-docstring)
************* Module taucmdr.mvc.model
C: 75, 4: Missing method docstring (missing-docstring)
C:349,16: Missing function docstring (missing-docstring)
************* Module taucmdr.cf.storage.sqlite3_file
W:539, 0: TODO This could be made more efficient if we make this function less general. (fixme)
W:682, 0: TODO Change this to use a single operation (fixme)
C:227, 0: Wrong continued indentation (add 5 spaces).
                  log=False)
                  ^    | (bad-continuation)
W:112,29: Use % formatting in logging functions and pass the % parameters as arguments (logging-format-interpolation)
W:136,25: Use % formatting in logging functions and pass the % parameters as arguments (logging-format-interpolation)
C:276, 4: Missing method docstring (missing-docstring)
C:295, 4: Missing method docstring (missing-docstring)
R:419,12: Unnecessary "else" after "return" (no-else-return)
************* Module taucmdr.cf.software.tau_installation
C:875, 0: Wrong continued indentation (remove 1 space).
                  ] if flag]
                 |^ (bad-continuation)
C:932, 0: Line too long (123/120) (line-too-long)
C:826,45: Variable name "f" doesn't conform to u'[a-z_][a-z0-9_]{2,30}$' pattern (invalid-name)
C:929, 4: Missing method docstring (missing-docstring)
R:954, 4: Either all return statements in a function should return an expression, or none of them should. (inconsistent-return-statements)
R:1732, 4: Either all return statements in a function should return an expression, or none of them should. (inconsistent-return-statements)
C:1881, 4: Missing method docstring (missing-docstring)
C:1906, 8: Variable name "p" doesn't conform to u'[a-z_][a-z0-9_]{2,30}$' pattern (invalid-name)
C:1907, 8: Variable name "m" doesn't conform to u'[a-z_][a-z0-9_]{2,30}$' pattern (invalid-name)
R:169, 0: Too many public methods (27/25) (too-many-public-methods)
************* Module taucmdr.cli
R:230, 0: Either all return statements in a function should return an expression, or none of them should. (inconsistent-return-statements)
************* Module taucmdr.cli.arguments
C:476, 0: Line too long (123/120) (line-too-long)
************* Module taucmdr.cli.commands.select
R:118, 4: Either all return statements in a function should return an expression, or none of them should. (inconsistent-return-statements)
************* Module taucmdr.cli.commands.trial.renumber
W: 84, 8: Specify string format arguments as logging function parameters (logging-not-lazy)
************* Module taucmdr.cli.commands.experiment.edit
C: 39, 0: Missing class docstring (missing-docstring)
************* Module taucmdr.cli.commands.target.edit
W: 61,12: Parameters differ from overridden '__call__' method (arguments-differ)
************* Module taucmdr.cli.commands.target.copy
W: 59,12: Parameters differ from overridden '__call__' method (arguments-differ)
************* Module taucmdr.cli.commands.target.create
W: 68,12: Parameters differ from overridden '__call__' method (arguments-differ)
************* Module taucmdr.model.application
C: 50, 0: Missing function docstring (missing-docstring)
************* Module taucmdr.model.project
W: 98, 4: Parameters differ from overridden 'delete' method (arguments-differ)
R:142, 4: Too many local variables (21/20) (too-many-locals)
R:150, 8: Too many nested blocks (8/5) (too-many-nested-blocks)
************* Module taucmdr.model.trial
R:353, 4: Either all return statements in a function should return an expression, or none of them should. (inconsistent-return-statements)
R:531, 4: Too many local variables (24/20) (too-many-locals)
W:589,24: Exception arguments suggest string formatting might be intended (raising-format-tuple)
************* Module taucmdr.model.target
W:683, 0: FIXME: not implemented (fixme)
R:680, 4: Either all return statements in a function should return an expression, or none of them should. (inconsistent-return-statements)
************* Module taucmdr.model.experiment
W:109, 4: Parameters differ from overridden 'one' method (arguments-differ)
W:113, 4: Parameters differ from overridden 'all' method (arguments-differ)
W:117, 4: Parameters differ from overridden 'count' method (arguments-differ)
W:123, 4: Parameters differ from overridden 'search' method (arguments-differ)
W:127, 4: Parameters differ from overridden 'exists' method (arguments-differ)
W:134, 4: Useless super delegation in method 'create' (useless-super-delegation)
W:137, 4: Useless super delegation in method 'update' (useless-super-delegation)
W:140, 4: Useless super delegation in method 'unset' (useless-super-delegation)
W:143, 4: Parameters differ from overridden 'delete' method (arguments-differ)
C:432, 4: Missing method docstring (missing-docstring)
Stderror 
Using config file /home/runner/work/taucmdr/taucmdr/pylintrc