benducke / Network-reconstruction-tools-for-GRASS-GIS

Node-based network model generator for GRASS GIS 7.x (and QGIS 3.x)
GNU General Public License v3.0
4 stars 1 forks source link

Replace v.db.select with db.select where appropriate #62

Open benducke opened 3 weeks ago

benducke commented 3 weeks ago

Replacing this: VAL=v.db.select map="${input}" layer="1" columns="${field_name}" where="${condition}" -c --quiet``

With this: VAL=db.select database="${atts_db}" sql="SELECT ${field_name} FROM ${input} WHERE ${condition};" -c --quiet``

... will give a speed-up of almost 60% for minimal refactoring effort!

benducke commented 3 weeks ago

This is appropriate, if:

neteler commented 3 weeks ago

Not sure if this hint is helpful (and it requires a SQLite/PG/MySQL backend): encapsulating SQL statements into a TRANSACTION notably speeds up the SQL backend processing:

BEGIN TRANSACTION
...
<SQL commands here>
....
END TRANSACTION

Python example:

Shell example:

benducke commented 3 weeks ago

Thanks! I completely forgot about SQL transactions. This gives a nice speed-up, especially when collating SQL statements before feeding them to the DBMS.

benducke commented 2 weeks ago

The use of v.db.select is also very common in all models support by v.net.models: These commonly reduce network connections by querying some sort of link attributes first.