commontk / CTK

A set of common support code for medical imaging, surgical navigation, and related purposes.
https://commontk.org
Apache License 2.0
857 stars 490 forks source link

optimize cache tag on insert #224

Closed pieper closed 12 years ago

pieper commented 12 years ago

All tags to cache should be collected in a single load of the dicom file (not one load per access as we do now).

Also the insert to the database should be done in a single query and not one insert per tag.

This builds on previous issue #216

nolden commented 12 years ago

Have a look at https://github.com/nolden/CTK/commit/67a65cf4eca0caa71c0c32ec478c460440c20ee7

I did some work there to iterate over the dataset and generate a QMap containing all requested tag values in one read.

It should be easy to generate also a single insert from that.

Unfortunately I won't be able to work on this myself but if it's useful and you have questions I can help.

On 08/02/2012 12:44 AM, Steve Pieper wrote:

All tags to cache should be collected in a single load of the dicom file (not one load per access as we do now).

Also the insert to the database should be done in a single query and not one insert per tag.

This builds on previous issue #216


Reply to this email directly or view it on GitHub: https://github.com/commontk/CTK/issues/224


Dipl.-Inform. Med. Marco Nolden Deutsches Krebsforschungszentrum (German Cancer Research Center) Div. Medical & Biological Informatics Tel: (+49) 6221-42 2325 Im Neuenheimer Feld 280 Fax: (+49) 6221-42 2345 D-69120 Heidelberg eMail: M.Nolden@dkfz.de

pieper commented 12 years ago

Yes, looks very useful. This is kind of in my critical path, so I will try to work on it.

pieper commented 12 years ago

Looking again at the sqlite documentation, we don't need to combine the inserts into a single operation because we are using the PRAGMA synchronous=OFF directive that accomplishes the same thing:

http://www.sqlite.org/faq.html#q19

Extracting all the tag values to cache with a single load of the dataset is still important for speeding up the process.