Princeton-CDH / htr2hpc

Apache License 2.0
1 stars 0 forks source link

Determine configurations to improve process runtimes #21

Open cmroughan opened 4 weeks ago

cmroughan commented 4 weeks ago

Testing has been in progress with Selenium, focusing on file uploads/deletions and segmentation recognition (as per issue #12 ).

As of 2024-10-25, these rounds of testing have been run on the following VM configurations:

However, these CPU and memory increases have not led to any real improvement in runtimes.

CPUs Memory Import test average runtime Segmentation (lines) runtime Segmentation (lines + regions) runtime
2 16 GB 0:27:32 1:05:00 -
4 16 GB 0:29:24 - 0:58:00
4 32 GB 0:28:32 1:01:00 1:01:00

Also ran a segmentation test (4 CPUs, 32 GB, lines) a second time on a fresh set of images, in case there was a difference in runtime between documents that had already been segmented and those that had not. Runtime ended up being exactly the same.

Should look at celery worker configurations to start, see whether changes there make a difference.