ccao-data / model-res-avm

Automated valuation model for all class 200 residential properties in Cook County (except vacant land and condos)
GNU Affero General Public License v3.0
20 stars 3 forks source link

Temporarily update Dockerfile to run python/comps.py #237

Closed jeancochrane closed 2 months ago

jeancochrane commented 2 months ago

This PR is a companion to https://github.com/ccao-data/model-res-avm/pull/236, intended to benchmark the current performance of the comps algorithm using numba. I don't plan to merge it and instead will close it once benchmarking is complete.

Findings

20k observations, 10k comparisons

framework instance type arch time logs
taichi g5.12xlarge x86 2.36s link
taichi g5.12xlarge CUDA 4.33s link
taichi m4.10xlarge x86 4.44s link
numba g5.12xlarge x86 6.07s link
numba m4.10xlarge x86 10.52s link

100k observations, 50k comparisons

framework instance type arch time logs
numba g5.12xlarge x86 31.87s link
taichi c5.24xlarge x86 31.93s link
taichi m4.10xlarge x86 34.09s link
numba c5.24xlarge x86 37.31s link
taichi g5.12xlarge x86 37.75s link
taichi g5.12xlarge CUDA 43.58s link
numba m4.10xlarge x86 64.19s link
jeancochrane commented 2 months ago

Closing, see #236 for full results.