dabreegster
commented
2 years ago I'll upload to Azure this afternoon. The new perf table is slower, because I'm using a slower laptop than last time. :(
| study_area |num_msoas|num_households|num_people|pb_file_size| runtime |commuting_runtime|memory_usage|
|------------------------|---------|--------------|----------|------------|----------|-----------------|------------|
| bedfordshire | 74 | 271,487 | 650,950 | 99.38MiB |15 seconds| 8 seconds | 309.72MiB |
| berkshire | 107 | 363,653 | 878,045 | 133.58MiB |22 seconds| 12 seconds | 316.24MiB |
| bristol | 55 | 196,230 | 456,532 | 71.91MiB | 7 seconds| 2 seconds | 159.43MiB |
| buckinghamshire | 99 | 324,843 | 759,879 | 114.40MiB |15 seconds| 8 seconds | 313.42MiB |
| cambridgeshire | 98 | 346,532 | 834,141 | 126.23MiB |20 seconds| 9 seconds | 316.19MiB |
| cheshire | 139 | 463,106 | 1,040,634| 158.75MiB |22 seconds| 9 seconds | 321.21MiB |
| cornwall | 74 | 246,873 | 564,604 | 87.49MiB |10 seconds| 3 seconds | 293.28MiB |
| cumbria | 64 | 224,779 | 485,035 | 74.14MiB |12 seconds| 5 seconds | 159.98MiB |
| derbyshire | 131 | 457,791 | 1,024,952| 156.26MiB |23 seconds| 13 seconds | 320.70MiB |
| devon | 156 | 521,790 | 1,178,315| 180.29MiB |30 seconds| 16 seconds | 588.32MiB |
| dorset | 95 | 344,246 | 751,334 | 114.69MiB |15 seconds| 7 seconds | 315.03MiB |
| durham | 117 | 406,164 | 904,785 | 137.19MiB |14 seconds| 6 seconds | 315.76MiB |
| east_sussex | 102 | 380,180 | 830,761 | 126.10MiB |17 seconds| 8 seconds | 315.81MiB |
|east_yorkshire_with_hull| 75 | 261,267 | 579,746 | 89.48MiB |11 seconds| 3 seconds | 294.73MiB |
| essex | 211 | 771,734 | 1,798,893| 273.67MiB |56 seconds| 39 seconds | 632.78MiB |
| gloucestershire | 107 | 392,120 | 901,395 | 136.39MiB |20 seconds| 9 seconds | 318.22MiB |
| greater_london | 983 | 3,135,814 | 8,672,103| 1.30GiB |18 minutes| 17 minutes | 4.57GiB |
| greater_manchester | 346 | 871,651 | 2,746,858| 409.29MiB | 2 minutes| 2 minutes | 1.15GiB |
| hampshire | 225 | 775,203 | 1,803,991| 275.67MiB |55 seconds| 39 seconds | 633.90MiB |
| herefordshire | 23 | 83,115 | 191,282 | 29.17MiB | 5 seconds| 2 seconds | 78.76MiB |
| hertfordshire | 153 | 492,783 | 1,144,974| 173.70MiB |26 seconds| 16 seconds | 587.93MiB |
| isle_of_wight | 18 | 64,602 | 135,125 | 21.46MiB | 4 seconds| 1 second | 73.98MiB |
| kent | 220 | 692,896 | 1,808,206| 272.43MiB |42 seconds| 27 seconds | 631.51MiB |
| lancashire | 191 | 597,317 | 1,472,550| 220.98MiB |31 seconds| 18 seconds | 624.11MiB |
| leicestershire | 120 | 417,621 | 1,043,283| 155.47MiB |24 seconds| 15 seconds | 318.24MiB |
| lincolnshire | 134 | 434,060 | 1,064,174| 159.85MiB |18 seconds| 9 seconds | 583.27MiB |
| merseyside | 184 | 546,791 | 1,401,012| 209.82MiB |35 seconds| 23 seconds | 622.66MiB |
| norfolk | 110 | 379,188 | 891,006 | 136.35MiB |16 seconds| 6 seconds | 318.36MiB |
| north_yorkshire | 138 | 434,489 | 1,069,514| 162.03MiB |20 seconds| 10 seconds | 584.46MiB |
| northamptonshire | 91 | 293,580 | 733,190 | 111.56MiB |14 seconds| 7 seconds | 313.19MiB |
| northumberland | 40 | 113,436 | 316,618 | 47.39MiB | 6 seconds| 1 second | 147.70MiB |
| northwest_transpennine | 829 | 1,919,598 | 6,419,933| 962.23MiB | 9 minutes| 8 minutes | 2.31GiB |
| nottinghamshire | 138 | 413,097 | 1,139,096| 172.00MiB |27 seconds| 16 seconds | 585.32MiB |
| oxford_cambridge_arc | 353 | 1,152,245 | 2,823,838| 429.93MiB | 2 minutes| 2 minutes | 1.22GiB |
| oxfordshire | 86 | 254,974 | 669,237 | 101.27MiB |13 seconds| 6 seconds | 295.56MiB |
| rutland | 5 | 16,688 | 39,475 | 5.81MiB | 2 seconds| 1 second | 19.54MiB |
| shropshire | 62 | 153,284 | 497,064 | 74.08MiB | 8 seconds| 3 seconds | 158.25MiB |
| somerset | 124 | 384,165 | 944,394 | 144.82MiB |19 seconds| 9 seconds | 318.63MiB |
| south_yorkshire | 172 | 358,717 | 1,373,401| 201.49MiB |34 seconds| 24 seconds | 586.02MiB |
| staffordshire | 143 | 439,176 | 1,104,925| 165.50MiB |20 seconds| 10 seconds | 584.64MiB |
| suffolk | 90 | 326,760 | 739,296 | 111.96MiB |19 seconds| 5 seconds | 312.89MiB |
| surrey | 151 | 461,466 | 1,136,090| 172.31MiB |29 seconds| 18 seconds | 587.70MiB |
| tyne_and_wear | 145 | 414,128 | 1,111,239| 165.43MiB |19 seconds| 9 seconds | 583.12MiB |
| warwickshire | 108 | 319,511 | 933,391 | 139.14MiB |22 seconds| 13 seconds | 316.46MiB |
| west_midlands | 314 | 523,264 | 2,475,918| 365.32MiB |59 seconds| 41 seconds | 1.11GiB |
| west_sussex | 100 | 373,326 | 838,440 | 127.89MiB |16 seconds| 8 seconds | 316.58MiB |
| west_yorkshire | 299 | 501,156 | 2,272,063| 336.82MiB |62 seconds| 44 seconds | 1.11GiB |
| wiltshire | 89 | 305,679 | 686,963 | 105.47MiB |11 seconds| 4 seconds | 311.38MiB |
| worcestershire | 85 | 254,383 | 572,751 | 87.93MiB |11 seconds| 5 seconds | 293.73MiB |
Not yet regenerating all data or updating docs with the new output.
I'm regenerating everything now. This is an example PR for plumbing new fields into SPC from the input data. My process was:
1) Add fields to the .proto and rearrange stuff 2) Run
cargo checkand fix all the errors. I didn't have to remember where we parse data from the TUS files or write the protobuf, because the compiler told me