ComputeStatsThread parallelism

[yakra]

Separate ComputeStatsThread into:

• [x] CompStatsRThread
  • [x] Mutexes per route rather than per segment.
• [x] CompStatsTThread
  • [x] A small increase in overhead during ConcAugThread. Should be well worth it.
  • [x] Iterate by TravelerList, not HighwaySystem->Route-HighwaySegment.
  • [x] No mutexes!
  • [x] More refactoring (Python speedup too)!
• [x] How do the various alternatives perform with threading disabled?
  • _cs1s2lt: 8.66s
  • _cs1s2lhr: 7.26s
  • _cs1s1l: 7.18s

---

• [x] "compute lots of stats" comment can be cleaned up & clarified in Python too

[yakra]

# user logs
for u in *; do
  echo $u
  diff <(egrep -c 'System .+ \(active) overall: [0-9]+.[0-9]{2} of [0-9]+.[0-9]{2} mi \(100.00%\)' $u) <(tail -n 2 $u | head -n 1 | cut -f7 -d' ')
  diff <(egrep -c 'System .+ \(preview) overall: [0-9]+.[0-9]{2} of [0-9]+.[0-9]{2} mi \(100.00%\)' $u) <(tail -n 1 $u | cut -f7 -d' ')
done

---

https://github.com/TravelMapping/DataProcessing/blob/a20ee6dcccbac73135a6b73488251c4b4652817f/siteupdate/cplusplus/classes/TravelerList/userlog.cpp#L48-L50 How did calculation of t_system_overall & h->total_mileage() change?

• h->total_mileage()

  unchanged

• t_system_overall

  old	new
  TravelerList::system_region_miles(h)	TravelerList::system_region_miles(h)
  sum of doubles in system_region_mileages	sum of doubles in system_region_mileages
  length/system_concurrency_count, system -> route -> segment	length/system_concurrency_count, TravelerList -> clinched_segments

• [x] convert double to float for comparison

[yakra]

cd tm/UserData/
git show -s
cd ../HighwayData
git show -s
cd ~/ytm/DataProcessing/siteupdate/cplusplus/
git branch
git checkout -- `git status | grep modified | sed 's~.*:~~'`
git pull
rm classes/TravelerList/compute_stats_t.cpp threads/CompStatsTThread.cpp
git checkout 258b6130f2ea997f2c5779efa44751cb70f6fa75; g++ siteupdate.cpp -std=c++11 -pthread -o siteupdate_cs1tn
git checkout ee6d5304ce88fd52a67a9b2aa6903bacb05ea69f; g++ siteupdate.cpp -std=c++11 -pthread -o siteupdate_cs1ti #cs1t
git checkout 7629ee13d4892be464e05eb02ed242005b9f6520; g++ siteupdate.cpp -std=c++11 -pthread -o siteupdate_cs1sn
git checkout 3e391a77f202a1431089e47020ddfe110cff2c81; g++ siteupdate.cpp -std=c++11 -pthread -o siteupdate_cs1si #cs1s
git checkout 8efec7a9a29dba14416d9dfaf25191d3cf9c1340; g++ siteupdate.cpp -std=c++11 -pthread -o siteupdate_cs1x  #cs1x
emacs check10.sh; ./m1.sh _cs1x

# _cs1tn _cs1ti _cs1sn _cs1si _cs1x

[yakra]

31ccd30c985c95960a3b5b57850d89e7f463ecd8 ComputeStatsThread parallelism phase 1

squashed on BiggaTomato:

head	msg
7629ee13d4892be464e05eb02ed242005b9f6520	_cs1sn (#366)
3e391a77f202a1431089e47020ddfe110cff2c81	_cs1si (#366)
92b0d9552007341be28432b97eeed466dd0ba77b	ST ConcAug fix; ComputeStats -> CompStatsR
56d47eaed1161ff7d72b18314f0eccabe73f6d8e	_cs1s2lhr
2daec1a10ef19b5565de09112be4aaac9c6ce312	_cs1s1l
7e78da0c769b0e4971bac4ed41fffa6082ed5d12	CompStatsRThread comments
5aced65db3df159e97036e49b5524df3351b506c	remove extraneous active/preview checks
278f5b56e4eaa536a64d68095da87f572d6e9c3c	clean up "compute lots of stats" comment