TravelerList::userlog speed ideas

[yakra]

Yes, some of these items are mutually exclusive. :)

• [x] rename TravelerList::system_region_miles (not speed related; this just needs to happen)
• [x] remove extraneous if up top
• [x] remove extraneous if (r->last_update); this was checked prior to insertion https://github.com/yakra/DataProcessing/blob/29a6edfddc1132bfac95e0bbc59157f27fade551/siteupdate/cplusplus/classes/TravelerList/userlog.cpp#L122
• [ ] save system_region_mileages iterator as stashed in 8ccb8b7
• [ ] HighwaySystem::total_mileage
• [ ] avoid std::string concatenation and extraneous malloc/free of temporaries
  • [ ] Route::readable_name()
  • [ ] ConnectedRoute::readable_name()
• [ ] save iterator & use ternary (this may not do much; branch prediction works well & compilers optimize this stuff):
  • [ ] t_active_miles
  • [ ] system_region_mileage
• [x] Region objects contiguously stored in a TMArray sorted by code (see below)
• [ ] Don't keep sorting sysmbr (AKA HighwaySystem::mileage_by_region) contents; replace with an ordered map Doesn't do well on BiggaTomato, but how does it scale beyond 4 threads? A code string-based comparator may make the original setup too slow because !comp(a, b) && !comp(b, a). What if objects are sorted by code into a TMArray and compared based on address?
  • [ ] Another option if a still sorting an unordered_set is to make a list of pairs and avoid the sysmbr.at(region) lookup.
• [ ] These items are only used once; may or may not be worth it to convert to map:
  • [ ] active_preview_mileage_by_region Some options in decreasing expected efficiency...
  • [ ] Option 1: Convert to ordered map anyway
  • [ ] Option 2: Iterate thru pre-sorted Region::allregions; start by trying to lookup t_active_prev_miles
  • [ ] Option 3: Continue to sort temporary travregions as done now, but use the entire pair (requires a new comparison function). This avoids another active_preview_mileage_by_region lookup.
  • [ ] system_region_mileages
  • [ ] Outer: iterate this instead of iterating HighwaySystem::syslist & checking membership
  • Inner can remain an unordered_map, as it's not iterated -- sysmbr is iterated instead in order to include zero-mileage regions; regional lookups are performed.
  • [ ] updated_routes
• [ ] single TravelerList::mileage_by_region table Previously marked "lowest priority", but I'm reconsidering that:
  • There is a noticeable speed increase on BiggaTomato, even if a very slight one. This could potentially be one of those improvements that's a drop in the bucket for Linux, but a big difference for FreeBSD. Won't know until I try it out.
  • TravelMapping#584 has changed the game with the early threadsafe setup of key/value pairs.
  • Affected tasks:
  • Route::store_traveled_segments: Setup 1 table, not 2.
  • ConcAugThread: Setup 1 table, not 2.
  • Region::compute_stats: Computation. Can do 1 lookup instead of 2, save a reference, update active/preview, update active-only as needed.
  • TravelerList::userlog: Lookups in "Overall by region" table. 1 lookup if using a sorted unordered_map; 0 if using an ordered map.
  • sqlfile1: Avoid active-only lookup; just iterate.

[yakra]

https://github.com/yakra/DataProcessing/blob/5563f0fb1b49cbd2587b1666f4f4e8f43c25b0b5/siteupdate/cplusplus/classes/TravelerList/userlog.cpp#L77-L78

https://github.com/yakra/DataProcessing/blob/5563f0fb1b49cbd2587b1666f4f4e8f43c25b0b5/siteupdate/cplusplus/classes/Route/Route.cpp#L160-L165

All the hash table lookups make this pretty expensive.

• [x] TMBitset should yield a big improvement. Edit: Yes, it did! As of this writing, only the first 3 check boxes in the OP are checked. This takes priority over any other remaining items.
• ~While at it, go branchless: miles += s->clinched_by(t) * s->length;~
• ~Quick-n-dirty alternative: Can go branchless now. Probably won't be much help though, based on similar experiments.~
• Didn't go branchless after all. Don't know if it was maybe a dependency chain or what, but it was decidedly slower.

[yakra]

Bad ideas

• for (HighwaySystem *h : HighwaySystem::syslist)
  • [ ] auto& region_mileages = system_region_mileages.at(h); and put the whole "stats by system" routine in one big try block. Significantly slower. Verify that it's slower on FreeBSD too.
  • [ ] auto it = system_region_mileages.find(h); and then dereference, as stashed in 8ccb8b7. Still a bit slower though not as much. Verify slower on FreeBSD.
• Instead, either:
  • [ ] Sort a list of key/value pairs
  • [ ] Make system_region_mileages an ordered map