search

yakra / DataProcessing

Data Processing Scripts and Programs for Travel Mapping Project
0 stars 0 forks source link

specify # threads for each task #116

Open yakra opened 4 years ago

yakra commented 4 years ago
yakra commented 4 years ago

# of threads for optimum efficiency as of ef1894f:

thread BT
(4)		 lab1
(8)		 lab1.5
(8)		 lab2
(16)		 lab3
(24)
ReadWptThread 4 8 4 8 9
NmpSearchThread 4 8 7-8 9-16 19
NmpMergedThread 4 6 8 15 21
ReadListThread 4 8 8 15 15
ConcAugThread 4 3 4 5 3
ComputeStatsThread 4 3-8 3-4 10-11 12
UserLogThread 4 8 8 16 24
SubgraphThread 4 8 8 14 10
yakra commented 4 years ago

Well this is mysterious. On BiggaTomato, when <4 ReadWptThreads are followed by 4+ NmpSearchThreads, siteupdate crashes when we try to create ifstream nmpfpfile(args.highwaydatapath+"/nmpfps.log");, of all things.

yakra commented 4 years ago 
( echo -e "mnt\trwt\tnst\tresult"; \
for mnt in {1..9}; do
  for rwt in {1..9}; do
    for nst in {1..9}; do
      rm -rf graphs logs nmp_merged stats
      mkdir -p graphs logs/users nmp_merged stats

      echo -en "$mnt\t$rwt\t$nst\t"
      ./siteupdate_numthr -l logs -n nmp_merged -c stats -g graphs \
      -u /home/yakra/tm/UserData/list_files -w /home/yakra/tm/HighwayData \
      -t $mnt --ReadWptThreads $rwt --NmpSearchThreads $nst \
      > sulogs/siteupdate-$mnt-$rwt-$nst.log

      lastline=$(tail -n 1 sulogs/siteupdate-$mnt-$rwt-$nst.log)
      if [[ $(echo $lastline | grep -c '^Total run time: \[[0-9]\+\.[0-9]\]$') != 1 ]]; then
        echo "crash"
      else
        echo "OK"
      fi
    done
  done
done ) | tee crashtest.log
yakra commented 4 years ago

On lab1.5 (UbuntuMate 18.04) I'm seeing

yakra commented 4 years ago

Results are not deterministic. Value of NumThreads & # of hardware threads does not appear to affect things. Crashes so far only happen when NmpSearchThreads > ReadWptThreads, but not every time. Not all crashes produce error messages in shell. ToDo: tail siteupdate.log for crashes

yakra commented 4 years ago

Crashes so far only happen when NmpSearchThreads > ReadWptThreads, but not every time.

This pattern has not held.

Of 125 runs on BiggaTomato:

Of 400 runs so far on lab1.5:

yakra commented 4 years ago

Duh.

(I am not worthy of saying "When the light went on it nearly blinded me.")

Before, thread **thr = new thread*[args.numthreads]; was just fine, with every array of threads being the same size. But now, it's declared as thread **thr = new thread*[args.ReadWptThreads]; -- and I then go on to access it as if it's any old size. Even a bigger size. Wacky antics ensue.

I was looking right at this and didn't notice it...