search

SpatioTemporal / STARE

SpatioTemporal Adaptive Resolution Encoding, a unifying index for geo-located data.
Other
10 stars 7 forks source link

NonConvexHull() call is sucking down the electricity!! #69

Open captainkirk99 opened 4 years ago

captainkirk99 commented 4 years ago

That NonConvexHull() call really gives my CPU a workout! ;-)

Is there an easy way to make that multi-threaded? Because actually I have multiple cores, and the test is running, and one core is pegged at 100% while the rest are mostly idle...

captainkirk99 commented 4 years ago

Oh dear. It's also causing a core dump:

../src/createSidecarFile -d MOD09 -r /var/lib/jenkins/workspace/AAA_ejh_STAREmaster/test /home/ed/MYD09.A2020058.1515.006.2020060020205.hdf
GeoFile constructor
ModisL2GeoFile constructor
250m Data Lines
250m Data Samples
500m Data Lines
500m Data Samples
1km Data Lines
1km Data Samples
terminate called after throwing an instance of 'std::out_of_range'
  what():  vector::_M_range_check: __n (which is 0) >= this->size() (which is 0)
./run_large_file_tests.sh: line 2: 29063 Aborted                 (core dumped) ../src/createSidecarFile -d MOD09 -r `pwd` /home/ed/MYD09.A2020058.1515.006.2020060020205.hdf
Build step 'Execute shell' marked build as failure
Finished: FAILURE
michaelleerilee commented 4 years ago

Best & easiest way is to extract g-ring info. NonConvexHull probably isn’t too hard to parallelize.

michaelleerilee commented 4 years ago

I haven’t looked at MOD09, not sure if something specific to ‘05 got into the generic MOD L2 code.

michaelleerilee commented 4 years ago

Note, for MODIS data we can probably introduce a stride of 10 because of the number of detectors along the track.

michaelleerilee commented 4 years ago

A little help is available with the PR https://github.com/SpatioTemporal/STAREmaster/pull/103 which parallelizes a loop in the MOD05 STARE indexing, but not the NonConvexHull. We should leave this issue open as we should still look at parallelizing STARE and NonConvexHull.