search

TheDigitalFrontier / parallel-decision-trees

Semester project in CS205 Computing Foundations for Computational Science at Harvard School of Engineering and Applied Sciences, spring 2020.
MIT License
3 stars 1 forks source link

AWS setup for MPI + OpenMP + OpenACC #13

Closed johannes-kk closed 4 years ago

wfseaton commented 4 years ago

I was able to successfully get bin/test_random_forest and demo/demo_rf_serial.cpp to compile and run on AWS. I chose a t2.2xlarge instance with Ubuntu Server 16.04 and no other modifications - this runs demo_rf_serial.cpp in ~3.5 seconds so is plenty large.

We should consider acquiring a larger dataset for testing given the limited availability of speed up on 3.5 seconds.

To configure an AWS instance to run serial, you'll need to install g++ with below commands:

$ sudo apt-get install software-properties-common $ sudo add-apt-repository ppa:ubuntu-toolchain-r/test $ sudo apt-get update $ sudo apt-get install g++ $ gcc --version # To test g++ installation and version

Next, compile and run demo/demo_rf_serial.cpp using command: $ g++ -std=c++14 -g3 ../demo/demo_rf_serial.cpp -o demo_rf_serial $ time ./demo_rf_serial Make sure you compile demo file within the /demo/ folder, otherwise the absolute path to the dataset will fail.

wfseaton commented 4 years ago

I also added file speedup/speedup_plot.py with Python code to plot two lists of speedup numbers. Feel free to change to fit your tests. Run it locally so you don't need to install Python on the AWS instance.

wfseaton commented 4 years ago

For whichever parallel language you're working on, you'll need additional installed packages. Instructions here for ease of reference: MPI OpenACC OpenMP

johannes-kk commented 4 years ago

We should consider acquiring a larger dataset for testing given the limited availability of speed up on 3.5 seconds.

And a much higher ntrees. 10 is very low, the default for sklearn is 500, I think.

gpestre commented 4 years ago

Thanks for the instructions @wfseaton ! Fyi I included the -g3 in the compile scripts for testing, but it's the "debug" flag. It makes it easier to get a traceback in GDB, but probably impacts performance. In our experiments I think we should choose a different flag as our baseline.

johannes-kk commented 4 years ago

We've been using O0 thus far when testing, as it seems like the least disruptive compilation – though it does in fact seem to speed up quite a bit compared to no flag at all, which is odd, as I thought O0 was the default ¯_(ツ)_/¯

wfseaton commented 4 years ago

Closing as testing is underway in AWS by Hardik