cooperative-computing-lab / cctools

The Cooperative Computing Tools (cctools) enable large scale distributed computations to harness hundreds to thousands of machines from clusters, clouds, and grids.
http://ccl.cse.nd.edu
Other
135 stars 120 forks source link

Doc: Improve WQ Example Application #2542

Open dthain opened 3 years ago

dthain commented 3 years ago

The current example WQ program (gzip some files) isn't an ideal example to show to new users. It's simple, which is good, but doesn't represent a typical use of WQ. And it has strange ergonomics in that the user must provide some extra files to compress.

Let's think of a better example application. It should have the following properties:

Suggestions?

dthain commented 3 years ago

Some ideas:

btovar commented 3 years ago

We can do a brute-force algorithm for some known number, such as pi by probability:

https://mste.illinois.edu/activity/buffon/

Each task does the same thing, and then we collect everything in a final task.