search

cooperative-computing-lab / makeflow-examples

Example workflows for the Makeflow workflow system.
33 stars 18 forks source link

Adds Montage workflow example #20

Open nkremerh opened 6 years ago

dthain commented 6 years ago

This looks like a modification of an existing program. What's the original source?

nkremerh commented 6 years ago

@dthain The mDAG.c file is from Montage. It is used to generate a DAX for Pegasus in their release, but it is modified to produce a makeflow for us. I will add proper documentation for that.

nkremerh commented 6 years ago

Putting this pull request on hold as the version of Montage (C as opposed to the newer Python implementation) we use is no longer fully supported and relies on downloading some files (the pieces of the mosaic) which are no longer where they need to be for Montage to work.

dthain commented 6 years ago

Can you put some additional notes here about the "new" montage workflow, so we have some breadcrumbs to review the problem in the future?

nkremerh commented 6 years ago

The new Montage workflow (https://github.com/pegasus-isi/montage-workflow-v2) is written in Python and uses the Pegasus DAX API. The older C version (and the Python version) both have issues with the data find functionality (downloading images to form the mosaic) at the time this comment was posted.

Our best bet going forward is that the Python version will get the fix for this, but the C version may not. Since the modified Montage makeflow generator we have been using is based on the C version, we need to think carefully about which iteration of Montage we plan on supporting or putting in the effort to get both working once the data find issue is resolved on their end.