Feature/synapse tasks

[andre-merzky]

Hi Dan, Yadu, Zhao,

we are planning some more skeleton experiments :) Amongst them, we would like to compare synapse emulation with what skeletons do in task.c. We also would like to couple synapse with the skeleton ability to render multi-stage workloads.

So we implemented a drop-in replacement for task.c which accepts exactly the same parameters as task.c, but forwards them to synapse for emulation. A simple switch in the executable name thus switches the emulation layer.

This pull requests adds exactly that file. It would be great if you could review it, and, if you are so inclined, merge and release to pypi? The latter would be great, as it makes deployment on the target resources, specifically OSG, so much easier...

Let is know if you have any questions, comments or concerns, obviously!

Best, Andre.

[andre-merzky]

PS.: I should mention that this needs the feature/named_storage branch of radical.synapse to function.

[danielskatz]

@zhaozhang , @yadudoc , can one of you review this? If not, I will, but maybe not for a few days.

[zhaozhang]

Hi Andre,

1) Is there a way to explicitly specify the dependency of "feature/named_storage" branch of radical.synapse?

2) Can we make the task length compatible with exponential format as well as the old format? e.g., 1e10 and 10000000000.

3) I see in PR 18, there is a unit test for a two-stage workflow, is that right?

Zhao

[andre-merzky]

1) Is there a way to explicitly specify the dependency of "feature/named_storage" branch of radical.synapse?

If the state of affairs look ok from your end, I'll push a synapse release to pypi to make sure the dependency gets cleanly resolved.

2) Can we make the task length compatible with exponential format as well as the old format? e.g., 1e10 and 10000000000

hehe, I can see why that is useful. Counting zeros is boring :P Let me have a look at that.

3) I see in PR 18, there is a unit test for a two-stage workflow, is that right?

Indeed - I changed the first stage there to use the flop emulation via synapse, so we end up with a mixed-typed workflow, which seems to work nicely!

[andre-merzky]

Number (2) should be easy:

>>> n="1E10"
>>> float(n)
10000000000.0
>>> int(float(n))
10000000000

[zhaozhang]

Hi Andre,

Can you revise the PR with this number format change and resubmit it?

Thanks, Zhao

On Wed, Jul 6, 2016 at 1:10 PM, Andre Merzky notifications@github.com wrote:

Number (2) should be easy:

n="1E10" float(n) 10000000000.0 int(float(n)) 10000000000

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/applicationskeleton/Skeleton/pull/17#issuecomment-230857091, or mute the thread https://github.com/notifications/unsubscribe/AEh7OIYq__q7ndwy4BXFLBnlna05ekS4ks5qS--LgaJpZM4I9Omq .

[andre-merzky]

It actually works out of the box already:

./bin/aimes-skeleton-synapse.py serial flops 1 1E11 65536 65536 0 0 0

[zhaozhang]

Awesome! I will go ahead and accept it.

Zhao On Jul 6, 2016 1:44 PM, "Andre Merzky" notifications@github.com wrote:

It actually works out of the box already:

./bin/aimes-skeleton-synapse.py serial flops 1 1E11 65536 65536 0 0 0

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/applicationskeleton/Skeleton/pull/17#issuecomment-230866887, or mute the thread https://github.com/notifications/unsubscribe/AEh7OMRZRtxSt7xmTn1YYoe95DNPN_4Lks5qS_emgaJpZM4I9Omq .