search

CODARcode / Example-Heat_Transfer

3 stars 3 forks source link

Improve DataSpaces usability and documentation #6

Closed philip-davis closed 7 years ago

philip-davis commented 7 years ago

Modified run-workflow.sh and workflow.swift to launch correctly-configured dataspaces_server based on command line arguments. This is done by adding a second argument to the run-workflow.sh script, which is used to

  1. do necessary precursor steps for the transports
  2. pass on to the workflow.swift script as the rmethod for stage_write

Also, paramaterized process counts in workflow.swift using a set of int values at the top of the script to make it easier to change process count and arrangement.

It is now possible to switch between DataSpaces and FlexPath by just changing the second argument to run-workflow.sh and appropriately modifying heat_transfer.xml.

These steps have been documented as well.

ianfoster commented 7 years ago

This is a new experience for me. I'd like to try this, but I think that I need it integrated it into savanna@develop so I can pull it. Is that feasible?

j-woz commented 7 years ago

This looks fine to me. In the near future we will want to be able to use the same workflow.swift for both modes FLEXPATH and DATASPACES so that the science logic is reused. But for now this is great.

jychoi-hpc commented 7 years ago

It looks like one step is missing. After running DataSpaces server and before running heat transfer app and stager, "conf" should be read and export two env variables: P2TNID and P2TPID.

In bash, we usually do as follows: ` --Export the main server config to the environment while read line; do export set "${line}" done < conf

--Then, run heat_transfer_adios2 and stage_write `

@philip-davis Do we need this step? Am I right? @jmjwozniak What would be the best way to read "conf" (DataSpaces server creates this file after running) and export env variables in Swift/T?

philip-davis commented 7 years ago

Thanks for looking at this @jychoi-hpc. Sourcing the conf isn't necessary in recent versions of DataSpaces. The clients read from the conf file directly now.

jychoi-hpc commented 7 years ago

I see. In my test run, it hangs without proceeding. I will check my run again.

jychoi-hpc commented 7 years ago

My fault. I forgot to change xml file. It works now.