horovod in powerai 1.6.2, error occurs with hvd.init()

[wacoder]

Hi, Try to install Horovod with PyTorch based on powerai image following official tutorial, error occurs when try to hvd.init(). Is there any material on how to set up DDL_OPTIONS to use ddl? Thanks

• code

  import horovod.torch as hvd hvd.init() [2020-06-11 03:01:22.297540: W horovod/common/ops/ddl_operations.cc:105] DDL backend has been deprecated. Please, start using the NCCL backend by building Horovod with 'HOROVOD_GPU_ALLREDUCE=NCCL HOROVOD_GPU_BROADCAST=NCCL'. terminate called after throwing an instance of 'std::logic_error' what(): DDL_OPTIONS env variable needs to be set to use DDL. Aborted (core dumped)`

---

I also tried to set up env variable for DDL_OPTIONS with export DDL_OPTIONS=DDL it produce another error

`

import horovod.torch as hvd hvd.init() [2020-06-11 03:14:33.814715: W horovod/common/ops/ddl_operations.cc:105] DDL backend has been deprecated. Please, start using the NCCL backend by building Horovod with 'HOROVOD_GPU_ALLREDUCE=NCCL HOROVOD_GPU_BROADCAST=NCCL'. E 03:14:33.814 634 652 DDL:86 ] ill-formated option :[DDL] terminate called after throwing an instance of 'std::system_error' what(): Resource deadlock avoided Aborted (core dumped) `

[bnemanich]

You will need to launch the job using the ddlrun tool. This will setup the appropriate environment variables. For more information on ddlrun, see https://www.ibm.com/support/knowledgecenter/SS5SF7_1.6.2/navigation/wmlce_ddlrun.html.

[wacoder]

@bnemanich thanks very much. Does that mean I can use horovod in script, and use ddlrun to run?