search

firemodels / fds

Fire Dynamics Simulator
https://pages.nist.gov/fds-smv/
Other
663 stars 622 forks source link

Win Cluster config problem - FDS and pyrosim #2950

Closed bvsrl closed 8 years ago

bvsrl commented 9 years ago

see attached file for problem with cluster configuration - windows system -and mpi - pyrosim 2015 with fds 6.2

any help welcome regards

drjfloyd commented 9 years ago

If you are having problems with running FDS through Pyrosim, contact Thunderhead Engineering customer support.

bvsrl commented 9 years ago

We doing a test with 2 PC to set-up and configure fds Cluster simulation with FDS pyrosim software. This is my configuration for RUN Cluster

PC1 - Computer name AABB1122 Win 8.1 64 bit User account XXZZ password PC2 _aabb (same for PC1 and PC2 ) FDS 6.2.0 SMV 6.2.2 PyroSim 2015 2 0604 x64 with license active MPI version 1.4.1p1 PyrosimCluster 2014-4-1105-x64 installed

Pyrosim file is in shared folder C:\abc\abc.psm

PC2 - Computer name TTWW8899 Win 7 64 bit User account XXZZ (same PC1) password PC1 FDS 6.2.0 SMV 6.2.2 PyroSim 2015 2 0604 x64 installation only MPI version 1.4.1p1 x86-64 PyrosimCluster 2014-4-1105-x64 installed

We encountered a number of difficulties in the configuration of ports and services, Some problems solved concern the Hydra_service. Finally, the Cluster fds command run with this log file, but it does not produce any result or out file.

Question: It is be found a configuration problem with the MPI or Hydra ? which inspections or tests can be performed ?

observations:

the issue was discussed in this forum - Pyrosim program
http://www.thunderheadeng.net/forum/index.php/topic,2686.0.html the problem is being analyzed.

after running the Cluster fds simulation command, the Time elapsed: counts Time remaining: set at Zero - No counts Progress: ( work done ) no running - no count

reply received from thunderheadeng forum, after many test: "For whatever reason, Intel's HYDRA process scheduler is just hanging and not producing output - error or otherwise."

This is log text starting cluster fds simulation

Starting FDS: fds.exe...

host: tow7prosbv106 host: tow8sbv50

mpiexec options:

Base path: C:\Program Files\PyroSim 2015\fds\ Launcher: service Debug level: 1 Enable X: -1

Global environment:

ADSK_MAXDES_x64_2014=C:\Program Files\Autodesk\3ds Max Design 2014\
ALLUSERSPROFILE=C:\ProgramData
APPDATA=C:\Users\utente locale\AppData\Roaming
asl.log=Destination=file
CM2014DIR=C:\Program Files (x86)\Common Files\Autodesk Shared\Materials\
CM2015DIR=C:\Program Files (x86)\Common Files\Autodesk Shared\Materials\
CommonProgramFiles=C:\Program Files\Common Files
CommonProgramFiles(x86)=C:\Program Files (x86)\Common Files
CommonProgramW6432=C:\Program Files\Common Files
COMPUTERNAME=TOW8SBV50
ComSpec=C:\windows\system32\cmd.exe
FP_NO_HOST_CHECK=NO
HOMEDRIVE=C:
HOMEPATH=\Users\utente locale
ILBDIR=C:\Program Files (x86)\Common Files\Autodesk Shared\Materials\
ILLDIR=C:\Program Files (x86)\Common Files\Autodesk Shared\Materials\
ILMDIR=C:\Program Files (x86)\Common Files\Autodesk Shared\Materials\
LOCALAPPDATA=C:\Users\utente locale\AppData\Local
LOGONSERVER=\\TOW8SBV50
MPICH_PORT_RANGE=8670:8690
MPIEXEC_PORT_RANGE=8670:8690
MSMPI_BIN=C:\Program Files\Microsoft MPI\Bin\
NUMBER_OF_PROCESSORS=8
OMP_NUM_THREADS=4
OnlineServices=Online Services
OS=Windows_NT
Path=C:\Program Files\Microsoft MPI\Bin\;C:\\shortcuts;C:\FDS6\bin;C:\ProgramData\Oracle\Java\javapath;c:\Program Files (x86)\Intel\iCLS Client\;c:\Program Files\Intel\iCLS Client\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\IPT;c:\Program Files\Intel\WiFi\bin\;c:\Program Files\Common Files\Intel\WirelessCommon\;C:\Program Files\Hewlett-Packard\SimplePass\;C:\Program Files (x86)\QuickTime\QTSystem\;C:\Program Files\Common Files\Autodesk Shared\;C:\Program Files (x86)\Autodesk\Backburner\;C:\Program Files\PyroSim 2015\fds
PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC
Platform=HPD
PROCESSOR_ARCHITECTURE=AMD64
PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 60 Stepping 3, GenuineIntel
PROCESSOR_LEVEL=6
PROCESSOR_REVISION=3c03
ProgramData=C:\ProgramData
ProgramFiles=C:\Program Files
ProgramFiles(x86)=C:\Program Files (x86)
ProgramW6432=C:\Program Files
PSModulePath=C:\windows\system32\WindowsPowerShell\v1.0\Modules\
PUBLIC=C:\Users\Public
SESSIONNAME=Console
SystemDrive=C:
SystemRoot=C:\windows
TEMP=C:\Users\UTENTE~1\AppData\Local\Temp
TMP=C:\Users\UTENTE~1\AppData\Local\Temp
USERDOMAIN=TOW8SBV50
USERDOMAIN_ROAMINGPROFILE=TOW8SBV50
USERNAME=utente locale
USERPROFILE=C:\Users\utente locale
windir=C:\windows

Hydra internal environment:

MPIR_CVAR_NEMESIS_ENABLE_CKPOINT=1
GFORTRAN_UNBUFFERED_PRECONNECTED=y

User set environment:

OMP_STACKSIZE=16M

Proxy information:
*********************
  [1] proxy: tow7prosbv106 (2 cores)
  Exec list: C:\Program Files\PyroSim 2015\fds/fds.exe (2 processes); 

  [2] proxy: tow8sbv50 (1 cores)
  Exec list: C:\Program Files\PyroSim 2015\fds/fds.exe (1 processes);

[mpiexec@TOW8SBV50] Timeout set to -1 (-1 means infinite) [mpiexec@TOW8SBV50] Got a control port string of TOW8SBV50:8670

Proxy launch args: C:\Program Files\PyroSim 2015\fds\pmi_proxy --control-port TOW8SBV50:8670 --debug --pmi-connect lazy-cache --pmi-aggregate -s 0 --rmk user --launcher service --demux select --pgid 0 --enable-stdin 1 --retries 10 --control-code 6635 --usize -2 --proxy-id

Arguments being passed to proxy 0: --version 3.1.2 --iface-ip-env-name MPIR_CVAR_CH3_INTERFACE_HOSTNAME --hostname tow7prosbv106 --global-core-map 0,2,3 --pmi-id-map 0,0 --global-process-count 3 --auto-cleanup 1 --pmi-kvsname kvs_2020_0 --pmi-process-mapping (vector,(0,1,2),(1,1,1)) --binding omp=4 --topolib ipl --ckpointlib blcr --ckpoint-prefix /tmp --ckpoint-preserve 1 --ckpoint off --ckpoint-num -1 --global-inherited-env 49 'ADSK_MAXDES_x64_2014=C:\Program Files\Autodesk\3ds Max Design 2014\' 'ALLUSERSPROFILE=C:\ProgramData' 'APPDATA=C:\Users\utente locale\AppData\Roaming' 'asl.log=Destination=file' 'CM2014DIR=C:\Program Files (x86)\Common Files\Autodesk Shared\Materials\' 'CM2015DIR=C:\Program Files (x86)\Common Files\Autodesk Shared\Materials\' 'CommonProgramFiles=C:\Program Files\Common Files' 'CommonProgramFiles(x86)=C:\Program Files (x86)\Common Files' 'CommonProgramW6432=C:\Program Files\Common Files' 'COMPUTERNAME=TOW8SBV50' 'ComSpec=C:\windows\system32\cmd.exe' 'FP_NO_HOST_CHECK=NO' 'HOMEDRIVE=C:' 'HOMEPATH=\Users\utente locale' 'ILBDIR=C:\Program Files (x86)\Common Files\Autodesk Shared\Materials\' 'ILLDIR=C:\Program Files (x86)\Common Files\Autodesk Shared\Materials\' 'ILMDIR=C:\Program Files (x86)\Common Files\Autodesk Shared\Materials\' 'LOCALAPPDATA=C:\Users\utente locale\AppData\Local' 'LOGONSERVER=\TOW8SBV50' 'MPICH_PORT_RANGE=8670:8690' 'MPIEXEC_PORT_RANGE=8670:8690' 'MSMPI_BIN=C:\Program Files\Microsoft MPI\Bin\' 'NUMBER_OF_PROCESSORS=8' 'OMP_NUM_THREADS=4' 'OnlineServices=Online Services' 'OS=Windows_NT' 'Path=C:\Program Files\Microsoft MPI\Bin\;C:\shortcuts;C:\FDS6\bin;C:\ProgramData\Oracle\Java\javapath;c:\Program Files (x86)\Intel\iCLS Client\;c:\Program Files\Intel\iCLS Client\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\IPT;c:\Program Files\Intel\WiFi\bin\;c:\Program Files\Common Files\Intel\WirelessCommon\;C:\Program Files\Hewlett-Packard\SimplePass\;C:\Program Files (x86)\QuickTime\QTSystem\;C:\Program Files\Common Files\Autodesk Shared\;C:\Program Files (x86)\Autodesk\Backburner\;C:\Program Files\PyroSim 2015\fds' 'PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC' 'Platform=HPD' 'PROCESSOR_ARCHITECTURE=AMD64' 'PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 60 Stepping 3, GenuineIntel' 'PROCESSOR_LEVEL=6' 'PROCESSOR_REVISION=3c03' 'ProgramData=C:\ProgramData' 'ProgramFiles=C:\Program Files' 'ProgramFiles(x86)=C:\Program Files (x86)' 'ProgramW6432=C:\Program Files' 'PSModulePath=C:\windows\system32\WindowsPowerShell\v1.0\Modules\' 'PUBLIC=C:\Users\Public' 'SESSIONNAME=Console' 'SystemDrive=C:' 'SystemRoot=C:\windows' 'TEMP=C:\Users\UTENTE~1\AppData\Local\Temp' 'TMP=C:\Users\UTENTE~1\AppData\Local\Temp' 'USERDOMAIN=TOW8SBV50' 'USERDOMAIN_ROAMINGPROFILE=TOW8SBV50' 'USERNAME=utente locale' 'USERPROFILE=C:\Users\utente locale' 'windir=C:\windows' --global-user-env 1 'OMP_STACKSIZE=16M' --global-system-env 2 'MPIR_CVAR_NEMESIS_ENABLE_CKPOINT=1' 'GFORTRAN_UNBUFFERED_PRECONNECTED=y' --proxy-core-count 2 --mpi-cmd-env C:\Program Files\PyroSim 2015\fds\mpiexec.exe -v -gwdir \TOW8SBV50\abc -gpath C:\Program Files\PyroSim 2015\fds -genv OMP_STACKSIZE 16M -host tow7prosbv106 -n 2 fds.exe \TOW8SBV50\abc\A2.fds : -host tow8sbv50 -n 1 fds.exe \TOW8SBV50\abc\A2.fds --exec --exec-appnum 0 --exec-proc-count 2 --exec-local-env 0 --exec-wdir \TOW8SBV50\abc --exec-args 2 C:\Program Files\PyroSim 2015\fds/fds.exe \TOW8SBV50\abc\A2.fds

Arguments being passed to proxy 1: --version 3.1.2 --iface-ip-env-name MPIR_CVAR_CH3_INTERFACE_HOSTNAME --hostname tow8sbv50 --global-core-map 0,1,3 --pmi-id-map 0,2 --global-process-count 3 --auto-cleanup 1 --pmi-kvsname kvs_2020_0 --pmi-process-mapping (vector,(0,1,2),(1,1,1)) --binding omp=4 --topolib ipl --ckpointlib blcr --ckpoint-prefix /tmp --ckpoint-preserve 1 --ckpoint off --ckpoint-num -1 --global-inherited-env 49 'ADSK_MAXDES_x64_2014=C:\Program Files\Autodesk\3ds Max Design 2014\' 'ALLUSERSPROFILE=C:\ProgramData' 'APPDATA=C:\Users\utente locale\AppData\Roaming' 'asl.log=Destination=file' 'CM2014DIR=C:\Program Files (x86)\Common Files\Autodesk Shared\Materials\' 'CM2015DIR=C:\Program Files (x86)\Common Files\Autodesk Shared\Materials\' 'CommonProgramFiles=C:\Program Files\Common Files' 'CommonProgramFiles(x86)=C:\Program Files (x86)\Common Files' 'CommonProgramW6432=C:\Program Files\Common Files' 'COMPUTERNAME=TOW8SBV50' 'ComSpec=C:\windows\system32\cmd.exe' 'FP_NO_HOST_CHECK=NO' 'HOMEDRIVE=C:' 'HOMEPATH=\Users\utente locale' 'ILBDIR=C:\Program Files (x86)\Common Files\Autodesk Shared\Materials\' 'ILLDIR=C:\Program Files (x86)\Common Files\Autodesk Shared\Materials\' 'ILMDIR=C:\Program Files (x86)\Common Files\Autodesk Shared\Materials\' 'LOCALAPPDATA=C:\Users\utente locale\AppData\Local' 'LOGONSERVER=\TOW8SBV50' 'MPICH_PORT_RANGE=8670:8690' 'MPIEXEC_PORT_RANGE=8670:8690' 'MSMPI_BIN=C:\Program Files\Microsoft MPI\Bin\' 'NUMBER_OF_PROCESSORS=8' 'OMP_NUM_THREADS=4' 'OnlineServices=Online Services' 'OS=Windows_NT' 'Path=C:\Program Files\Microsoft MPI\Bin\;C:\shortcuts;C:\FDS6\bin;C:\ProgramData\Oracle\Java\javapath;c:\Program Files (x86)\Intel\iCLS Client\;c:\Program Files\Intel\iCLS Client\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\IPT;c:\Program Files\Intel\WiFi\bin\;c:\Program Files\Common Files\Intel\WirelessCommon\;C:\Program Files\Hewlett-Packard\SimplePass\;C:\Program Files (x86)\QuickTime\QTSystem\;C:\Program Files\Common Files\Autodesk Shared\;C:\Program Files (x86)\Autodesk\Backburner\;C:\Program Files\PyroSim 2015\fds' 'PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC' 'Platform=HPD' 'PROCESSOR_ARCHITECTURE=AMD64' 'PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 60 Stepping 3, GenuineIntel' 'PROCESSOR_LEVEL=6' 'PROCESSOR_REVISION=3c03' 'ProgramData=C:\ProgramData' 'ProgramFiles=C:\Program Files' 'ProgramFiles(x86)=C:\Program Files (x86)' 'ProgramW6432=C:\Program Files' 'PSModulePath=C:\windows\system32\WindowsPowerShell\v1.0\Modules\' 'PUBLIC=C:\Users\Public' 'SESSIONNAME=Console' 'SystemDrive=C:' 'SystemRoot=C:\windows' 'TEMP=C:\Users\UTENTE~1\AppData\Local\Temp' 'TMP=C:\Users\UTENTE~1\AppData\Local\Temp' 'USERDOMAIN=TOW8SBV50' 'USERDOMAIN_ROAMINGPROFILE=TOW8SBV50' 'USERNAME=utente locale' 'USERPROFILE=C:\Users\utente locale' 'windir=C:\windows' --global-user-env 1 'OMP_STACKSIZE=16M' --global-system-env 2 'MPIR_CVAR_NEMESIS_ENABLE_CKPOINT=1' 'GFORTRAN_UNBUFFERED_PRECONNECTED=y' --proxy-core-count 1 --mpi-cmd-env C:\Program Files\PyroSim 2015\fds\mpiexec.exe -v -gwdir \TOW8SBV50\abc -gpath C:\Program Files\PyroSim 2015\fds -genv OMP_STACKSIZE 16M -host tow7prosbv106 -n 2 fds.exe \TOW8SBV50\abc\A2.fds : -host tow8sbv50 -n 1 fds.exe \TOW8SBV50\abc\A2.fds --exec --exec-appnum 1 --exec-proc-count 1 --exec-local-env 0 --exec-wdir \TOW8SBV50\abc --exec-args 2 C:\Program Files\PyroSim 2015\fds/fds.exe \TOW8SBV50\abc\A2.fds

[mpiexec@TOW8SBV50] Launch arguments: C:\Program Files\PyroSim 2015\fds\pmi_proxy --control-port TOW8SBV50:8670 --debug --pmi-connect lazy-cache --pmi-aggregate -s 0 --rmk user --launcher service --demux select --pgid 0 --enable-stdin 1 --retries 10 --control-code 6635 --usize -2 --proxy-id 0 [mpiexec@TOW8SBV50] Launch arguments: C:\Program Files\PyroSim 2015\fds\pmi_proxy --control-port TOW8SBV50:8670 --debug --pmi-connect lazy-cache --pmi-aggregate -s 0 --rmk user --launcher service --demux select --pgid 0 --enable-stdin 1 --retries 10 --control-code 6635 --usize -2 --proxy-id 1

END log

drjfloyd commented 9 years ago

We cannot help you if you are running FDS through pyrosim's installation. We don't have any control over their installation process. Try fully uninstalling FDS and Pyrosim. Then re-install FDS only using our precompiled binaries and try to launch an mpi job.

bwklein commented 9 years ago

Please email support@thunderheadeng.com for assistance with the cluster run option in PyroSim.

Best Regards, -Bryan Klein Thunderhead Engineering

On Tue, Oct 13, 2015, 07:08 Jason Floyd notifications@github.com wrote:

We cannot help you if you are running FDS through pyrosim's installation. We don't have any control over their installation process. Try fully uninstalling FDS and Pyrosim. Then re-install FDS only using our precompiled binaries and try to launch an mpi job.

— Reply to this email directly or view it on GitHub https://github.com/firemodels/fds-smv/issues/2950#issuecomment-147725492 .