search

firemodels / fds

Fire Dynamics Simulator
https://pages.nist.gov/fds-smv/
Other
673 stars 627 forks source link

parallel tunnel calc gives different results than serial calc #794

Closed gforney closed 9 years ago

gforney commented 9 years ago 
Please complete the following lines...

Application Version:5.3
SVN Revision Number:3182
Compile Date:
Operating System:

Describe details of the issue below:
Hello,
I run the same tunnel case in serial and parallel and get different 
results.
The parallel version was run on a cluster at University.
The serial version was run on my PC. Both are working with FDS 5.3.

Thanks for explaining me where the problem is. I read a while ago that 
there was a problem (http://groups.google.com/group/fds-
smv/browse_thread/thread/51b2a0613a9023b7/551311f4bfcfae14?
lnk=gst&q=pressure_correction&fwc=2&pli=1)
with the pressure correction term and i honestly think this has not been 
solved yet.

best regards,
Xavier

Original issue reported on code.google.com by deckers_xavier@hotmail.com on 2009-07-21 10:19:24

gforney commented 9 years ago 
Have you used the latest version of FDS (5.3.1)? Also, can you demonstrate via a 
picture or plot how the cases are different? This is a complicated case and it will

take us time to reproduce the problem.

Original issue reported on code.google.com by mcgratta on 2009-07-21 13:16:52

gforney commented 9 years ago 
yes it was with FDS 5.3.1.
In attachment you can see some differences

Original issue reported on code.google.com by deckers_xavier@hotmail.com on 2009-07-21 14:31:25

gforney commented 9 years ago 
Do both the serial and parallel cases use the same input file with multiple meshes?

Original issue reported on code.google.com by mcgratta on 2009-07-21 16:49:45

gforney commented 9 years ago 
yes, both are using the same input file with 6 meshes.

The input file is the memorial tunnel with natural ventilation (inclination of 3.2%)
This is a part of my masters thesis.

Original issue reported on code.google.com by deckers_xavier@hotmail.com on 2009-07-21 16:58:05

gforney commented 9 years ago 
Randy -- can you run this case with the latest version of the serial and the 
parallel. Same input file, but one run with serial, the other with MPI. My computer

is still down, but I can access email.

Original issue reported on code.google.com by mcgratta on 2009-07-21 18:28:46

gforney commented 9 years ago 
I ran both cases with PRESSURE_CORRECTION=F and have different results after about
600 sec.  Qualitatively, the serial case looks stable and the mpi case oscillates
more and looks unphysical.  This lead me to believe that the problem is in the
message passing and not the PC scheme.

Original issue reported on code.google.com by randy.mcdermott on 2009-07-23 12:54:47

gforney commented 9 years ago 
Remove BAROCLINIC=.TRUE. and PRESSURE_CORRECTION=.TRUE.

The BAROCLINIC option can be fragile on multiple meshes, and we plan to add some 
warnings to the Guide about this. 

As Randy says, we will probably remove the PRESSURE_CORRECTION option. It was 
originally used to stabilize calculations like yours, but we have found in some 
cases that it can lead to spurious behavior.

Also, restore RADIATION to the calculation. 

I ran the case in serial and parallel mode as described above and the results seem

reasonable and do not differ noticeably.

I am going to mark this as "Fixed" even though the only thing we have done is to add

warnings to the Guide. The BAROCLINIC option is a long-term issue that we probably

will not solve now.

Original issue reported on code.google.com by mcgratta on 2009-07-25 18:25:55

gforney commented 9 years ago 
Xavier,

I talked with Kevin and I think we agree that there are still questions about this
case that go beyond BAROCLINIC (BARO) and PRESSURE_CORRECTION (PC).  So, I am
changing the status to OnHold and we will try to resolve this issue for FDS 5.5.

Before I get to your questions below, I want to emphasize that there is a more
fundamental problem in that the serial and parallel cases, which should be providing
the same results, in fact do not.  Any other "fix" to this problem would simply be
masking this more fundamental flaw.

About the recommendation not to use BARO in parallel calcs, personally I always use
BARO=T.  This is part of the default set of options for FDS6=T
(http://groups.google.com/group/fds-smv/browse_thread/thread/7bb35719784756b7#). I
think what Kevin is saying is that BARO may be part of the instability problem you
are seeing (I emphasize "may be") and since there is a bit of a history of BARO
causing problems, then he wants to put out a more explicit warning.  However, we are
working hard toward getting FDS into a state where such instabilities do not occur
and I agree with you that the physics of the baroclinic torque needs to be included.

Regarding the use of PC... First, the only write up is in the Tech Guide.  It is
sufficiently spelled out there.  Second, this scheme is usually helpful when the flow
is predominantly in one direction (think pipe flow) and normal to the mesh face.  If
the flow is parallel with the mesh face, PC is terribly ill-conditioned.  So, it is
not really that PC is a bad scheme.  When you know how to use it, it can be very
helpful and fast.  But it is just not as robust as we would like.  To answer your
question about how we can ensure volume (or mass) conservation across mesh interfaces
without such a scheme, we cannot.  Therefore we are pursuing alternative approaches.
 The most robust (but most expensive) approach is a global iterative solver and this
is being developed (is quite close to completion actually) by Susan Kilian in
Germany.  I also have some other ideas that may leverage a pseudo-compressible
formulation, but have not had time to work on this too much.

So, I hope this convinces you that we are not giving up on your problem.  I am
copying Susan Kilian on this thread, in case she feels like her solver is at a point
where it could be used here.  However, like I said, there is something we do not
understand about the parallel algorithm, whether it is a bug in the message passing
or simply an order of operations issue, and fixing this--in my opinion--is the first
step toward getting your case resolved.

Best,
Randy

Hello mister McDermott,

I posted a bug report on fds-issue forum about different results from parallel and
serial calcs in FDS. As this is part of my masters thesis as a fire safety engineer
I
would appreciate if you could explain me (or reference me to an article) a little bit
more about this pressure_correction term that is supposed to account for the volume
loss at a mesh boundary in parallel tunnel calcs. I understand that the new
recommendation is not to use baroclinic generation of vorticity in parallel calcs,
but I don’t see the link between both. Also if you will not use a pressure correction
term any more, how can you solve the volume loss issue at mesh boundaries?

I do think this baroclinic generation of vorticity can play an important role when
having to accurately describe the distance of backlayering in a tunnel with high
mechanical ventilation and high HRR.

Thanks for your reply,

Best regards,

Xavier

Original issue reported on code.google.com by randy.mcdermott on 2009-07-28 13:30:33

gforney commented 9 years ago 
Closing. No activity in 2 years.

Original issue reported on code.google.com by randy.mcdermott on 2011-08-26 19:42:15