Problem with resuming DSMC case

[jaimakhijaa]

Hello, I have a DSMC case which I am unable to resume running the case after I stop it once. The case was set up without any errors, right from meshing, initializing, decomposing to run the case in parallel, renumbering the mesh, and run the case itself using mpirun. The initial run was for a flow time of 0.28 seconds, where I stopped the case by modifying the controlDict file by changing the endTime to writeNow. When I resumed the case by calling the mpirun command again (not modifying anything within the processor folders), I faced an error of floating-point exception. This is an inconvenience as restarting the simulation from time = 0 renders the initial run data useless. I have also attached a log file showing the error. Any help with this would be really helpful.

Regards, Jai log.txt

[timteichmann]

Hello Jai, One possible reason I can think of is that the controlDict is also present in the decomposed processor folder. Can you check if this is the case (if yes there will be one or more files at processorN/system/controlDict). However, from personal experience I can recommend to set up the simulation once and then let it run without changing any parameters (except the starting of the averaging ofc). Many of the settings in the original OpenFOAM are not respected by dsmcFoam+ and some of them might lead to crashes like the one you described.

From your log I can tell that it is probably failing in

hyStrath/src/lagrangian/molecularDynamics/general/time/timeDataMeas/timeDataMeas.C line 361 ff.

Maybe this helps you locate the exact cause of this bug.

[jaimakhijaa]

Hello Tim, Thank you for responding to my query. I did have a look in multiple processor folders and other than the time steps, the folders present are 'constant', 'boundaries', and 'fieldMeasurements'. So I don't think the controlDict file is present there and therefore, not the cause of the error. I shall look into the file whose directory you have shared. Could you tell me what you saw in the log file that you suggested I have a look in the timeDataMeas.C file? Neither the file name nor the directory was mentioned in the log file.

Regards, Jai

[vincentcasseau]

Hi Jai, Could you please provide a minimal, reproducible test case? Regards, Vincent

[vincentcasseau]

Here's a summary of where we are:

• [x] allow developers to set the random seed (testing, debugging, regression only): c3a750e254d487d784bce19e9d39b12646e41142
• [x] remove regression cases and leave only 5 test cases (axisymmetric blunt cylinder, orion capsule, couette flow, hypersonic corner, supersonic flat plate), as discussed in #71;
• [x] update test cases: sampling at time: #55, load balancing until time: #56 (not used, but entry added in loadBalancingDict): e08f1338ecc28306bce1f57b583f9ce523554fc0
• [x] fix the bug in the calculation of the mixture vibrational temperature: e20c91e45c22dae263b748dbbead08989f94ac52
• [x] fix the dsmcN* and dsmcNMean* fields at boundaries (zeroGradient): e20c91e45c22dae263b748dbbead08989f94ac52
• [x] update results folder for the axisymmetric blunt cylinder, hypersonic corner, supersonic flat plate test cases: 767e6b2f424be155271e222c41c7eefad3226e9e
• [x] provide results folder for the couette test case: be4718c1adf5a4d1983802ddfb31fea222f99e08
• [x] averagingAcrossManyRuns: fix the bug that prevents from resuming parallel DSMC simulations: ec6229e96c7a5d81c91a6bbbed3949f23cec4b99
  • the simulation must to be resumed using the same number of processors
  • If you're planning on resuming your simulation in the future, here's what you can do
    • reconstrucPar -latestTime
    • delete everything but the time/uniform/ sub-folder in the processors directories and store them in backup-processors
    • to resume the simulation: cp -r backup-processors/processor* . && decomposePar -force -latestTime
• [ ] update loadBalancingDict and fieldPropertiesDict if entries in #55 and #56 are edited during the execution

[jaimakhijaa]

Yeah sure, have zipped it in here. I have removed few files to upload the files within the limit of GitHub (10 MB). Also, I would like to know if there are any differences between how rhoNMean is calculated in dsmcFoam and dsmcFoam+. I cannot calculate macroscopic variables such as velocity, temperature, and pressure in dsmcFoam but dsmcFoam+ can calculate the same.

Regards, Jai

hollow_cylinder_flare_dsmcFoam+.zip

[timteichmann]

Thanks for the fixes Vincent! Also apologies that my changes did apparently create issues with the hybrid code after all. Do you think that the bug regarding averagingAcrossManyRuns is related with #55 or #56?

@jaimakhijaa: I do not understand your question regarding the number density. How is this related to the velocity etc.? You can compare the calculation routines directly. The source code is available in both cases. In case of dsmcFoam+ you can find it here: hyStrath/src/lagrangian/dsmc/macroscopicProperties/derived/combined/dsmcVolFields/dsmcVolFields.C, in case of dsmcFoam OpenFOAM-8/src/lagrangian/DSMC/clouds/Templates/DSMCCloud/DSMCCloud.C (e.g. for v8 of OpenFOAM). Hope this helps a little.

[vincentcasseau]

Not at all Tim. It was an excellent step in the right direction. This file, and many others that you flagged as deprecated (WIP), are/were chaotic. averagingAcrossManyRuns actually works in serial. I compared the current implementation to the one before #55 and I can't see a problem. It makes me wonder if it has ever been tested in parallel... I will check out dsmcFoam+ as of last September and test it later this week.

PS: Thanks Jai for the zip!

[vincentcasseau]

In the next commit, I am planning on renaming a few fields that are unnecessarily long:

• UMean -> U
• translationalT -> Ttra
• rotationalT -> Trot (printed only if this mode is included)
• vibrationalT -> Tvib (printed only if this mode is included)
• electronicT -> Telec (printed only if this mode is included)
• overallT -> Tov (printed if there is at least one internal energy mode included)
• variableHardSphereMeanFreePath -> mfp
• mfpCellRatio -> mfpToDx
• mctTimeStepRatio -> mctToDt
• meanCollisionTime -> mct
• meanCollisionSeparation -> mcs
• surfaceHeatTransfer -> wallHeatFlux (harmonisation with CFD)
• surfaceShearStress -> wallShearStress (harmonisation with CFD)

Does anyone have any objection? Thanks

Update: done in ec6229e96c7a5d81c91a6bbbed3949f23cec4b99

[vincentcasseau]

Hi Jai, This is fixed. Please have a look at this comment https://github.com/vincentcasseau/hyStrath/issues/73#issuecomment-840176373 I'll close this issue once all checkboxes are ticked. Thanks, Vincent

[jaimakhijaa]

Thank you for your help Vincent and Tim! Could you share the paper which you've used for validating the Orion CEV case and building the tutorial? I couldn't find it in your website.

[vincentcasseau]

On the Orion capsule, we have @rophys's PhD thesis: R. C. Palharini, 09/2014: Atmospheric Reentry Modelling Using an Open-Source DSMC Code, pp. 86-99 but the altitude and the angle of attack look different. Do you have a conf. paper or journal article on the Orion, Rodrigo?

[rophys]

Hi Vicent,

The Orion case is based on a paper published by Moss in 2006. Please, find it attached.

Cheers,

Rodrigo

On Tue, 25 May 2021 at 08:47, Vince @.***> wrote:

On the Orion capsule, we have @rophys https://github.com/rophys's PhD thesis: R. C. Palharini, 09/2014: Atmospheric Reentry Modelling Using an Open-Source DSMC Code https://github.com/vincentcasseau/hyStrath/blob/master/doc/PhDthesis-rodrigopalharini.pdf, pp. 86-99 but the altitude and the angle of attack look different. Do you have a conf. paper or journal article on the Orion, Rodrigo?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/vincentcasseau/hyStrath/issues/73#issuecomment-847839277, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALMNNZKOCOLJE5SD6QIDLKLTPOL6XANCNFSM43OYZ4UQ .

[jaimakhijaa]

Hi @rophys, Is this the paper you're talking about? https://arc.aiaa.org/doi/10.2514/6.2006-8081 Can't find any attachment in your comment.

[vincentcasseau]

It looks like it: R. C. Palharini, 09/2014: Atmospheric Reentry Modelling Using an Open-Source DSMC Code, Ref 193 [193] J. N. Moss, K. A. Boyles, and F. A. Greene. Orion aerodynamics for hypersonic free molecular to continuum conditions. In 14th AIAA/AHI International Space Planes and Hypersonic Systems and Technologies Conference. Canberra, Australia, 2006.

The attachment didn't work because he replied by email. However, please do not attach papers here for copyright reasons.

[rophys]

Hi Jai,

Yes, it is the correct paper. You will find all the information that you need in it. In addition, you can have a look at my PhD thesis https://pureportal.strath.ac.uk/en/publications/atmospheric-reentry-modelling-using-an-open-source-dsmc-code .

On Wed, 26 May 2021 at 08:50, Vince @.***> wrote:

It looks like it: R. C. Palharini, 09/2014: Atmospheric Reentry Modelling Using an Open-Source DSMC Code https://github.com/vincentcasseau/hyStrath/blob/master/doc/PhDthesis-rodrigopalharini.pdf, Ref 193 [193] J. N. Moss, K. A. Boyles, and F. A. Greene. Orion aerodynamics for hypersonic free molecular to continuum conditions. In 14th AIAA/AHI International Space Planes and Hypersonic Systems and Technologies Conference. Canberra, Australia, 2006.

The attachment didn't work because he replied by email. However, please do not attach papers here for copyright reasons.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/vincentcasseau/hyStrath/issues/73#issuecomment-848742059, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALMNNZJIABRVQY2DWD4APZ3TPTVCFANCNFSM43OYZ4UQ .

[vincentcasseau]

I will open a new issue and close this one for clarity.

[jaimakhijaa]

Hello @vincentcasseau I am facing an issue in starting a new simulation now using dsmcFoam+. I have attached the log file for your reference. The case setup was modified to run the case using staticFvMesh and the error of floating point exception was generated. Any form of help would be appreciated.

Regards, Jai log.txt

[vincentcasseau]

Hi Jai,

The case setup was modified to run the case using staticFvMesh

Why would you like to do this?

[jaimakhijaa]

Hi Jai,

Why would you like to do this?

I did it for faster simulation time as this is a trial run, not the final run. I also did try running with the default dynamicMeshDict but did face the same floating point exception error.

[vincentcasseau]

Are the tutorial cases running? i.e., is your code correctly installed? There are no useful info in the log.

[jaimakhijaa]

I just tried running the orion107kmNR tutorial and did get an error in running the case using the Allrun script. Didn't face any error during installation though. Any idea what might have caused it? Attached the log for the tutorial case in case it helps. log_dsmcFoam+.txt

[vincentcasseau]

Change dir to username-v1706/src/ and username-v1706/applications/ and type in both wclean all Then, sync the DSMC module in hyStrath/ using the install script.

[jaimakhijaa]

I just tried what you've mentioned above and I'm getting the same error for the orion107kmNR tutorial. Didn't face any error during synchronization though. Any idea what might have caused it? Attached the log for the tutorial case in case it helps. log_orion107kmNR.txt

[vincentcasseau]

Hi, I reinstalled everything from scratch at my end and it works. As anticipated, there are no issues with the current commit. So you should do the same:

1. delete src/, applications/ and platforms/ in $WM_PROJECT_USER_DIR;
2. run git pull
3. install what you need Thanks, Vincent