How to confirm the total simulation step

[yangtianyong]

Hello: I expand the simulation area and leave the other conditions the same as before, the simulation_duration=2.0,dt=0.0005,and I think the total simulation step=4000,however,when I run the base_line.py or single_runner.py（First I run make_mesh.py）,there are more than 4000 steps ,would you like to offer any suggestions ? Thank you,best wishes! Simu_lift | Ep N: 0, step: 4000, Rec Area: 0.0547, drag: -0.1108, lift: 0.0021 Simu_lift | Ep N: 0, step: 4100, Rec Area: 0.0542, drag: -0.1109, lift: 0.0018 Simu_lift | Ep N: 0, step: 4200, Rec Area: 0.0544, drag: -0.1110, lift: 0.0010 Simu_lift | Ep N: 0, step: 4300, Rec Area: 0.0535, drag: -0.1113, lift: -0.0003 Simu_lift | Ep N: 0, step: 4400, Rec Area: 0.0550, drag: -0.1116, lift: -0.00

[jerabaul29]

Hi!

A few notes:

• It looks like some other things were also modified in the simulation (if I am not wrong?), because the value of the drag is much lower than anything I have seen before. Did you add some jets that can perform blowing, modify some other parameters, changed the Reynolds number, or something else? ;) (note that if you are doing some form of research using the code you do not really need to give me a specific answer, but I would be interested to get confirmation that you also modified some other stuff).

• There are many parameters that can influence the duration of the simulation, as the simulation can stop either using the maximum number of control flag (nb_actuations here):

https://github.com/jerabaul29/Cylinder2DFlowControlDRL/blob/6eb30aa4856c86a40a800b75b117a95fd2c68028/Cylinder2DFlowControlWithRL/ANN_controlled_flow_learning/perform_learning.py#L126

or the terminal condition itself:

https://github.com/jerabaul29/Cylinder2DFlowControlDRL/blob/6eb30aa4856c86a40a800b75b117a95fd2c68028/Cylinder2DFlowControlWithRL/Env2DCylinder.py#L823-L836

I think in this version of the code, we always set terminal to False, so the simulation never ends because of the terminal flag in the execute function, but always instead because of hitting the max_episode_timesteps in the runner.

Note that the max_episode_timesteps value is related to the number of steps, but this is not a 1 to 1 correspondance as the network does not update the control at each time step:

https://github.com/jerabaul29/Cylinder2DFlowControlDRL/blob/6eb30aa4856c86a40a800b75b117a95fd2c68028/Cylinder2DFlowControlWithRL/Env2DCylinder.py#L779

Does that make sense? :)

[jerabaul29]

PS: Ok, I guess by 'extending the simulation area' you may mean 'making it wider', then this may explain for the change in drag coefficient ;)

[yangtianyong]

Yes,I mean make it wider.And,those parameters you talked about can explain my confusion! Thank you!

[jerabaul29]

Ok. The code is a bit messy, you may need to fix / rewrite some parts to make it more fitted to cases outside of the two cases we shipped.

[jerabaul29]

By the way, I am a bit curious about what you want to use our code for. Are you a student / researcher / other? If you want more detailed help and comments we can discuss about a more formal collaboration / working together on something. Of course I perfectly understand if you are not interested / this does not fit you.

[yangtianyong]

Hello,I am a student .The main purpose that use your code is to learn Deep reinforcement learning.I mean,you are the first time to use deep neural networks and reinforcement learning to the field of active flow control.And I want to know the details how reinforcement learning to train deep neural networks.Moreover,how does the code achieve those. yes,I am interested in this work,also I want to use this idea to something else.However,I am a new learner of this field,and I normally feel confused to learn Deep reinforcement learning,case it's to difficult for me ,and few material to get! Still,thank you for your help all the time!

[jerabaul29]

That sounds good! Good luck with your applications, hope you get successful! No worries, happy to help DRL diffuse to other people!

[jerabaul29]

PS: are you a PhD student or masters student or other? :)

[yangtianyong]

A masters student （my work maybe involved this field）,you are really good at the Deep reinforcement learning.Would you have any great suggestions to learn about it?

[jerabaul29]

I am not that good, but I am trying to be ^^

Have you been through the materials here?

https://github.com/jerabaul29/Cylinder2DFlowControlDRL/blob/master/LearningMaterial/README.md

Otherwise mostly googling around, reading code and ArXiv papers should help you.

[yangtianyong]

yes ,I have been read those materials .In addition ,I usually surf the Internet to search some basic knowledge(but can't form a system).

[jerabaul29]

Good!

Yes, the field is very new so information is still scattered. This is both the exciting and the annoying part about working in a novel, modern, booming field.

Closing this issue as it seems everything is working as you want now.