Exawind / nalu-wind

Solver for wind farm simulations targeting exascale computational platforms
https://nalu-wind.readthedocs.io
Other
119 stars 81 forks source link

output channels with Cp, Ct from openFAST using Nalu-wind actuator disc simulations #625

Closed tchatte3 closed 4 years ago

tchatte3 commented 4 years ago

Hi Nalu-wind team,

Even though this question is tied to openFAST simulations, I am asking this question here for better relevance. I have some questions related to the interpretation of the results. I am running actuator disc simulations using Nalu-wind (latest build). In the turbine section, you see that I set t_max = 0.625, after the completion of the simulations, when I open the output file of openFAST (.out file), I would get plots similar to what I show below. (different colors are different mesh settings). My question is how to interpret the results? I read the nalu wind manual, as well the the openFAST C++ API guide (https://openfast.readthedocs.io/en/master/source/user/cppapi/) but it is still confusing to me.

My actual NALU-wind simulation is running for some ~ O(10000) seconds, and if I set t_max = 0.625 secs in the actuator section for the openFAST C++ API, is the openFAST stopping after t_max? Does not seem like it, since I can plot the actuator_force and see it is still there till the end of the simulation. Any feedback or help?

actuator:
      type: ActDiskFASTNGP
      search_method: stk_kdtree
      search_target_part: [fluid_part,fluid_part.pyramid_5._urpconv,fluid_part.tetrahedron_4._urpconv]

      n_turbines_glob: 1
      dry_run:  False
      debug:    False
      simStart: init
      t_start: 0.0
      t_max:    0.625
      dt_fast: 0.00625
      n_every_checkpoint: 100

      Turbine0:
        num_force_pts_blade: 40
        num_force_pts_tower: 20
        nacelle_cd: 1.0
        nacelle_area: 6.53
        air_density: 1.0000
        epsilon: [4.5, 4.5, 4.5]
        turbine_base_pos: [2500.0, 1450.0, 0.0 ]
        turbine_hub_pos:  [2500.0, 1450.0, 80.00 ]
        restart_filename: "blah"
       fast_input_filename: "nrel5mw.fst.in"
        turb_id:  1
        turbine_name: machine_one

image

image

sayerhs commented 4 years ago

@tchatte3 I am pretty sure that I remember @gantech mentioning that OpenFAST t_Max has to be greater than how long the Nalu simulations are running, otherwise the results would not be correct. Ganesh can you confirm?

tchatte3 commented 4 years ago

@sayerhs Thanks.

lawrenceccheung commented 4 years ago

Hi @tchatte3,

So if you're starting a simulation from scratch, the actuator disk and actuator line code will happily keep advancing OpenFAST as long as necessary regardless of t_max, and I think the text output in the *.out should be continue to be written as long as needed (similar to what you might have observed).

However, the issues might come when you want to restart the openFAST calculation or look at the other data files being written out. I did a quick look through OpenFAST.cpp, and the t_max parameter is used to set up the appropriate dimensions of the hdf5 velocity file.

There's also this comment in the FAST_Prog.cpp driver code which warns about the relationship between t_max in the Nalu-Wind input file and the tMax in the FAST fst input file:

// tMax is the total duration to which you want to run FAST. This should be the same or greater than the max time given in the FAST fst file. Choose this carefully as FAST writes the output file only at this point if you choose the binary file output.

So the best practice is to probably set t_max to be greater than the Nalu-Wind runtime, unless you never restart or need to use the other FAST output files. @gantech might be able to confirm this or point out other issues.

Lawrence

tchatte3 commented 4 years ago

@lawrenceccheung This makes perfect sense. Thanks for clarifying. As a sidenote, I think it is a bit confusing, since to run the simulations 3 variables need to be considered, termination_time in the Time integrator class in Nalu-wind, t_max in the glue code C++ API and tMax in .fst input file.

Best Regards,

lawrenceccheung commented 4 years ago

@tchatte3 I agree, this is another example where things could be better integrated with OpenFAST, and have less redundancy in the user input. It's been mentioned before, but some of the actions will require a coordinated update in both the OpenFAST and Nalu-Wind codes.

Lawrence

tchatte3 commented 4 years ago

Hi @lawrenceccheung , @sayerhs @gantech ,

Following the thread discussion, when I run my simulations I make sure that times in openFAST and NALU-wind are consistent. For example,

The time-integrator section looks like this:

Time_Integrators:
  - StandardTimeIntegrator:
      name: ti_1
      start_time: 0
      time_step: 0.25
      termination_time: 4000.0
      time_stepping_type: variable
      time_step_count: 0
      second_order_accuracy: yes

The actuator initialization section from openFAST C++ API looks like this:

actuator:
      type: ActDiskFASTNGP
      search_method: stk_kdtree
      search_target_part: [fluid_part,fluid_part.pyramid_5._urpconv,fluid_part.tetrahedron_4._urpconv]

      n_turbines_glob: 1
      dry_run:  False
      debug:    False
      simStart: init
      t_start: 0.0
      t_max:    5000.0
      dt_fast: 0.00625
      n_every_checkpoint: 100

And, in the .fst.in file, I have the following inputs

------- FAST v8.16.* INPUT FILE ------------------------------------------------
FAST Certification Test #01: NREL 5.0 MW Baseline Wind Turbine (Onshore)
---------------------- SIMULATION CONTROL --------------------------------------
false         Echo            - Echo input data to <RootName>.ech (flag)
"FATAL"       AbortLevel      - Error level when simulation should abort (string) {"WARNING", "SEVERE", "FATAL"}
    5000.00   TMax            - Total run time (s)
    0.00625   DT              - Recommended module time step (s)
          2   InterpOrder     - Interpolation order for input/output time history (-) {1=linear, 2=quadratic}
          0   NumCrctn        - Number of correction iterations (-) {0=explicit calculation, i.e., no corrections}
      99999   DT_UJac         - Time between calls to get Jacobians (s)
      1E+06   UJacSclFact     - Scaling factor used in Jacobians (-)
---------------------- FEATURE SWITCHES AND FLAGS ------------------------------

With this setting, when I look at the integrated qty like power, Cp, Ct from the openFAST output channels, I get really inconsistent/unphysical results.

Any feedback/help on why this is going wrong?

image

image

image

For the first few steps, atleast the values were not defying sanity, C_p pretty much stayed ~ 0.65, C_T ~ 0.75 and Rotor Aero Power ~ 3.5 MW, and then after a few hundred timesteps the integrated quantities are completely on haywire.

lawrenceccheung commented 4 years ago

Hi @tchatte3,

Yes, that's very strange behavior. What is the inflow condition for the turbine? Is it a uniform inflow or a precursor/turbulent inflow?

Here's what I would do to narrow down the problem.

  1. First, try replacing the OpenFAST turbine model with another known model (like the NREL 5MW). If it behaves the same way, then there might be something wrong with the inflow or the actuator disk implementation. If it works fine, then there's a problem with the OpenFAST model.

  2. If there's a problem with the turbine model, then try running it in standalone OpenFAST, using a simple constant uniform wind. Ideally the turbine should converge to a nice, steady-state behavior after a minute or two.

  3. Once you have a known, good turbine model, you can test it in a uniform flow Nalu-Wind case. Ideally it should recover the same behavior in step 2. I'd track the rotor speed, hub-height wind speed, rotor torques, and other things to see when exactly the problem develops.

Hopefully this helps. If you need any more ideas, let me know.

Lawrence

tchatte3 commented 4 years ago

@lawrenceccheung , thanks for the feedback. Yes, you are right, there is a problem with the openFAST model. I am actually trying to simplify the openFAST model as much as possible. I am reducing the degrees of freedom with Elastodyn and checking if that works. Thanks for the detailed feedback.

psakievich commented 4 years ago

@sayerhs @gantech are there any reasons why we don't automatically just set t_max to termination_time internally and remove the t_max flag all together? As @tchatte3 mentioned it does seem rather cumbersome.

sayerhs commented 4 years ago

The FAST tMax should be as long as the total solve time including all the restarts you ever plan to use. I suspect that's why @gantech set it the way it is. Using termination time means it won't work with restarts correctly. Another issue is with precursor restarts. Then you want fast times to start at zero when nalu-wind is at 20000. If you do 300000 timesteps, you'll likely blow up memory in FAST

gantech commented 4 years ago

t_max for OpenFAST is set at the beginning of the simulation and cannot be changed. OpenFAST allocates all output arrays based on t_max and stores them throughout the simulation. https://github.com/OpenFAST/openfast/blob/master/modules/openfast-library/src/FAST_Subs.f90#L2113 As @sayerhs said, this is a potential memory issue. Also, you cannot set t_max to say 300s and then decide to run a little bit further if you decide the simulation has not run enough yet. Also, I believe this changed recently, previously OpenFAST would choose the maximum of the t_max specified between Nalu-wind and OpenFAST, now it seems to set t_max based on external input alone if available https://github.com/OpenFAST/openfast/blob/master/modules/openfast-library/src/FAST_Subs.f90#L1580.

So bottom line t_max has been a weird issue with Nalu-wind + OpenFAST for a while now. Sorry about that. We can remove t_max from the user inputs in Nalu-wind and get it from OpenFAST initialization, but that will require a bit of rewrite. It's been on my case to fix this for a while. I'll see if I can fix this issue within the next 2-3 weeks, but I'm afraid I don't have bandwidth to do this any sooner.

tchatte3 commented 4 years ago

@lawrenceccheung Getting more realistic results of C_p, C_t rotor power now. Elastodyn inputs had some blade DOF's turned on, which perhaps were making it unrealistic. Making the blades/tower rigid solves this problem. I am closing this issue. Thanks for the valued feedback.

psakievich commented 4 years ago

@sayerhs @gantech thanks for the clarification. that makes sense.