Using the minimum required variables generates atm_ and grid_ files for only the parcel origin file and not for any of the following timesteps

hs2112 commented 1 year ago

Using ERA5 analysis, model level data
minimum requirement as given here - https://mptrac.readthedocs.io/en/latest/meteorological-input-data/ - is lnsp,z,t,u,v,w,q
after reading file for a certain timestep, it generates the following warmings and moves on to the next step without writing

lars2015 commented 1 year ago

Hello hs2112,

I fixed some issues in MPTRAC with reading meteo netCDF files as described in issue #13. If you still find no output is being written with the revised code, please let me know.

For debugging, it would be helpful if you could please share the run script and the full logs so that we can check the control parameter settings of the run.

Best regards Lars

hs2112 commented 1 year ago

Hi @lars2015

For my workflow involving the minimum required variables, the code seems to spend time at the following lines but then moves on without writing the files -

Warning (libtrac.c, read_met_surface, l4462): Cannot read surface meridional wind!
  Apply periodic boundary conditions...
  Calculate geopotential heights...
  Calculate potential vorticity...
  Calculate planetary boundary layer...
  Calculate tropopause...
  Calculate cloud data...
  Calculate CAPE...
Read meteo data: data_in/ea_2005_06_20_21.nc

run_min.txt

logfile_min.txt

Sharing three files; one for the origin time and two of the following timesteps - https://drive.google.com/drive/folders/1Wd9hSEnbrxljNOtqnV6Kz_6qSjmOJv_u?usp=sharing

Hope you can take a look and let me know what steps I need to change. Thanks!

lars2015 commented 1 year ago

Hi @hs2112,

from the log file, the only issue I spotted is that you need to change the time interval of the output. Currently, the default time interval of 86400 s is being used for output. This will only create the files for the initial time step, but no further output will be written during the 21 h of the simulation time.

For example, for hourly output, you would have to add these control parameters:

    ATM_DT_OUT = 3600
    GRID_DT_OUT = 3600

You should also adapt the time interval for meteo data sampling to match the output frequency:

    MET_DT_OUT = 3600

Everything else looks fine.

Best regards Lars

hs2112 commented 1 year ago

Oh. Thanks or sharing! Added those params and it does write the files now. I still can not find these anywhere in the run.sh file in your example which contains only MET_DT_OUT and DT_MET.

Will try out the plots next and shall keep you updated about issues if any!

Best, Hardik

hs2112 commented 1 year ago

Able to generate plots.

For backward trajectories, any changes required other than reversing t0 and t1?

lars2015 commented 1 year ago

Hello Hardik,

happy to hear you got it running!

The example provided with MPTRAC is rather simple, many control parameters are assigned default values when not provided in the control parameter file or at the command line. The default values actually being used can be found in the log output.

The doxygen manual provides brief descriptions of all the control parameters: https://slcs-jsc.github.io/mptrac/doxygen/structctl__t.html

For backward trajectories, you need to set DIRECTION = -1 and reverse t0 and t1. Trajectories will be calculated from the initial time of the air parcels (or the time T_START, if provided as a control parameter) back to T_STOP.

Best regards Lars

hs2112 commented 1 year ago

Looks like https://slcs-jsc.github.io/mptrac/doxygen/structctl__t.html has a lot of parameters 😂 Could you help me out with the role of CAPE/ CIN in the model, and how to use diffusivity at various scales of motion? The context of the question is that we want to calculate trajectories for equally spaced parcel origin heights from near-surface up to the tropopause.

hs2112 commented 1 year ago

Rather, the link initially worked but points to '404-not found' now. I went through your paper here - https://gmd.copernicus.org/articles/15/2731/2022/ - and understand the following -

Boundary layer -

Boundary layer depth depends on extent of convective mixing, which is estimated using CAPE
CAPE must be calculated outside of MPTRAC and provided in the input data (no role of CIN as of now)
- If that is correct, please clarify relevant details - can every timestep have CAPE values, what should be name of the variable, etc. Or is it provided by one static CONV_CAPE parameter?
CAPE0 threshold is to be manually provided in run.sh
- What is the parameter in the code and where does it need to be declared?

Perturbation of positions and wind speeds -

Any given position is stochastically perturbed at every timestep as per diffusivities Dx, Dz
Wind speeds at a given gridpoint are perturbed by sub-grid scale fluctuations controlled by the scaling factor f

Overall,

Could you please help out with relevant parameters for all quantities mentioned here and help out with details?
Please let me know if I need to consider something else here
Please send an active link to the parameter page

lars2015 commented 1 year ago

Hello Hardik,

Rather, the link initially worked but points to '404-not found' now. I went through your paper here - https://gmd.copernicus.org/articles/15/2731/2022/ - and understand the following -

The autobuild of the doxygen manual was broken, I am afraid. I fixed this.

Boundary layer -

* Boundary layer depth depends on extent of convective mixing, which is estimated using CAPE

MPTRAC does not feature any advanced parametrization schemes for the boundary layer. It will apply the same diffusion and subgrid-scale wind fluctuation schemes as used for the free troposphere. The extreme convection parametrization will impose vertical mixing from the surface to the equilibrium level.

* CAPE must be calculated outside of MPTRAC and provided in the input data (no role of CIN as of now)

MPTRAC provides an internal code to calculate CAPE and CIN from the meteo input data. It requires the temperature and water vapor to be provided in the input data.

The meteorological data pre-processing and calculation of CAPE is described in more detail in the supplement of Hoffmann et al. (2022): https://gmd.copernicus.org/articles/15/2731/2022/gmd-15-2731-2022-supplement.pdf

There is also a new paper draft online that describes more details of extreme convection parametrization: Hoffmann, L., Konopka, P., Clemens, J., and Vogel, B.: Lagrangian transport simulations using the extreme convection parametrization: an assessment for the ECMWF reanalyses, EGUsphere [preprint], https://doi.org/10.5194/egusphere-2023-72, 2023.

  * If that is correct, please clarify relevant details - can every timestep have CAPE values, what should be name of the variable, etc. Or is it provided by one static CONV_CAPE parameter?

* CAPE0 threshold is to be manually provided in run.sh

The parameter CONV_CAPE will activate the convection parametrization. CONV_CAPE = 0 is the extreme case, applying parametrized convection wherever CAPE exists. For CONV_CAPE > 0, it is considered as a threshold level, above which parametrized convection is applied. For CONV_CAPE < 0 (the default), parametrized convection is switched off and only explicitly resolved convection in the meteo input data is considered.

  * What is the parameter in the code and where does it need to be declared?

There are additional parameters for the convection module, please see the doxygen manual. But for a first test to see whether parametrized convection would be relevant, you could try CONV_CAPE = 0.

Perturbation of positions and wind speeds -
* Any given position is stochastically perturbed at every timestep as per diffusivities Dx, Dz

You can use the control parameters TURB_DX_TROP, TURB_DZ_TROP, TURB_DX_STRAT, and TURB_DZ_STRAT to specify constant horizontal and vertical diffusivities [m^2/s] separately for the troposphere and stratosphere.

MPTRAC uses a monthly mean zonal mean tropopause climatology from the NCEP/NCAR Reanalysis-1 to decide whether a particle is located in the troposphere or stratosphere. There is +/- 1 km transition layer in between.

The default parameter settings for the diffusivities are the same as for the FLEXPART model. You would need to set all the parameters to zero to switch off diffusion.

* Wind speeds at a given gridpoint are perturbed by sub-grid scale fluctuations controlled by the scaling factor f

This is the second module to add stochastic perturbations to the trajectories. In contrast to the diffusivities described earlier, the subgrid-scale perturbations are correlated over time. This approach also follows the FLEXPART model.

The parameters TURB_MESOX and TURB_MESOZ provide scaling factors for the grid-scale variances to the subgrid scales in the horizontal and vertical direction, resp. The default value is 0.4 x 0.4 = 0.16, i.e., 40% of grid-scale variance is transferred to the subgrid scales.

Overall,

* Could you please help out with relevant parameters for all quantities mentioned here and help out with details?

* Please let me know if I need to consider something else here

* Please send an active link to the parameter page

I hope the brief descriptions above are helpful. If you have any further questions, please let me know.

Best regards Lars

hs2112 commented 1 year ago

Thanks. Looks like CONV_CAPE, TURB_DX_TROP, TURB_DZ_TROP, TURB_DX_STRAT, and TURB_DZ_STRAT, TURB_MESOX and TURB_MESOZ are the parameters I'm looking for.

So, providing CONV_CAPE = 0 or CONVCAPE > 1500 (for e.g.) will initiate implicit calculation of CAPE values by the internal code and based on that, impose a vertically mixed convective boundary layer (provided T and q are present in the input data)? Is there a way to print these implicit calculations in a atm.tab or grid_.tab file?

lars2015 commented 1 year ago

Hello Hardik,

you can apply the met_map tool to extract maps of CAPE, CIN, the lifted condensation level, the level of free convection, and the equilibrium level as calculated by the meteo pre-processing code of MPTRAC for given meteorological input data.

It is also possible to get the data interpolated along the trajectories by specifying them as quantities for output:

    NQ = 5
    QNT_NAME[0] = cape
    QNT_NAME[1] = cin
    QNT_NAME[2] = pel
    QNT_NAME[3] = plfc
    QNT_NAME[4] = plcl

Best regards Lars

hs2112 commented 1 year ago

Still don't completely get it. Trying to understand how the BL is treated differently than the free troposphere; please help me understand better? "The parameter CONV_CAPE will activate the convection parametrization. CONV_CAPE = 0 is the extreme case, applying parametrized convection wherever CAPE exists. For CONV_CAPE > 0, it is considered as a threshold level, above which parametrized convection is applied. For CONV_CAPE < 0 (the default), parametrized convection is switched off and only explicitly resolved convection in the meteo input data is considered." "MPTRAC does not feature any advanced parametrization schemes for the boundary layer. It will apply the same diffusion and subgrid-scale wind fluctuation schemes as used for the free troposphere. The extreme convection parametrization will impose vertical mixing from the surface to the equilibrium level."

So, in general (CONV_CAPE < 0 or when CAPE < positive CONV_CAPE threshold), the BL is treated same as the free troposphere with its diffusivities, but when CAPE > positive CONV_CAPE threshold, you are allowing for an additional 'convective mixing' which does not involve Dx and Dz. Yes?
- And then do you recompute 'mixed' parcel properties in the 'convective boundary layer' to reflect vertical mixing?
The supplement also mentions that BL height is calculated using the Bulk Richardson number. Does it mean that the BL height is estimated using Ri for all cases? Asking because you mentioned that MPTRAC does not feature any advanced parametrization schemes for the BL.

Thanks, Hardik

hs2112 commented 1 year ago

doxygen manual not working again. Would you mind sending me an offline copy?

lars2015 commented 1 year ago

Dear Hardik,

it seems the web site with the doxygen manual breaks when we push to the GitHub repository too frequently. It works fine now. In case it breaks again, you can find a pdf copy of the doxygen manual at this link: https://github.com/slcs-jsc/mptrac/blob/master/docs/refman.pdf

Your description of the convection and diffusion modules in the previous post is correct. Convection and diffusion/subgrid-scale mixing are treated by separate modules, but they can both be applied at the same time.

Convective mixing (= random vertical redistribution between the surface and the equilibrium level) is activated with the control parameter CONV_CAPE, where CONV_CAPE < 0 means parametrized convection is switched off, CONV_CAPE = 0 implies extreme convective (whenever CAPE is present in the atmosphere), and CONV_CAPE > 0 limits the parametrized convection to cases where CAPE >= CAPE_CONV.

This new paper draft provides more information on the extreme convection parametrization as implemented in MPTRAC: Hoffmann, L., Konopka, P., Clemens, J., and Vogel, B.: Lagrangian transport simulations using the extreme convection parametrization: an assessment for the ECMWF reanalyses, EGUsphere [preprint], https://doi.org/10.5194/egusphere-2023-72, 2023.

In its present form, MPTRAC will apply the diffusion/subgrid-scale mixing scheme with the same parameter settings, diffusivities, etc. as used for the free troposphere also for the boundary layer. This means MPTRAC does not have any advanced parameterizations of mixing/turbulence for the boundary layer and that there is actually no distinction between the PBL and the free troposphere in terms of the mixing schemes, as in other Lagrangian particle dispersion models.

Best regards Lars

hs2112 commented 1 year ago

Aa great! Thanks for the quick reply.

do you update the parcel properties in the 'convectively mixed layer' to reflect the mixing?
if convective mixing is controlled by CAPE alone, then what is the role of the Richardson number based BL height?
Strangely, no matter what INIT_Z0 and INIT_Z1 values I provide, I get very similar plots for altitudes of past parcels. Will look more closely at it tomorrow and let you know

lars2015 commented 1 year ago

Hello Hardik,

do you update the parcel properties in the 'convectively mixed layer' to reflect the mixing?

The particle properties like mass, vmr, etc are not being changed in the convection module. Only the vertical positions of the air parcel are modified.

if convective mixing is controlled by CAPE alone, then what is the role of the Richardson number based BL height?

The PBL pressure is only calculated as diagnostic information and not used in the convection module. You can extract PBL pressure for the location of each air parcel to check whether it is within or above the PBL, for example.

Strangely, no matter what INIT_Z0 and INIT_Z1 values I provide, I get very similar plots for altitudes of past parcels. Will look more closely at it tomorrow and let you know

This should work fine. If you need help, please let me know.

Best regards Lars

hs2112 commented 1 year ago

Thank you. So the BL is merely a diagnostic and implies no change in workflow. Only the convective layer causes redistribution of parcels, which seems fair given that the stable BL is pretty small anyway. Couple more questions -

You have mentioned that the minimum requirements for trajectory calculations are u, v and w. When T and q are provided in addition, do you incorporate in the trajectory equation?
I can't find the descriptions for SPLIT_DX, INIT_Z1, SPLIT_M, etc. in the doxygen manual. Why do you need to initialize both _Z0 and _Z1, _LON0 and LON1, etc. instead of just once, and what do the various SPLIT params do?

Could you please share descriptions for parameters like the ones you have mentioned here - https://slcs-jsc.github.io/mptrac/applications/?

Best Hardik

slcs-jsc / mptrac

Using the minimum required variables generates atm_ and grid_ files for only the parcel origin file and not for any of the following timesteps #12