search

mabarnes / moment_kinetics

Other
2 stars 4 forks source link

Make some dependencies optional, reorganise package structure, improve setup scripts #174

Closed johnomotani closed 5 months ago

johnomotani commented 6 months ago

Some dependencies are heavy, and take a long time to precompile, in particular Symbolics, Makie and Plots. Symbolics is only needed to run MMS tests, and only one of Makie or Plots is needed. This PR allows all these dependencies to be optional.

To allow making dependencies optional, the structure of the package (now packages) is reorganised. moment_kinetics, makie_post_processing, and plots_post_processing are now separate packages, contained in subdirectories. To use moment_kinetics a project is created in the top-level directory of the repo into which some or all of the packages and optional dependencies can be installed.

To simplify the setup of moment_kinetics (especially since all the optional things now make it more complicated), the setup script in machines/machine_setup.sh, etc., is updated and upgraded. It now supports setup on a laptop/desktop (the 'generic-pc' target), has a template setup for a 'generic-batch' machine that can be used to support HPC machines other than the explicitly included ones, and has some extra features too.

For more info, see the updated documentation: https://mabarnes.github.io/moment_kinetics/previews/PR174/getting_started/#Getting-started https://mabarnes.github.io/moment_kinetics/previews/PR174/developing/#Package-structure https://mabarnes.github.io/moment_kinetics/previews/PR174/manual_setup/#Manual-setup https://mabarnes.github.io/moment_kinetics/previews/PR174/machine_setup_notes/

mrhardman commented 5 months ago

The current package structure seems to require that python packages are installed even if one uses the "Plots" based post processing module.

pkg> develop ./plots_post_processing/plots_post_processing/
    ... lots of output
    Building Conda ─→ `~/excalibur/moment_kinetics_geometry/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/51cab8e982c5b598eea9c8ceaced4b58d9dd37c9/build.log`
    Building PyCall → `~/excalibur/moment_kinetics_geometry/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/9816a3826b0ebf49ab4926e2b18842ad8b5c8f04/build.log`
johnomotani commented 5 months ago

The current package structure seems to require that python packages are installed even if one uses the "Plots" based post processing module.

Some of the plots use PyPlot (specifically the plots of the moment-kinetic distribution function using the unnormalised velocity coordinates as axes, as Plots.jl does not support 2d x- and y-positions). It would be technically possible to make that an optional dependency if someone wants to... but if it's causing you a problem I'd suggest hacking - just comment out the PyPlot/PyCall bits in plots_post_processing/plots_post_processing/src/plots_post_processing.jl and plots_post_processing/plots_post_processing/Project.toml.

johnomotani commented 5 months ago

@mrhardman @mabarnes I think this is ready to merge now. Do you think there's anything left to fix or document better?