Packaging - Githubissues

This makes the project a pip-installable package:

$ python3 -m venv ~/.venv/vmec2gk # Create new virtual python environment
$ source ~/.venv/vmec2gk/bin/activate # Enter new environment
$ cd path/to/VMEC2GK
$ pip install -e . # -e makes it an 'editable install'

(If you use Anaconda, you might need to repeatedly call deactivate/conda deactivate first, as Anaconda likes to take over the system Python)

Later, we'll be able to publish it to PyPI so that pip install VMEC2GK will work anywhere. You can also install developer/testing tools using:

$ pip install -e .[dev,tests]

Each of the scripts in the project have been converted to modules containing a single function (or a few of them in the case of bishoper_ball.py). Instead of running eikcoefs_final.py, you should run __main__.py. Instead of coordinating data between files using pickles, it now passes bishop_dict directly to the relevant functions, and pickling is now optional. After installing, you can run it from anywhere on the system using:

$ python3 -m VMEC2GK

I also added a command line interface, so you can supply a different input file/output directory:

$ python -m VMEC2GK --input_file my_input.txt --output_dir ./my/output/dir

You can also set other options from the command line, which override whatever's in the input file (or default input file if none is provided):

$ python -m VMEC2GK --surf_idx=150 --save_gs2

Issues with the command line interface:

If the input file says to save GS2/save GX/do ball scan, you can't override this on the command line.
The interaction between the default input file, user provided input file, and command line arguments is a bit weird. It'll take some more thought to come up with a better system.
The output dir specified by the default input file is within the project. This will have to change before publishing to PyPI, as this would save files within the user's Python environment by default...

I'm leaving this as a WIP for now, as I'd like to get some tests written and add continuous integration. Let me know what you think in the meantime. Happy to explain any aspect of the update that isn't clear to you.

(Note: I renamed src to VMEC2GK, as this makes it easier to get packaging to work. I've yet to figure out the exact series of steps needed to get setuptools to recognise src as the project location without the package itself being named src)

Sorry for the lack of progress lately, I had to take some time off work for personal matters.

I've added a new dependency for tqdm, which is a library for creating progress bars on the command line. It has a method that allows you to run a bunch of parallel jobs similarly to multiprocessing, but you also get a nice progress bar and an estimated time to completion.

More importantly, I've added a bunch of unit tests for utils.py, and an integrated test that ensures that VMEC -> GX is working as expected. It uses reference input/output files that were in use before I started work, so we'll be able to see if any code changes are affecting the results (so far, it looks like we're good!). I'd like to add an equivalent test for GS2, and it would also be good to write an integrated test for bishoper_ball.py. I haven't yet written unit tests for some of the more complex functions in utils.py, but at some point it'd be a good idea to return to them. Tests can be run as follows:

$ python3 -m pip install -e .[tests]    # from the root directory
$ pytest

After I've finished writing tests, I think it would be a good idea to get continuous integration set up with GitHub workflows (should be a fairly easy job), and that'll be it for this PR. Let me know if you have any comments/criticism for the work so far.

I've added some tests to ensure the GS2/ball scan parts of the code are working as expected, and I set up automated testing with GitHub workflows. I'm not sure how to get them to run on this PR, so I set up a test PR on my own account which shows what it'll look like after merging. It's currently failing due to an xarray error with Python 3.7, but works fine with Python 3.8 and 3.9. We could fix the xarray version at a point where it works for all Python versions, but that could lead to dependency conflicts with other projects. It may be better to just set the minimum Python version to 3.8.

I think I'm happy with this PR for now, but there are plenty of areas that could be improved further. The command line interface in particular could use some work. Going forwards, I think it would be a good idea to reorganise some of the code into smaller functions, as that would make it easier to merge in some features from Miller cookbook (it would also make testing a bit easier!).

As always, let me know if you disagree with anything I've done here, or if there's anything else you'd like me to do. It might be a good idea for us to have a video chat at some point in the near future to discuss the work I've done so far, and to talk about the future goals of the project, but there's no hurry.

rahulgaur104 / VMEC2GK

Packaging #8