adding tests for kernel of NIST web reflectivity calculator

reflectivity / analysis

Data analysis for Neutron and X-ray Reflectometry

https://www.reflectometry.org/

Creative Commons Zero v1.0 Universal

0 stars 4 forks source link

adding tests for kernel of NIST web reflectivity calculator #68

Closed bmaranville closed 2 years ago

bmaranville commented 2 years ago

test the WebAssembly kernel used in the NIST reflectometry calculator at https://pages.nist.gov/reflectometry-calculators/reflectivity-calculator.html

No resolution correction is available in the web calculator, and those tests are skipped.

When the polarised-beam tests in #66 are merged, this can also be updated to check the kernel for the polarised calculator at https://pages.nist.gov/reflectometry-calculators/magnetic-reflectivity-calculator.html

andyfaff commented 2 years ago

Do you know why it's necessary to set Pythonpath?

bmaranville commented 2 years ago

I should have made a separate PR for the work on the BornAgain test... But the reason I put in the PYTHONPATH is because the package gets installed in a non-standard place when you run make install - it seems to go in <prefix>/python/<python version>/dist-packages and python doesn't look there by default for modules. The BornAgain test passes now.

andyfaff commented 2 years ago

Have you thought about using pyodide so that you could use refl1d directly?

bmaranville commented 2 years ago

Yes, I like pyodide and I've tried to think about running refl1d that way for a while, but there are a few hurdles:

we are using numba to make the calculation kernel fast (recently rewritten into python from C) and there is no numba support in pyodide right now (see https://github.com/pyodide/pyodide/issues/621)
the original C kernel from refl1d, compiled to WebAssembly, is what we're already using in the web calculator, so it's hard to imagine much performance benefit from changing to pyodide for that part
startup times for pyodide are still a bit long, though they're getting better (python core + load numpy + load scipy can take many seconds to start)

It would be cool to use bumps to drive the web fitting though, so this is still something to think about!

bmaranville commented 2 years ago

What is the plan for the "lint" test - as far as I can tell almost everything fails that test right now. Can we turn it off for now, then turn it on after someone has reformatted the test modules so that they pass?

Right now it is kind of a useless check for new code (provides no information) since it fails for the existing codebase. It seems like it would be most useful for checking new code, after the existing code is made "conformant".

andyfaff commented 2 years ago

The lint action is passing on master (I just checked on my local copy which is up to date with HEAD), but failing on the 2 PRs. The test is black --check -l 80 validation/scripts/*.py

To autoformat your code I think you'd just do black -l 80 validation/scripts/*.py (or the relevant path). To see what changes the autoformat would do: black -l80 validation/scripts/*.py --diff

bmaranville commented 2 years ago

I had some extra .py files floating around in my scripts folder that aren't checked in, and they were all getting flagged - which was my source of confusion. The formatting is fixed.

bmaranville commented 2 years ago

It is very odd that the BornAgain tests succeeded 13 days ago, but then the same test (unchanged) failed 6 days ago (not able to find bornagain package in python). Something significant must have changed on their end.

Ah - not so mysterious after all - previous to https://jugit.fz-juelich.de/mlz/bornagain/-/commit/7a4518c196010072e4e1975fb8aafbb41f07cbd1 (1 week ago) the package was installed in a standard python path.

andyfaff commented 2 years ago

The BA install in this repo works on the tip of the git repo. I originally did that in part to keep me skilled on how to build BA, rather than being surprised at each release. However, that also means that when the project changes the Python API we get surprised. I don't mind it as it is, it just needs tweaking every few weeks. One of the advantages of writing the validation code is that you get familiarised with different packages for calculating e.g. unpolarised/polarised NR.

andyfaff commented 2 years ago

Do the new cmake compilation options make the compile process go faster?

bmaranville commented 2 years ago

The compile is faster without building the GUI, though it seems to trigger a bug in one of the tests if you build with BA_GUI=OFF (which is why I disabled the tests too)... But I can't create an issue on their git server because I'm not an authorized user.

EDIT: Joachim showed me the way to log into the Juelich git server with gitub credentials. So I will file a bug report.