search

nomad-coe / nomad

NOMAD lets you manage and share your materials science data in a way that makes it truly useful to you, your group, and the community.
https://nomad-lab.eu
Apache License 2.0
77 stars 19 forks source link

Installing an oasis, nomad-lab and using parsers #83

Open hertznek opened 1 year ago

hertznek commented 1 year ago

Hello,

I am trying to follow the online documentations available at https://nomad-lab.eu, among other things for getting a NOMAD Oasis running as well as using/developing a parser.

NOMAD I couldn't manage to get a base installation (without Docker); I followed the instructions here, and everything went fine untill the end, in the section Running NOMAD, where nomad admin ops gui-config complains about gui-config:

Usage: nomad admin ops [OPTIONS] COMMAND [ARGS]...
Try 'nomad admin ops --help' for help.

Error: No such command 'gui-config'.

Also, right after that, it mentions To run NOMAD, you must run two services. One is the NOMAD app, it serves the API and GUI:, but the box underneath (where there should be a command to run) is empty.

Parsers Following How to write a parser, installed the example parser withpip install -e ., and then running the example python -m exampleparser tests/data/example.out led to an error:

  File "/[...]/nomad-parser-example/exampleparser/parser.py", line 25, in <module>
    from nomad.datamodel.metainfo.public import section_run as Run
ModuleNotFoundError: No module named 'nomad.datamodel.metainfo.public'

In this environment, there is nomad-lab, nomad-lab[parsing], nomad-lab[dev] and nomad-lab[infrastructure] installed (possibly most are not needed, but they are there).

Environment

conda 23.7.3
nomad-lab 1.2.0
python 3.9.18
Ubuntu 22.04.3 LTS running Linux 5.15

Other parsing questions Slightly offtopic, but I also tried using the parser available in the NOMAD package i.e. nomad parse [...] with a NeXus/HDF5 file (parsing an HDF5 file is the end goal). The example file is available in the NeXus Format website, in particular simple_example_write1.hdf5.

After changing the extension to nxs (otherwise the parser complains about unknown format), it seems to be correctly parsed. The file sizes are:

3.2K  simple_example_write1_h5dump.txt  # hdf5 dump, all fields available
 11K  simple_example_write1.hdf5     # downloaded file
 11K  simple_example_write1.nxs    # same as HDF5, simply changed the extension

Running the parser:

time nomad parse simple_example_write1.nxs 

real    0m17.520s
user    0m17.226s
sys 0m1.367s

Is that timing expected, or is there potentially something wrong in my installation/environment?

Thanks in advance.

PS: In case it may be useful: using a conda environment, pip install nomad-lab only worked under Python 3.9 (tested 3.8, 3.10, 3.11, all of them issued one error at some point, either with nomad-lab or nomad-lab[parsing]). With a python virtualenv, python 3.10 also issued an error installing nomad-lab[parsing] (did not try other python versions). I can provide the error logs if they are relevant.

markus1978 commented 1 year ago

Thanks for your feedback. It is appreciated, even though this reply is super delayed.

We need to update the base installation (without docker). The gui-config step is not necessary anymore and the command was removed.

The issue about the example parser not working is something we need to fix. There have been some breaking changes in nomad-lab not reflected in the example parser. I opened an issue our internal gitlab: https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1723

The parser timing is that bad unfortunately. The import structure needs to be optimised. At the moment we are converting all application definitions (50+) to the NOMAD schema language on the fly. This is only happening once on import. The actual parsing is much quicker, e.g. when many files are processed in a loop or something. You can run the parser programatically: https://nomad-lab.eu/prod/v1/staging/docs/apis/local_parsers.html#from-a-python-program