Empty output file from quip command

[MES-physics]

Hello again, I had a fresh MPI version of QUIP/GAP installed on our cluster, then obtained a potential. (I asked about the following problem in an issue that is now closed. So I will describe again.) Since @albapa advised to compile a non-MPI version to do evaluation, I started from scratch and git cloned and installed a python environment, and into it installed the QUIP/GAP and quippy packages, and all went well with no errors. (QUIP_ARCH=linux_x86_64_gfortran_openmp)

Then trying to use the command : quip E=T F=T atoms_filename=Atoms.xyz param_filename=Cpotential.xml | grep AT | sed 's/AT//' > quip_train.xyz , it produces an empty file in a couple seconds. No errors to look at. Can you help again? I appreciate it so much.

[jameskermode]

What do you get as output without piping to grep and sed?

[MES-physics]

I get the empty .xyz file again. With no errors.

[albapa]

The quip program will send output to the standard output. When you say

I get the empty .xyz file again. With no errors.

Do you mean you get no output whatsoever?

[MES-physics]

Right. I am trying to use this version (QUIP_ARCH=linux_x86_64_gfortran_openmp) on my laptop to evaluate only (Ubuntu 22.04). It produces a file name with no contents. On my laptop there are no error output files or any immediate errors. It just runs for 2 seconds and makes the file name. The potential was made on our cluster with the MPI version. Both compilations are new from this week. Could there be something missing in the way I made the potential? Thanks very much.

[albapa]

If you run just this command

quip E=T F=T atoms_filename=Atoms.xyz param_filename=Cpotential.xml

(and nothing else) it should not produce any files, just some output on the standard output. Also, the way you write it suggests that quip is in your PATH variable. You may want to do

which quip

to establish which executable you are actually running.

[MES-physics]

$which quip
/home/myname/quippy/bin/quip

Is this right?

[albapa]

I don't know. Did you copy the quip binary there after you complied QUIP with QUIP_ARCH=linux_x86_64_gfortran_openmp?

[MES-physics]

Yes the binary is there. Even when I redirect the quip command to the build directory linux_x86_64_gfortran_openmp where the original binary is, it still doesn't work, same issue , produces nothing. I have a question that @gabor1 is in the middle of helping about doing the same quip command in the MPI compilation on our cluster and that is having segfault problems.

[MES-physics]

Additional info: I used to use this command all the time when I was doing the 2-step (to MPI) method for making the potential, with older versions before single step MPI was implemented. Is it possible the one step program affects this? Maybe I should go back and test 2-step to make the potential again? But I would be doing it with the new MPI compilation.

[MES-physics]

Dear GAP people, This is still an issue . I cannot get the evaluation file for GAP potentials with the quip command. It produces an empty file when using the ```QUIP_ARCH=linux_x86_64_gfortran_openmp'''' architecture, and segfaults if I use the MPI architecture on our cluster. I have been asking our admin, and they get the same result when they do the installation. @gabor1 was looking into this? Thanks for help, I'm stuck.

[albapa]

It would help if you shared your potential XML and a small XYZ. You can send it to me if you don't want to upload it here.

[MES-physics]

I sent it all to @gabor1 ‘s email with google drive link. The potential is too large to upload here, and I wouldn’t want to show publicly at the moment. thanks!

[MES-physics]

Thanks @albapa . Thanks for looking into this. Sorry it seems I copied the input StartStructures.xyz file for you the wrong way. When I try that one, it does give me the error you showed. But when I correct it, it still gives me an empty output file.
Originally, it did not give me that malformed frame error with my real file.

New.xyz.txt

Here is one I corrected. Can you tell me which version/architecture you ran to make the evaluation file? Does that matter? My potential was made with a new MPI one step version. I used to run this quip script last year with no problems when I was using the 2-step potential build. Now when I try to run this quip command with the openmp version, it gives an empty file, no errors. When I try it with the MPI build, it gives me segfaults with missing libraries . Maybe this helps:

forrtl: severe (174): SIGSEGV, segmentation fault occurred Image PC Routine Line Source
quip 0000000000CCD71A Unknown Unknown Unknown libpthread-2.28.s 000014E021CA4C20 Unknown Unknown Unknown libiomp5.so 000014E021FFB5EC Unknown Unknown Unknown libiomp5.so 000014E021FFB5BA Unknown Unknown Unknown libiomp5.so 000014E021FFD6AF Unknown Unknown Unknown quip 0000000000CFF8BC Unknown Unknown Unknown quip 00000000005F8411 ipmodel_gap_modul 503 IPModel_GAP.f95 libiomp5.so 000014E021FF1BB3 kmp_invoke_micr Unknown Unknown libiomp5.so 000014E021F6DFAC kmp_fork_call Unknown Unknown libiomp5.so 000014E021F2FCB5 kmpc_fork_call Unknown Unknown quip 00000000005F5949 ipmodel_gap_modul 431 IPModel_GAP.f95 quip 0000000000566944 ip_module_mp_ip_c 872 IP.f95 quip 0000000000512E75 potentialsimple 936 Potential_simple.f95 quip 000000000045FFFC potentialmodule 937 Potential.f95 quip 0000000000472E7C potentialmodule 129 Potential_Sum_routines.f95 quip 000000000045FE75 potentialmodule 940 Potential.f95 quip 000000000040A68E MAIN 709 quip.f95 quip 0000000000D44726 Unknown Unknown Unknown libc-2.28.so 000014E01F579493 __libc_start_main Unknown Unknown quip 0000000000406BAE Unknown Unknown Unknown

[albapa]

I simply used export QUIP_ARCH=linux_x86_64_gfortran for the make. Try to see if that works.

Did you try pip install quippy-ase? It may be the easiest solution for you.

We recommend the MPI version for training only.

export QUIP_ARCH=linux_x86_64_gfortran_openmp should also work for evaluation.

[albapa]

My worry is that if I simply run quip with no arguments whatsoever, I still get the following output:

libAtoms::Hello World: 2024-06-06 22:49:16
libAtoms::Hello World: git version  https://github.com/libAtoms/QUIP.git,v0.9.14-10-g694dc9c0d-dirty
libAtoms::Hello World: QUIP_ARCH    linux_x86_64_gfortran
libAtoms::Hello World: compiled on  Jun  6 2024 at 14:25:51
libAtoms::Hello World: Random Seed = 82156498
libAtoms::Hello World: global verbosity = 0

Calls to system_timer will do nothing by default

SYSTEM ABORT: quip_params.xml does not exist
STOP 1

It appears that your executable never gets even here. I don't know why this is the case.

[MES-physics]

I have QUIP_ARCH=linux_x86_64_gfortran installed on my laptop. And when I retried pip install quippy-ase to make sure I had it, it gives a bunch of messages saying this: Requirement already satisfied: python-dateutil>=2.7 in /usr/lib/python3/dist-packages (from matplotlib>=3.1.0->ase>=3.17.0->quippy-ase) (2.8.1) And when I give the command quip alone it gives what you said above, SYSTEM ABORT: quip_params.xml does not exist I reinstalled this several times already and the same thing happens. Even when using the QUIP_ARCH=linux_x86_64_gfortran build on our cluster, the same problem happens. Is there anything else I can try? Thanks very much.

[bernstei]

pip installs pre-built binaries, so should have nothing to do with QUIP_ARCH. That's for compiling manually.

[jameskermode]

It is important to check where your quip executable is coming from, however, with which quip. The one installed by pip install will likely be in ~/.local/bin or similar.

[bernstei]

That's probably just the python wrapper. I get the actual binary executable in ~/.local/lib/pythonN.M/site-packages/quippy/quip.

Running file path_to_quip_executable and ldd path_to_quip_executable (assuming this is on Linux) might provide useful info.

[MES-physics]

I can see the quip executable in that directory (~/.local/lib/pythonN.M/site-packages/quippy/quip), and it is labelled as executable in the properties window. I ran these:

file /home/name/.local/lib/python3.10/site-packages/quippy/quip
/home/name/.local/lib/python3.10/site-packages/quippy/quip: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=6172d77e064dea0751af03065578ecff380c657f, for GNU/Linux 3.2.0, not stripped

ldd /home/name/.local/lib/python3.10/site-packages/quippy/quip
    linux-vdso.so.1 (0x00007ffc8adfd000)
    liblapack.so.3 => /lib/x86_64-linux-gnu/liblapack.so.3 (0x0000747497c00000)
    libblas.so.3 => /lib/x86_64-linux-gnu/libblas.so.3 (0x000074749835a000)
    libgfortran.so.5 => /lib/x86_64-linux-gnu/libgfortran.so.5 (0x0000747497800000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x0000747497b19000)
    libmvec.so.1 => /lib/x86_64-linux-gnu/libmvec.so.1 (0x0000747497703000)
    libgomp.so.1 => /lib/x86_64-linux-gnu/libgomp.so.1 (0x00007474976b9000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x0000747499342000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x0000747497400000)
    /lib64/ld-linux-x86-64.so.2 (0x000074749937d000)
    libquadmath.so.0 => /lib/x86_64-linux-gnu/libquadmath.so.0 (0x0000747497671000)

When I try to run the command going to that very path, on the small xyz file (and on my big real file)

 /home/name/.local/lib/python3.10/site-packages/quippy/quip E=T F=T atoms_filename=New.xyz param_filename=Cours348.xml | grep AT | sed 's/AT//' > quip_trainNew.xyz

It still makes an empty file with the requested name.

[bernstei]

Stop filtering the output. What does the command itself output, before you remove all useful information with sed and grep?

[MES-physics]

Sorry I didn't know all the details of doing this.

/home/name/.local/lib/python3.10/site-packages/quippy/quip
libAtoms::Hello World: 2024-06-07 12:19:22
libAtoms::Hello World: git version  https://github.com/libAtoms/QUIP.git,v0.9.14-10-g694dc9c0d-dirty
libAtoms::Hello World: QUIP_ARCH    linux_x86_64_gfortran_openmp
libAtoms::Hello World: compiled on  May  2 2024 at 12:03:28
libAtoms::Hello World: OpenMP parallelisation with 12 threads
WARNING: libAtoms::Hello World: environment variable OMP_STACKSIZE not set explicitly. The default value - system and compiler dependent - may be too small for some applications.
libAtoms::Hello World: Random Seed = 44362155
libAtoms::Hello World: global verbosity = 0

Calls to system_timer will do nothing by default

SYSTEM ABORT: quip_params.xml does not exist   
STOP 1

[bernstei]

Yes, that makes sense, because you did not pass it a param_filename=... argument. What happens if you run the full command, with all of its arguments, but not piping the output to grep or sed?

[bernstei]

Also, please ask someone in your group for information, tutorials, etc, whatever local resources, about how the Linux command line works, what are program arguments, how pipes (|) work, etc.

[MES-physics]

/home/name/.local/lib/python3.10/site-packages/quippy/quip E=T F=T atoms_filename=New.xyz param_filename=Cours348.xml > quip_trainNew.xyz
Segmentation fault (core dumped)

[bernstei]

Please, stop redirecting the output. We need to know how far it gets before it crashes.

[MES-physics]

Sorry.

/home/name/.local/lib/python3.10/site-packages/quippy/quip E=T F=T atoms_filename=New.xyz param_filename=Cours348.xml
libAtoms::Hello World: 2024-06-07 12:27:35
libAtoms::Hello World: git version  https://github.com/libAtoms/QUIP.git,v0.9.14-10-g694dc9c0d-dirty
libAtoms::Hello World: QUIP_ARCH    linux_x86_64_gfortran_openmp
libAtoms::Hello World: compiled on  May  2 2024 at 12:03:28
libAtoms::Hello World: OpenMP parallelisation with 12 threads
WARNING: libAtoms::Hello World: environment variable OMP_STACKSIZE not set explicitly. The default value - system and compiler dependent - may be too small for some applications.
libAtoms::Hello World: Random Seed = 44855365
libAtoms::Hello World: global verbosity = 0

Calls to system_timer will do nothing by default

Using calc args:
Using pre-relax calc args:
Using param_filename: Cours348.xml
Using init args:
Segmentation fault (core dumped)

[bernstei]

One possibility is OpenMP. What does

env OMP_NUM_THREADS=1 /home/name/.local/lib/python3.10/site-packages/quippy/quip E=T F=T atoms_filename=New.xyz param_filename=Cours348.xml

output?

[bernstei]

Could also be a stacksize type limit. What does limit | grep stack or ulimit -s return?

[MES-physics]

env OMP_NUM_THREADS=1 /home/name/.local/lib/python3.10/site-packages/quippy/quip E=T F=T atoms_filename=New.xyz param_filename=Cours348.xml
libAtoms::Hello World: 2024-06-07 12:39:39
libAtoms::Hello World: git version  https://github.com/libAtoms/QUIP.git,v0.9.14-10-g694dc9c0d-dirty
libAtoms::Hello World: QUIP_ARCH    linux_x86_64_gfortran_openmp
libAtoms::Hello World: compiled on  May  2 2024 at 12:03:28
libAtoms::Hello World: OpenMP parallelisation with 1 threads
WARNING: libAtoms::Hello World: environment variable OMP_STACKSIZE not set explicitly. The default value - system and compiler dependent - may be too small for some applications.
libAtoms::Hello World: Random Seed = 45579151
libAtoms::Hello World: global verbosity = 0

Calls to system_timer will do nothing by default

Using calc args:
Using pre-relax calc args:
Using param_filename: Cours348.xml
Using init args:
Segmentation fault (core dumped)

[bernstei]

It's probably also worth testing env OMP_NUM_THREADS=1 OMP_STACKSIZE=512M /home/name/.local/lib....., although given that your input atoms file is small, that's very unlikely to be the issue.

[MES-physics]

ulimit -s
8192

[MES-physics]

env OMP_NUM_THREADS=1 OMP_STACKSIZE=512M /home/name/.local/lib/python3.10/site-packages/quippy/quip E=T F=T atoms_filename=New.xyz param_filename=Cours348.xml
libAtoms::Hello World: 2024-06-07 12:52:08
libAtoms::Hello World: git version  https://github.com/libAtoms/QUIP.git,v0.9.14-10-g694dc9c0d-dirty
libAtoms::Hello World: QUIP_ARCH    linux_x86_64_gfortran_openmp
libAtoms::Hello World: compiled on  May  2 2024 at 12:03:28
libAtoms::Hello World: OpenMP parallelisation with 1 threads
libAtoms::Hello World: OMP_STACKSIZE=512M
libAtoms::Hello World: Random Seed = 46328860
libAtoms::Hello World: global verbosity = 0

Calls to system_timer will do nothing by default

Using calc args:
Using pre-relax calc args:
Using param_filename: Cours348.xml
Using init args:
Segmentation fault (core dumped)

[bernstei]

Try running ulimit -s unlimited before running the quip executable. If that gives an error, maybe something like ulimit -s 128000, or whatever the largest value it will allow. Running ulimit -s should then show the new value.

[MES-physics]

Umlimited didn't show an error so I tried the undirected command and it started printing the data! So I went ahead and directed to the file output, and here it is! Does the file look ok? In the previous version I never got the heading info in this data file. quip_trainNew.xyz.txt

[bernstei]

Looks generally reasonable. I think your stacksize was just set too small for some internal Fortran array. You'll need to run that command every time you start a new shell (or put it in one of your startup files, ~/.bashrc or something like that)

[MES-physics]

Oh thank you VERY much! This gets me unstuck in my project!