Closed qzhu2017 closed 6 years ago
In crystal.py, the variable of timespent is not defined.
#Print additional information about the structure
if verbosity > 0:
print("Time required for generation: " + str(timespent) + "s")
print(rand_crystal.struct)
#If generation fails
else:
print('something is wrong')
print('Time spent during generation attempt: ' + str(timespent) + "s")
@qzhu2017 I've fixed the cif file output, so that it should succeed. It checks if the file already exists. If it does, it increases a variable called filecount by 1, which is appended to the end of the filename.
I will work on a master test script to comprehensively test all functionalities.
@qzhu2017 I have uploaded a file under test_cases called test_all.py As a script, it will check that all modules load and work properly. Then, it tests crystal generation for all space groups. Currently, it uses volume factor 1, a thickness of 3 Angstroms, Hydrogen for atomic crystals, and water for molecular crystals. These parameters are a bit slow for some groups, but for me most generate in less than a minute. Any groups slower than this will be noted in the command line. There is a separate function for each crystal type (3d/2d, atomic/molecular), as well as for testing the modules. I will work on adding some command-line options later.
Currently, when running the script, I get a segmentation fault error. I'm not sure if there's a memory leak somewhere, or if it's a different issue. Also, for molecular crystals, lattice generation fails most of the time (see the issue for molecular crystal generation).
@qzhu2017 I apologise for how long it is taking to fix the segmentation fault issue. Because it only occurs in the middle of the script, it takes a long time to reproduce. Also, while running the script with a debugger (dbg), I cannot reproduce the issue yet.
Each time the issue appears, it occurs for the second space group after generating all 230 space groups for a different crystal type. For example, test_atomic will run for all space groups, but then an error occurs for space group 2 of test_molecular. However, if I only run one test at a time (test_atomic, test_molecular, test_atomic_2D, or test_molecular_2D), they will run for all space groups without issue.
This is the first time I have encountered a segmentation fault issue, so I am having trouble diagnosing it. It seems to be a memory address problem, but I cannot find any variables which are being stored outside of the scope of the loop.
=== Testing generation of atomic 3D crystals. This may take some time. ===
Spacegroup # | Spacegroup Generated | Time Elapsed
1 | 2 | 0.00 s
2 | 2 | 0.02 s
223 | 229 | 5.27 s ~~
224 | 224 | 4.82 s ~~
225 | 225 | 85.08 s ~~~~
226 | 226 | 150.09 s ~~~~
227 | 227 | 67.78 s ~~~~
In order to save the time, we can set a list of unwanted space groups and do not run these groups during the test. Generate Oh group with general positions can be challenging.
This has been fixed
multiplicity = len(get_wyckoffs(sg)[0]) #multiplicity of the general position
I suggest we put some randomness here. Do not ask the program to generate the general positions all the type. We can set a random number, and generate some special positions if the symmetry is allowed.
=== Testing generation of molecular 3D crystals. This may take some time. === Couldn't generate crystal after max attempts.
This is a bit strange. Group 14 should be very easy to generate. I would suggest a profiling on this process.
@scottfredericks 72 | 164 | 164 | 0.61 s 73 | 168 | ??? | 0.16 s 74 | 174 | ??? | 0.38 s 75 | 175 | 175 | 0.67 s 76 | 177 | 177 | 0.83 s 77 | 183 | ??? | 0.73 s 78 | 187 | 187 | 4.85 s ~~ 79 | 189 | 189 | 52.24 s ~~~
Sometimes, I saw the detected space groups are '???', which seems to mean that SPGLIB cannot find the right space group. We need to store this information and study them case by case.
@qzhu2017 I have added a call to pymatgen.symmetry.analyzer.SpacegroupAnalyzer, in order to compare with spglib's output. It seems that in the cases when spglib is unable to detect the space group, pymatgen is able, which makes the "???"s less troubling. But, there are still some issues:
1) For 2D monoclinic groups, the generated space group is often incorrect. I think this is a problem with the 2D lattice generation. I will check the code again and make sure it is correct.
2) For many 3D space groups, the detected sg number is lower than the generated one, but typically only by 1 or 2. For these cases, the order of the symmetry group seems to be the same, so I am not sure if the result is actually incorrect or not. I'll have to look through some numerical data to check.
3) This seems to happen less often for molecular crystals. But when it does occur, the generated space group is sometimes significantly lower (for example, space group 127 generated 83 for me). For molecular crystals, I am not sure if the molecular symmetry always corresponds directly to the lattice symmetry, because we are comparing relative and absolute coordinates. I was under the impression that the point group only contains operations which are "compatible" with the lattice, but I need to check this more rigorously. I will try to look at a few cif files for these cases and see what the issue is.
Once the monoclinic issue is fixed, I will have the program make a note of any time the detected space group is different from the expected one, and give an option to output the cif files.
@qzhu2017 The monoclinic issue seems to be fixed now. I think I misunderstood how the permutation worked, and was changing the assignment of angles based on the PBC direction. If I understand correctly, the only purpose of defining the PBC while generating a 2D lattice, is to change the side lengths based on the thickness value.
I changed test_all, so that it looks for any crystal where the detected space group is lower than the expected space group. For these groups, it outputs a cif file.
Looking at some of the output cif files, it's hard to analyze the symmetry, since the cif file already has the lower-symmetry operations encoded. For some cases, it seems like the representative atoms are all in positions corresponding to the higher-symmetry group, but the cif file lists them as having a lower symmetry. Since only the representative atom is listed, I can't be sure what the other positions were originally. So, I will switch to POSCAR or some other output for testing.
@qzhu2017 I've looked at some particular cases. The easiest case to replicate is space group 95 for 3d molecular crystals, with 8 molecules in the primitive cell. This sometimes generates space group 87 instead.
It looks like the special Wyckoff position coordinates are correct, but the specific atomic coordinates are off. This means there is an issue placing the molecules after a Wyckoff position is chosen. I think it is most likely a misuse of np.dot or np.transpose; I will try to resolve this asap.
@qzhu2017 I have checked the cases for 3D atomic generation. There are many times where the detected space group is lower than the expected one, but this is just an error of spglib: it chooses the lowest symmetry group compatible with the coordinates. I verified that for each POSCAR file, the coordinates are also compatible with the expected space group. So, it seems that the output for 3D atomic is all correct.
Now that I know it is an issue specific to molecular crystals, I will spend the rest of the day trying to fix the symmetry for molecular_crystal, then move onto checking 2D and 1D.
@scottfredericks I also noticed a list of cases where spglib failed to return the correct symmetry. Could you please collect them? I would like to send it to the author of spglib and ask for the possible fixes.
@scottfredericks I take the previous statement back. The way how spglib determine the space group is correct. We don't need to worry about that. It looks like all these structures have simpler unit cell which point to the lowest symmetry. A simple fix could be to compare the point group between the desired space group and determined space group. If they point to the same point group. This should be a successful generation.
@qzhu2017 You are correct. I looked at every mismatch, and either the point group is the same, or the determined space group actually has a higher symmetry. I will create a simple function to check compare the two point groups, and stop marking these cases as incorrect.
Also, I made small change: I remembered that our functions take the stoichiometry for the primitive cell, so I was accidentally setting the stoichiometry higher than it needed to be for non-primitive lattices. I have fixed this by inserting the line "multiplicity = multiplicity / cellsize(sg)"
@qzhu2017 I have implemented a function called compare_wyckoffs within test_all.py. It takes two space group numbers, and returns whether the 2nd has equal or higher symmetry compared to the first. Now, we no longer mark spg as being incorrect for equivalent point groups. With this change, all 3D atomic space groups generate and are detected correctly.
@scottfredericks It looks great now. While testing on the 2D crystal, I constantly get the wrong results on layer group 48: Cmme.
from pyxtal.crystal import random_crystal_2D
from spglib import get_symmetry_dataset
my_crystal = random_crystal_2D(48, ['C'], [4], 2.0, 2.5)
print(get_symmetry_dataset(my_crystal.spg_struct, symprec=1e-1)['international'])
print(my_crystal.struct)
Here is the output:
Pmma
Full Formula (C8)
Reduced Formula: C
abc : 12.366487 9.407733 12.000000
angles: 90.000000 90.000000 90.000000
Sites (8)
# SP a b c
--- ---- -------- ---- --------
0 C 0.900964 0.25 0.536615
1 C 0.099036 0.25 0.536615
2 C 0.099036 0.75 0.463385
3 C 0.900964 0.75 0.463385
4 C 0.139643 0 0.5
5 C 0.860357 0.5 0.5
6 C 0.860357 0 0.5
7 C 0.139643 0.5 0.5
The first four atoms belong to (x,1/4,z) | (-x,1/4,z) | (-x,3/4,-z) | (x,3/4,-z) which is consistent with the first 4 positions in 8n in Cmme. However, it seems to forget to add the rest 4 position by translation (1/2, 1/2, 0).
I am suspecting that the A/B/C-centered lattices are not processed correctly.
@qzhu2017 I apologize - at one point I had a script to automatically add the centering translations to the Wyckoff positions, but I forgot to do this for the layer groups. I've corrected this, and made the corresponding changes to layer.csv, layer_symmetry.csv, and layer_generators.csv. This shouldn't be an issue for Rod groups, since they are all P-centered.
After making these changes and re-building, all atomic 2D groups are correct. Unfortunately, it seems spg and pymatgen are unable to detect a few certain layer groups, since they're in a non-standard setting. However, I have checked these cases by hand, and they match the Wyckoff positions given on Bilbao.
@scottfredericks , excellent! good to know we have resolved this issue.
The run on 1D case seems to go very fast. But there are some issues:
=== Testing generation of atomic 1D crystals. This may take some time. ===
Layergroup | Gen sg. (SPG) | Gen. sg (PMG) |Time Elapsed
1 | 2 | 2 | 0.00 s
2 | 2 | 2 | 0.00 s
3 | 12 | 2 | 0.00 s
4 | 10 | 10 | 0.00 s
5 | 10 | 10 | 0.00 s
6 | 2 | 2 | 0.00 s
7 | 10 | 10 | 0.01 s
8 | 10 | 10 | 0.01 s
9 | 2 | 2 | 0.00 s
/Users/qiangzhu/Desktop/github/PyXtal/pyxtal/crystal.py:772: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
pairs = pairs[sequence]
10 | 12 | 2 | 0.01 s
11 | 10 | 2 | 0.04 s
12 | 2 | 2 | 0.01 s
13 | 16 | 16 | 0.01 s
14 | 17 | 17 | 0.01 s
15 | ??? | 25 | 0.01 s
16 | 67 | 49 | 0.01 s
17 | 51 | 51 | 0.01 s
18 | ??? | 25 | 0.01 s
19 | 63 | 28 | 0.01 s
20 | 47 | 47 | 0.01 s
21 | 49 | 49 | 0.08 s
22 | ??? | 51 | 0.04 s
23 | 123 | 99 | 0.05 s
24 | 76 | 76 | 0.01 s
25 | 84 | 84 | 0.00 s
26 | 78 | 78 | 0.00 s
27 | 81 | 81 | 0.00 s
28 | 83 | 83 | 0.01 s
29 | 84 | 84 | 0.02 s
30 | 123 | 123 | 0.06 s
31 | 91 | 91 | 0.01 s
32 | 93 | 93 | 0.01 s
33 | 95 | 95 | 0.02 s
34 | ??? | 99 | 0.06 s
35 | 132 | 132 | 0.01 s
36 | 123 | 124 | 0.02 s
37 | 123 | 123 | 0.07 s
38 | 111 | 112 | 0.01 s
39 | ??? | 123 | 0.37 s
40 | 123 | 124 | 0.04 s
41 | 131 | 131 | 0.16 s
42 | 174 | 174 | 0.00 s
43 | 144 | 144 | 0.00 s
44 | 145 | 145 | 0.00 s
45 | 191 | 191 | 0.25 s
46 | 149 | 149 | 0.14 s
47 | 151 | 151 | 0.03 s
48 | 153 | 153 | 0.01 s
49 | 1 | 1 | 0.03 s
50 | 191 | 183 | 0.02 s
51 | ??? | ??? | 0.06 s xxxxx
52 | ??? | 40 | 0.05 s
53 | 175 | 175 | 0.01 s
54 | 169 | 169 | 0.01 s
55 | 171 | 171 | 0.01 s
56 | 176 | 176 | 0.18 s
57 | 172 | 172 | 0.01 s
58 | 170 | 170 | 0.01 s
59 | 174 | 174 | 0.01 s
60 | 175 | 175 | 0.06 s
61 | 176 | 176 | 0.06 s
62 | 191 | 191 | 0.07 s
63 | 178 | 178 | 0.02 s
64 | 180 | 180 | 0.08 s
65 | ??? | ??? | 0.06 s xxxxx
66 | 181 | 181 | 0.03 s
67 | 5 | 5 | 0.24 s
68 | 183 | 183 | 0.12 s
69 | 192 | 192 | 0.02 s
70 | 194 | 194 | 0.03 s
71 | 187 | 187 | 0.03 s
72 | 188 | 188 | 0.10 s
73 | 191 | 191 | 0.09 s
74 | ??? | ??? | 0.38 s xxxxx
75 | 194 | 194 | 0.53 s
Please remove this warning. Alternatively, I don't quite understand the followings 49 | 1 | 1 | 0.03 s 65 | ??? | ??? | 0.06 s xxxxx 74 | ??? | ??? | 0.38 s xxxxx
@qzhu2017 In these cases, neither spg nor pymatgen are able to detect the space group, since the Rod groups do not correspond directly to a space group. I am currently working on a simple function to test whether or not the structure matches the space, layer, or rod group expected. This should let us ignore these cases.
I will look into the warning; I have never seen it before.
@qzhu2017
I made the following change:
pairs1 = deepcopy(pairs)
pairs=pairs1[sequence]
I was unable to replicate the issue, so please let me know if it persists.
@scottfredericks I just made a correct fix in a807371
@qzhu2017 I have implemented a function called check_struct_group within test_all.py. In the case that spg and pymatgen cannot determine the correct space group, this function checks if the structure is compatible with the symmetry operations in the group's general Wyckoff position. It is not a very fast algorithm, but it seems fast enough for this purpose, and seems to give the correct results for 3D and 2D cases.
For the 1D cases, we call this function for all Rod groups, since we have no way to compare the spg and pmg output. But, for some Rod groups it doesn't seem to work, even though checking by hand seems to give the correct results. I will look into this tomorrow.
In the mean time, we have eliminated the incorrect groups for the 2D case, which eliminates some clutter from the screen and from the output files.
@qzhu2017 There was an issue in the list processing within check_struct_group, which is now fixed. Based on my tests, all 2D and 1D atomic groups now generate with the correct symmetry.
@scottfredericks There are some strange output for molecular_2D.
=== Testing generation of molecular 2D crystals. This may take some time. ===
Layergroup | sg # Expected |Generated (SPG)|Generated (PMG)|Time Elapsed
1 | 1 | 1 | 1 | 0.05 s
2 | 2 | 2 | 2 | 0.13 s
3 | 3 | 1 | 1 | 0.17 s
4 | 6 | 1 | 1 | 0.07 s
5 | 7 | 1 | 7 | 0.05 s
6 | 10 | 2 | 1 | 0.30 s
7 | 13 | 2 | 2 | 0.15 s xxxxx
8 | 3 | 1 | 1 | 0.11 s
9 | 4 | 1 | 1 | 0.04 s
10 | 5 | 1 | 1 | 0.08 s
11 | 6 | 1 | 1 | 0.10 s
12 | 7 | 1 | 7 | 0.05 s
13 | 8 | 1 | 1 | 0.09 s
14 | 10 | 1 | 1 | 0.28 s
15 | 11 | 2 | 2 | 15.49 s ~~~ xxxxx
16 | 13 | 2 | 2 | 0.12 s xxxxx
17 | 14 | 2 | 2 | 0.11 s xxxxx
18 | 12 | 2 | 1 | 0.25 s
19 | 16 | 16 | 16 | 0.40 s
20 | 17 | 17 | 17 | 0.13 s
21 | 18 | 18 | 18 | 0.11 s
22 | 21 | 21 | 21 | 0.36 s
23 | 25 | 25 | 25 | 0.37 s
24 | 28 | 51 | 28 | 27.27 s ~~~
25 | 32 | 32 | 32 | 0.11 s
26 | 35 | 35 | 35 | 0.28 s
27 | 25 | 25 | 25 | 0.23 s
28 | 26 | 26 | 26 | 0.11 s
29 | 26 | 26 | 26 | 0.09 s
30 | 27 | 27 | 27 | 0.11 s
31 | 28 | 28 | 28 | 14.22 s ~~~
32 | 31 | 31 | 31 | 0.09 s
33 | 29 | 29 | 29 | 0.06 s
34 | 30 | 30 | 30 | 0.09 s
35 | 38 | 38 | 38 | 0.20 s
36 | 39 | 39 | 39 | 0.15 s
37 | 47 | 47 | 47 | 1.01 s ~
38 | 49 | 49 | 49 | 0.44 s
First of all, it seems that the generation of low index layer group structures all failed. Secondly, I don't know why the space group symmetry determination results from spg and omg sometimes are inconsistent. In principle, pmg just called spg to determine the symmetry!
@qzhu2017 I am not sure why pmg and spglib give different answers. spg is the base, but the SpacegroupAnalyzer class may have additional considerations, which allows it to work more consistently.
I have updated the method for placing the molecules within the crystal. I had made an error in the way the transformations were applied, which caused distortion in some cases.
However, this illuminated a different issue with the method; the marked layer groups do indeed have the incorrect symmetry. For monoclinic lattices, at least for the 2D case, the Wyckoff symmetry operations are not always Euclidean. For example, consider the special Wyckoff positions 2e [(0,0,z),(0,0,-z)] and 2i [(x,y,0),(-x,-y,0)] in the Layer group 6 (p 1 1 2/m). These positions have site symmetry ..2 and ..m respectively. Imagine a molecule centered at the origin, with atoms in both the 2e and 2i positions. For example, consider an XeF4 molecule with Xe at the origin, 2 F4 atoms in the x-y plane, and 2 atoms along the z axis: Xe: 0.000, 0.000, 0.000 Fe: 0.707, 0.707, 0.000 Fe: -0.707, -0.707, 0.000 Fe: 0.000, 0.000, 1.000 Fe: 0.000, 0.000, -1.000 (these being absolute coordinates) If the lattice were cubic, there would be no issue. However, for a monoclinic cell the molecule no longer meets the lattice's symmetry requirements: the atoms on the c axis would be skewed towards the a-b plane. However, our current method has no knowledge of the lattice, so it would only look at whether or not the molecule meets the symmetry requirements ..2 and ..m (it does). But, the Fe atoms are not actually in the 2i position.
It seems that this issue only applies to certain molecules and orientations; if we were dealing with an H2 molecule we could simply align the molecule along the c axis. But, this would still require a slightly different method from what we use now. I need to think about the best way to handle this. Since I have not seen this problem for other groups, it may not be as big an issue as I think.
@qzhu2017 I think the monoclinic symmetry issue may be a problem of finding the correct setting, rather than an issue based on our molecular orientation method. Consider the Rod groups Pm11 (4) and P11m (10). A monoclinic cell can only have mirror symmetry about 1 axis. Thus, these groups must have different settings, and our assumption about c always being the unique axis seems incorrect. I will try to figure out the correct settings tomorrow.
@qzhu2017 I have made two changes for 2D and 1D lattice generation. For 2D, I changed the use of the space group number to instead use the layer group number. For 2D and 1D, I added a "unique_axis" variable, which only applies to monoclinic cells. For layer groups 3-7 and Rod groups 8-12, the unique axis is c. For layer groups 8-18 and Rod groups 3-7, the unique axis is a. The unique axis decides whether the unique angle is alpha, beta, or gamma (unique axis a, b, or c respectively). You were correct about the periodicity, though - we can keep c as the periodic axis.
With these changes, I am no longer detecting any incorrect symmetry for molecular 1D and 2D groups. I think it is time to test some other molecular stoichiometries.
@scottfredericks , I just quickly ran the script. It looks great for atomic crystals. However, I observed something strange for atomic_2D.
All modules passed!
=== Testing generation of atomic 2D crystals. This may take some time. ===
Layergroup | sg # Expected |Generated (SPG)|Generated (PMG)|Time Elapsed
1 | 1 | 71 | 12 | 0.02 s
''''
29 | 26 | 51 | 26 | 1.58 s ~
30 | 27 | 225 | 49 | 0.06 s
The detection of space group 225 is kind of strange. Have you added the vacuum on the generated 2D /1D? If you have thick vacuum, I don't think it is possible to get cubic symmetry.
@qzhu2017 That is odd. Even without Add_vacuum, the lattice generated should not be cubic (though it is possible but unlikely for it to be close enough to be within numerical tolerance). I put the call to Add_Vacuum back into random_crystal_2D earlier today, so it's possible you are using a version without it.
I've altered my local version of test_all to only generate layer group 30. After generating several hundred times, I do not get a cubic space group, though I do occasionally get tetragonal (which I'm guessing is due to spg's numerical tolerance).
For 1D, I have not implemented the call to Add_vacuum, since it seems to alter the symmetry detection. If I understand correctly, we should be able to call it twice: once with dim=0, once with dim=1. But, when I make this change, most Rod groups are flagged as having the wrong symmetry.
@qzhu2017 I have also noticed that molecular_2D occasionally has flagged results. I have not found any incorrect atomic structures, but if you find any, please upload them so I can look at them.
@qzhu2017 Also, I think it would be useful to place the text output of test_all.py into a text file called "Summary.txt". Is there an easy way to do this within Python, without rewriting all of the print functions?
@scottfredericks, your request has been done in 8359153
regarding the output errors , please check these structures and collect some of them which you believe it is due to wrong symmetry determination due to spglib. We need to collect these data and send it to the author of spglib.
@scottfredericks When I tried to follow the tutorial to run the script, it returned some annoying messages. It would be good to create a simple test script and make sure you run it every time after you updated the code.