Certain unit cells, especially hexagonal cells, along some zone axis projection directions would attempt to tile an infinite number of times. Computers aren't good with infinity.
My fix was to instead of taking the 2 largest in-plane rotated unit cell lengths, take the 2 smallest out-of-plane lengths. This prevents the linear algebra degeneracy which led to the errors.
Certain unit cells, especially hexagonal cells, along some zone axis projection directions would attempt to tile an infinite number of times. Computers aren't good with infinity.
My fix was to instead of taking the 2 largest in-plane rotated unit cell lengths, take the 2 smallest out-of-plane lengths. This prevents the linear algebra degeneracy which led to the errors.