Tests on Debian hang

[merkys]

I am building FinalCif v113 and running its tests on Debian unstable. Some of the tests appear to hang indefinitely. It is difficult to pinpoint which of them exactly: at first it appears to be the tests in tests/test_workfolder.py, but if I disable it, hang happens earlier (!), most likely at tests/test_options.py.

Debian test system disallows network access (and sets NO_NETWORK=1 in environment, as discussed earlier), and it also forbids reading/writing in user's home directory. If hung tests need read/write to user's home directory, I can enable that, but I need to know how these directories are accessed. I tried using XDG or HOME environment variables, but they do not seem to work.

[dkratzert]

Hi, I thought the tests are failing because of gemmi changed the return type of find_pair() -> tuple instead of int.

I am referring to the error message

self = <tests.test_text.TestText testMethod=test_set_pair_delimited_empty>

    def test_set_pair_delimited_empty(self):
        self.block.set_pair('_foobar', delimit_string(''))
>       self.assertEqual(['_foobar', ''], self.block.find_pair('_foobar'))
E       AssertionError: ['_foobar', ''] != ('_foobar', '')

etc.

I did a fix in 34006bc2d8b0480208d2f9a5ee82d22ce9bbcb18, the commit after the version-113 tag. Does it work with that commit?

[merkys]

Thanks for the fix for gemmi incompatibility, however, this does not seem to change anything. Test runner just hangs at some point and has to be terminated manually. I recall there was similar problem before, maybe having something to do with GUI (which is not visible during Debian tests) or user interaction (which is also not possible during Debian tests).

[dkratzert]

I can only guess. There is one file that is downloaded to the home directory during a html checkcif. But that is never done inside tests. I made a commit b51d44e1598d357957827147fe7c19a4c03d67e5 to even remove the pathlib.Path instance to that file.

[dkratzert]

Uups, accidentally closed the issue. It was meant to stay open.

[merkys]

I made a commit b51d44e to even remove the pathlib.Path instance to that file.

Alas, this did not work. Maybe this has something to do with the fact that a display is not connected during tests. Interestingly, no such problem existed in v108. I can check whether it was v109 where the problem started appearing.

[merkys]

Indeed the problem started appearing in v109. I ran the tests twice killing them when they hang. At both times call stack shows that the interrupt happened while in setUp() of tests/test_workfolder.py, somewhere in setText() of finalcif/gui/custom_classes.py. Is it possible that infinite loop/recursion happens there under certain conditions?

[dkratzert]

Hm, strange. I can't see a possibility for an infinite loop. Does it work if you disable test_edit_values_and_save() of tests/test_workfolder.py?

[merkys]

Turns out tests do not hang, they merely require more RAM. I was working on a VM with 2 GB of RAM which sufficed before, but now tests/test_workfolder.py seems to require 2.8 GB RAM to run. Increasing RAM size to 4 GB did the trick. Sorry for the noise.

The last commit to "pass" the tests was 2bd28ffd844ed9f3fd8385113867b456d68150f0, thus the RAM-intensive code/test seems to have happened in ab118a2100c77938d5efb1aa00fd5ed7ff01f4b5.

[dkratzert]

No worries, thanks a lot for finding it out. I suspect the line numbered code editor for displaying the saved CIF as guilty. I should investigate if there is a less memory-wasting solution.