mkoeppe
commented
2 months ago Do the errors show up if you do make ptestlong-nodoc TEST_FILES=src/sage/doctest/control.py?
Open jhpalmieri opened 2 months ago
mkoeppe
commented
2 months ago Do the errors show up if you do make ptestlong-nodoc TEST_FILES=src/sage/doctest/control.py?
jhpalmieri
commented
2 months ago Do the errors show up if you do
make ptestlong-nodoc TEST_FILES=src/sage/doctest/control.py?
No errors with this; I tried it two or three times and it worked every time. With make ptestlong-nodoc TEST_FILES=src/sage/doctest/, I see an error in control.py but not in forker.py.
jhpalmieri
commented
2 months ago Do the errors show up if you do
make ptestlong-nodoc TEST_FILES=src/sage/doctest/control.py?No errors with this; I tried it two or three times and it worked every time. With
make ptestlong-nodoc TEST_FILES=src/sage/doctest/, I see an error incontrol.pybut not inforker.py.
Same thing happens on another machine.
mkoeppe
commented
2 months ago I can't seem to repro this on my machine. Could you post the config.log from the affected machines?
jhpalmieri
commented
2 months ago Let me revise that: when I run make ptestlong, I see
sage -t --long --warn-long 22.6 --random-seed=103198275017503752303450993870311094857 src/sage/doctest/forker.py # 5 doctests failed
sage -t --long --warn-long 22.6 --random-seed=103198275017503752303450993870311094857 src/sage/doctest/control.py # 8 doctests failedWhen I run make ptestlong-nodoc TEST_FILES=src/sage/doctest/, I see
sage -t --long --warn-long 23.4 --random-seed=279304829544013546166396286006292086053 src/sage/doctest/control.py # 3 doctests failedSo I see failures, but fewer failures, in one file.
jhpalmieri
commented
2 months ago Here is one config.log file: config.log
mkoeppe
commented
2 months ago Could you try with this:
diff --git a/src/sage/doctest/control.py b/src/sage/doctest/control.py
index ba78f9bc2a6..fa5b9a062c7 100644
--- a/src/sage/doctest/control.py
+++ b/src/sage/doctest/control.py
@@ -1030,8 +1030,8 @@ class DocTestController(SageObject):
sage: DC = DocTestController(DD, [dirname])
sage: DC.expand_files_into_sources()
sage: for i, source in enumerate(DC.sources):
- ....: DC.stats[source.basename] = {'walltime': 0.1*(i+1)}
- sage: DC.stats['sage.doctest.control'] = {'failed':True,'walltime':1.0}
+ ....: DC.stats[source.basename] = {'walltime': 0.1r*(i+1)}
+ sage: DC.stats['sage.doctest.control'] = {'failed':True,'walltime':1.0r}
sage: DC.filter_sources()
Only doctesting files that failed last test.
sage: len(DC.sources)
@@ -1061,7 +1061,7 @@ class DocTestController(SageObject):
sage: DC.expand_files_into_sources()
sage: DC.sources.sort(key=lambda s:s.basename)
sage: for i, source in enumerate(DC.sources):
- ....: DC.stats[source.basename] = {'walltime': 0.1*(i+1)}
+ ....: DC.stats[source.basename] = {'walltime': 0.1r*(i+1)}
sage: DC.sort_sources()
Sorting sources by runtime so that slower doctests are run first....
sage: print("\n".join(source.basename for source in DC.sources))
@@ -1202,7 +1202,7 @@ class DocTestController(SageObject):
sage: DC.sources.sort(key=lambda s:s.basename)
sage: for i, source in enumerate(DC.sources):
- ....: DC.stats[source.basename] = {'walltime': 0.1*(i+1)}
+ ....: DC.stats[source.basename] = {'walltime': 0.1r*(i+1)}
....:
sage: DC.run()I get
sage -t --long --warn-long 23.4 --random-seed=338769229448687006311559652536132196272 src/sage/doctest/control.py # 2 doctests failedso a slight improvement.
jhpalmieri
commented
2 months ago By the way, I think I have been seeing these failures for a long time, both on an Intel machine (which I no longer own) and Apple Silicon.
mkoeppe
commented
2 months ago I've found a few places more to change, please try #38122
jhpalmieri
commented
2 months ago For what it's worth, I did brew install make (which installs a newer version of make as gmake) and then export MAKE="gmake -j8", and I still get the same doctest failures with make ptestlong-nodoc TEST_FILES=src/sage/doctest/.
mkoeppe
commented
2 months ago Could you check if ./sage -pip uninstall gnumake_tokenpool makes a difference?
jhpalmieri
commented
1 month ago Could you check if
./sage -pip uninstall gnumake_tokenpoolmakes a difference?
Yes, that fixes it: make ptestlong passes those tests.
mkoeppe
commented
1 month ago Could you try with ./sage -pip install 'git+https://github.com/milahu/gnumake-tokenpool@py-subprocess-read-with-timeout' please?
jhpalmieri
commented
1 month ago I didn't undo the uninstallation of gnumake_tokenpool, just did ./sage -pip install 'git+https://github.com/milahu/gnumake-tokenpool@py-subprocess-read-with-timeout'. Testing with make ptestlong-nodoc TEST_FILES=src/sage/doctest/ failed to run:
Doctesting 12 files using 8 threads.
Exception ignored in: <function JobClient.__del__ at 0x12ca9bec0>
Traceback (most recent call last):
File "/Users/palmieri/Sage/TESTING/sage-10.4.beta7/local/var/lib/sage/venv-python3.12/lib/python3.12/site-packages/gnumake_tokenpool/tokenpool.py", line 205, in __del__
if self._fdFifo:
^^^^^^^^^^^^
AttributeError: 'JobClient' object has no attribute '_fdFifo'
----------------------------------------------------------------------
Doctests interrupted: 0/12 files tested
----------------------------------------------------------------------
Total time for all tests: 0.0 seconds
cpu time: 0.0 seconds
cumulative wall time: 0.0 seconds
Traceback (most recent call last):
File "/Users/palmieri/Sage/TESTING/sage-10.4.beta7/src/bin/sage-runtests", line 166, in <module>
err = DC.run()
^^^^^^^^
File "/Users/palmieri/Sage/TESTING/sage-10.4.beta7/src/sage/doctest/control.py", line 1575, in run
self.run_doctests()
File "/Users/palmieri/Sage/TESTING/sage-10.4.beta7/src/sage/doctest/control.py", line 1171, in run_doctests
self.dispatcher.dispatch()
File "/Users/palmieri/Sage/TESTING/sage-10.4.beta7/src/sage/doctest/forker.py", line 2146, in dispatch
self.parallel_dispatch()
File "/Users/palmieri/Sage/TESTING/sage-10.4.beta7/src/sage/doctest/forker.py", line 1855, in parallel_dispatch
job_client = JobClient(use_cysignals=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: JobClient.__init__() got an unexpected keyword argument 'use_cysignals'
make[2]: *** [test-nodoc] Error 1
make[1]: *** [ptest-nodoc] Error 2
make: *** [ptestlong-nodoc] Error 2Right, I need to be more careful with the closing references...
Steps To Reproduce
With a number of
homebrewpackages in place, and withexport MAKE="make -j8", run./configureandmake ptestlong.Expected Behavior
All tests should pass.
Actual Behavior
I get doctest failures:
and
Additional Information
Note that
./sage -tp --long src/sage/doctest/succeeds: I only see these tests when usingmake ptestlong, and I see them every time.In addition, if
MAKEandMAKEFLAGSare unset, then I do not see these failures withmake ptestlong.Environment
Checklist