coverage + nose + mako resulting in "No source for code:" on compiled module for template

[nedbat]

Originally reported by Michael Merickel (Bitbucket: mmerickel, GitHub: mmerickel)

---

I'm having trouble tracking this down but something is going on between nose and coverage.

The broken commit is here, where I tried to add coverage reports for our tests, as well as the source package.

https://github.com/Pylons/pyramid_debugtoolbar/compare/2a5a30a8d33cf0e51bfb50d8b3487f1e9c6d6b36...add-coverage-on-tests

Executing the following commands:

#!text

$ virtualenv env
$ env/bin/python setup.py dev
$ env/bin/coverage erase
$ env/bin/coverage run --source=pyramid_debugtoolbar,tests env/bin/nosetests
$ env/bin/coverage xml
No source for code: '/Users/michael/work/oss/pyramid_debugtoolbar/tests/pyramid_debugtoolbar_templates_exception_dbtmako': [Errno 2] No such file or directory: '/Users/michael/work/oss/pyramid_debugtoolbar/tests/pyramid_debugtoolbar_templates_exception_dbtmako'

The path munging code inside mako is generating pyramid_debugtoolbar_templates_exception_dbtmako and someone is turning that into tests/pyramid_debugtoolbar_templates_exception_dbtmako.

https://github.com/zzzeek/mako/blob/22fb044c3cb177aa197af1877c1c67e3f70066b7/mako/template.py#L247-254

Note that the original template is at the path pyramid_debugtoolbar/templates/exception.dbtmako.

The weirder parts are that nose's where=tests/ option in setup.cfg breaks this. The following patch will fix everything by specifying the tests folder on the CLI. However, if I use -w tests (effectively the same as the where=tests/) then things break.

#!diff

diff --git a/setup.cfg b/setup.cfg
index [58e2803 (bb)](https://bitbucket.org/ned/coveragepy/commits/58e2803)..7cbc718 [100644 (bb)](https://bitbucket.org/ned/coveragepy/commits/100644)
--- a/setup.cfg
+++ b/setup.cfg
@@ -6,7 +6,7 @@ zip_ok = false

 [nosetests]
 match=^test
-where=tests/
+#where=tests/
 nocapture=1

 [aliases]
diff --git a/tox.ini b/tox.ini
index [55cac15 (bb)](https://bitbucket.org/ned/coveragepy/commits/55cac15)..7666421 [100644 (bb)](https://bitbucket.org/ned/coveragepy/commits/100644)
--- a/tox.ini
+++ b/tox.ini
@@ -25,7 +25,7 @@ commands =
 [testenv:py2-cover]
 commands =
     pip install pyramid_debugtoolbar[testing]
-    coverage run --source=pyramid_debugtoolbar,tests {envbindir}/nosetests
+    coverage run --source=pyramid_debugtoolbar,tests {envbindir}/nosetests tests
     coverage xml -o coverage-py2.xml
 setenv =
     COVERAGE_FILE=.coverage.py2
@@ -33,7 +33,7 @@ setenv =
 [testenv:py3-cover]
 commands =
     pip install pyramid_debugtoolbar[testing]
-    coverage run --source=pyramid_debugtoolbar,tests {envbindir}/nosetests
+    coverage run --source=pyramid_debugtoolbar,tests {envbindir}/nosetests tests
     coverage xml -o coverage-py3.xml
 setenv =
     COVERAGE_FILE=.coverage.py3

In any scenario I can come up with so far running env/bin/nosetests --with-coverage --cover-package=pyramid_debugtoolbar,tests --cover-tests --cover-xml tests dumps out a valid xml file. However, I would like to use the coverage command directly and not through nose.

Please any insight would be helpful as the PylonsProject is investigating using this pattern in many projects.

I tested this against 3.7 and 4.0a5 with the same results.

Looking at env/bin/coverage debug data in both scenarios (coverage run and nosetests --with-coverage) shows the following:

#!text

<snip>
/Users/michael/work/oss/pyramid_debugtoolbar/tests/pyramid_debugtoolbar_templates_exception_dbtmako: 81 lines
/Users/michael/work/oss/pyramid_debugtoolbar/tests/pyramid_debugtoolbar_templates_exception_summary_dbtmako: 29 lines
/Users/michael/work/oss/pyramid_debugtoolbar/tests/pyramid_debugtoolbar_templates_redirect_dbtmako: 29 lines
<snip>

However if I remove the -w (the only working scenario described earlier using coverage run), those files are not listed in debug data.

---

• Bitbucket: https://bitbucket.org/ned/coveragepy/issue/365

[nedbat]

Original comment by Loic Dachary (Bitbucket: dachary, GitHub: dachary)

---

It looks like this is an issue best fixed outside of coverage.py. Is there a reason to keep this issue open ?

[nedbat]

@xistence if you could make the filenames be less misleading, that would be great :) Or give me some other idea about how to handle this. The ideal case is that someone writes a plugin for these template languages. Then you might also get coverage measurement of the templates!

[nedbat]

Original comment by Bert JW Regeer (Bitbucket: xistence, GitHub: xistence)

---

@nedbat I actually pushed that change to pyramid_zcml, and just posted regarding deform to https://bitbucket.org/ned/coveragepy/issues/98/no-source-for-code

It's all part of the same issue, whereby Mako/Chameleon generate bytecode, which is not persisted to disk.

[nedbat]

Original comment by Michael Merickel (Bitbucket: mmerickel, GitHub: mmerickel)

---

@nedbat Please also see https://github.com/Pylons/pyramid_zcml/pull/17 which was opened yesterday as another example.

[nedbat]

Original comment by Michael Merickel (Bitbucket: mmerickel, GitHub: mmerickel)

---

@nedbat Obviously it's been a while. I can tell you the behavior in the notes was observed by several other developers on their own machines. We spent several hours going through permutations that would dump out a proper report but in the end I think dropping the -w caused both systems to just ignore the missing coverage on the templates.

[nedbat]

@mmerickel Sorry I have not gotten to this. Is there any new information?

[sydoluciani]

@nedbat Is coveragepy multi threaded ?

collecting python file names and running test against them at the same time can be problematic if tests are creating temp files, which will be found by file collector and eventually below error: No source for code: deform/checkbox_fdc46098de12c7d295ff61cb419e5a84.py'. Aborting report output, consider using -i.

[nedbat]

@sydoluciani Can you provide me with a reproducible scenario?

[sydoluciani]

This is the branch I have requested a pull, https://github.com/sydoluciani/deform/tree/deform_bootstrap4

And here another pull request: https://github.com/Pylons/deform/tree/drop-py27-py34

It is failing on coverage with "No source for code" on many files, which their name after underscore is random hash, and we had to use regex to omit any files that has digit in their names to pass the missing files problem, while there is no file with such name exists in deform module, either with hash or without the hash part.

deform/deform/dateinput_83525f355147fecaef2c41f09e88f5e5.py deform/deform/checkbox_32bb822ccaed554768253405d5135c00.py deform/deform/checkbox_74b3186f303388f28170fc8d93e3e007.py

Thanks

[sydoluciani]

It is arguable that the template generates temp files shall create them in a /tmp directory instead of the module that is being tested, however for the sake of test, is it possible to change the behavior of coveragepy to single threaded or single process ?