Pony ORM lambda or generator expression function is marked as unexecuted

[jgirardet]

Hi, Here a small matter I have with ponyorm. In ponyorm you can make query to database with lambda or generator expressions. those 2 thing are equivalent :

return Praticien.select(lambda p: p.nom_d_exercice.lower().startswith(chaine.lower()))
return select(p for p in Praticien if p.nom_d_exercice.lower().startswith(chaine.lower()))

html report says :port says :

The line 98 didn't finish lmabda on line 98 or lne 98 didn't run generator expression in line 98

seems that smaller expression are ok :

Praticien.select()

I really would help to try fix it but I do not know where to start

Thank you Jimmy

[nedbat]

@jgirardet Thanks. One way to help is to provide me with a reproducible test case. Exactly what code should I run, with exactly what commands, to see the results you saw?

[jgirardet]

Ok I run into this. this is the response of one pony's maintainer :

Alexander Kozlovsky admin In some sense it is correct that lambda or generator passed to select is not executed. Pony just analyzes its AST to build an equivalent SQL query. Probably there needs some cooperation between Pony and coverage.py to not report this code as not executed

But after some tries, it appears coverage do not always fail. It fails with the branch option.

the code :

from pony.orm import Database, select, Optional

db = Database()

class A(db.Entity):
    aa = Optional(str)

def gen_exp():
    return select(p for p in A)

def lambdas():
    return A.select(lambda p: p)

db.bind(**{"provider": "sqlite", "filename": ":memory:"})  # pragma: no cover
db.generate_mapping(create_tables=True)  # pragma: no cover

if __name__ == "__main__":
    gen_exp()
    lambdas()

run with :

coverage run --branch __init__.py  
 coverage xml __init__

outputs:

<?xml version="1.0" ?>
<coverage branch-rate="0.5" branches-covered="3" branches-valid="6" complexity="0" line-rate="1" lines-covered="12" lines-valid="12" timestamp="1532369023831" version="4.5.1">
    <!-- Generated by coverage.py: https://coverage.readthedocs.io -->
    <!-- Based on https://raw.githubusercontent.com/cobertura/web/master/htdocs/xml/coverage-04.dtd -->
    <sources>
        <source>/home/jimmy/dev/sandbox/cov/cov</source>
    </sources>
    <packages>
        <package branch-rate="0.5" complexity="0" line-rate="1" name=".">
            <classes>
                <class branch-rate="0.5" complexity="0" filename="__init__.py" line-rate="1" name="__init__.py">
                    <methods/>
                    <lines>
                        <line hits="1" number="1"/>
                        <line hits="1" number="4"/>
                        <line hits="1" number="7"/>
                        <line hits="1" number="8"/>
                        <line hits="1" number="11"/>
                        <line branch="true" condition-coverage="50% (1/2)" hits="1" missing-branches="exit" number="12"/>
                        <line hits="1" number="15"/>
                        <line branch="true" condition-coverage="50% (1/2)" hits="1" missing-branches="exit" number="16"/>
                        <line branch="true" condition-coverage="50% (1/2)" hits="1" missing-branches="27" number="23"/>
                        <line hits="1" number="24"/>
                        <line hits="1" number="25"/>
                        <line hits="1" number="27"/>
                    </lines>
                </class>
            </classes>
        </package>
    </packages>
</coverage>

but :

coverage run __init__.py  
coverage xml __init__

returns

<?xml version="1.0" ?>
<coverage branch-rate="0" branches-covered="0" branches-valid="0" complexity="0" line-rate="1" lines-covered="12" lines-valid="12" timestamp="1532369222219" version="4.5.1">
    <!-- Generated by coverage.py: https://coverage.readthedocs.io -->
    <!-- Based on https://raw.githubusercontent.com/cobertura/web/master/htdocs/xml/coverage-04.dtd -->
    <sources>
        <source>/home/jimmy/dev/sandbox/cov/cov</source>
    </sources>
    <packages>
        <package branch-rate="0" complexity="0" line-rate="1" name=".">
            <classes>
                <class branch-rate="0" complexity="0" filename="__init__.py" line-rate="1" name="__init__.py">
                    <methods/>
                    <lines>
                        <line hits="1" number="1"/>
                        <line hits="1" number="4"/>
                        <line hits="1" number="7"/>
                        <line hits="1" number="8"/>
                        <line hits="1" number="11"/>
                        <line hits="1" number="12"/>
                        <line hits="1" number="15"/>
                        <line hits="1" number="16"/>
                        <line hits="1" number="23"/>
                        <line hits="1" number="24"/>
                        <line hits="1" number="25"/>
                        <line hits="1" number="27"/>
                    </lines>
                </class>
            </classes>
        </package>
    </packages>
</coverage>

so it's not exactly a coverage bug. do you see eventually something to do for it ?

[nedbat]

Hmm, fascinating, I had no idea Pony worked like this. I'll have to think about whether there's anything coverage.py can do. It would probably have to be something Pony invokes to tell coverage what is happening.

One thing you can do is to mark the lines with a pragma:

return Praticien.select(lambda p: p.nom_d_exercice.lower().startswith(chaine.lower()))    # pragma: no branch

[jgirardet]

ok I'll go with it for now thanks for help

[JaDogg]

Hello,

See this: https://stackoverflow.com/questions/53280985/how-to-ignore-ponyorm-generator-expressions-in-nose-tests-coverage/53281148#53281148

I've tried to solve is with #pragma: no branch, it works some time. but some time you need to also use #pragma: no coverage

[nedbat]

Some other possibilities of how to handle this more gracefully:

1. You can set the coverage.py pragma regex so that some lines are automatically pragma'd:

   [report]
   partial_branches = 
       pragma: no branch
       \.select\(lambda

   Now any line that matches either of the two regexes will be considered partial-branch, so your line is recognized even without a comment.

2. You can separate the definition of the lambda or generator expression from the line that uses them:

   to_select = lambda p: p.nom_d_exercice.lower().startswith(chaine.lower())    # pragma: no branch
   return Praticien.select(to_select)

   or:

   to_delete = (j for j in MyEntityClass if j.deleted == True)  # pragma: no branch
   delete(to_delete)

   This isolates the non-run code to its own line, so you don't run the risk of turning off coverage measurement on a line that truly needs it.