mypy==0.620 compatibility issue

[pawelswiecki]

Versions

• Python 3.6.5

• mypy==0.620+dev.bc0b551d8df2baadc44f0c3b0b801fcc12119658

• dataclasses==0.6

What's wrong

$ cat f2.py 
from dataclasses import dataclass
print('works!')
$ python f2.py 
works!
$ mypy f2.py 
f2.py:1: error: Cannot find module named 'dataclasses'
f2.py:1: note: (Perhaps setting MYPYPATH or using the "--ignore-missing-imports" flag would help)

So the natural way is to use MYPYPATH to point to the specific package, dataclasses in this case. The problem is dataclasses package is installed directly in site-packages (virtual_env_path/lib/python3.6/site-packages/dataclasses.py in my case), and it's an error (and a bad idea) to pass the whole site-packages to MYPYPATH.

When I manually move the file from /site-packages/dataclasses.py to /site-packages/dataclasses/dataclasses.py and pass the path to MYPYPATH it's working as expected: no mypy error. Unfortunately, this is not the solution I can use.

I reported this issue on mypy page (https://github.com/python/mypy/issues/5342) and was told that:

The backport package can fix this by including stubs (presumably the same ones that are in typeshed) and marking itself as typed by using py.typed.

Can anything be done about it? I cannot migrate to Python 3.7 yet and would benefit much from mypy actually understanding dataclasses.

[ericvsmith]

Have you tried installing the stubs yourself? I'd like to know that it works before adding it to the backport.

[pawelswiecki]

I just tried and failed.

1. Copied https://github.com/python/typeshed/blob/master/stdlib/3.7/dataclasses.pyi to my project.

2. Run mypy with MYPYPATH env var pointing to dir with the stub file.

3. Getting even more errors:

project_path/stubs/dataclasses.pyi:37: error: Overloaded function signatures 1 and 3 overlap with incompatible return types  # new error
project_path/stubs/dataclasses.pyi:42: error: Overloaded function signatures 2 and 3 overlap with incompatible return types  # new error
project_path/module/file.py:33: error: "field" does not return a value  # new error
project_path/module/file.py:127: error: Too many arguments for "MyDataclass"  # old error

Line 33 is:

field1: Optional[str] = field(init=False, default=None)

Line 127 is:

MyDataclass(field1='foo')

I may be doing something wrong.

Also, this solution, even if worked, would be cumbersome, since I'd have to manually update all my repos in which I use dataclasses. It's not that bad, but doing it by bumping lib version would be more convenient.

[pawelswiecki]

By the way, I have some ideas of dataclass enhancements (I am using the backported version for some time now). Where should I post them: here, on Python's official issue tracker, on python-ideas, on python-dev (probably not there), elsewhere?

(My "enhancements" I do not mean full-blown PEP. Those are smaller caliber than PEP.)

[ericvsmith]

@pawelswiecki : I'd bring it up on python-ideas.

[reinhrst]

It does work for me to add the stub file to the package:

In site-packages/:

• move the dataclasses.py file to dataclasses/__init__.py
• add an empty dataclasses/py.typed file (see https://www.python.org/dev/peps/pep-0561/)
• Actually, it works for me with or without the https://github.com/python/typeshed/blob/master/stdlib/3.7/dataclasses.pyi as dataclasses/__init__.pyi. However I'm sure there are some situations that need this stub file.

Afterwards mypy is able to check my dataclasses (check the __init__() method). Not sure about what else.

[pawelswiecki]

I cannot move dataclasses.py into a dictionary, since deployment scripts would have to be modified, which is out of the question, unfortunately. Development and production envs should be as similar to one another as possible.

[reinhrst]

My response was a reply to @ericvsmith 's request to check if installing the stubs work. I suggest the dataclasses package gets reworked into this format (rather than just installing the dataclasses.py file), so that it can contain a py.typed file (or maybe there is a better solution that I don't know about)

[pawelswiecki]

Oh, right. Yes, this is the easiest approach, I think.

[ericvsmith]

Does this problem only exist in the 3.6 backport version of dataclasses? I apologize for not being very familiar with stubs. Do the stubs work differently with the 3.7 stdlib version of dataclasses?

[gvanrossum]

ATM the typeshed stubs for dataclasses are only present for 3.7: https://github.com/python/typeshed/blob/master/stdlib/3.7/dataclasses.pyi

It makes sense to add a copy of that file to the backport, and for that to work the backport should indeed be turned into a directory. It should also contain a py.typed file (see PEP 561).

[micseydel]

@pawelswiecki did you end up getting a solution? I'm having this issue with Python 3.7 and mypy 0.620 on OS X, no backports.

Using MYPYPATH to specify a directory containing manually downloaded dataclasses.pyi gives me "invalid syntax" on line 7 (the first non-whitespace character).

[pawelswiecki]

I did not.

It's strange that the same problem occurs for Python 3.7: dataclasses should work with mypy==0.620 out of the box.

[micseydel]

Just tried to reproduce on Linux, and got a different issue that was solved by adding --python-version 3.7. Tried to repro on the same OS X project as before and did, but then that same flag fixed the different issue. Doesn't help you, unfortunately, but might help others coming across this thread, even though it's a workaround more than a fix.

[gvanrossum]

This should be solved once https://github.com/python/typeshed/pull/2354 is merged (and a mypy release including that is distributed, probably mypy 0.630).

[gvanrossum]

That PR got merged; you can close this issue now, or you can keep it open until mypy 0.630 is out.

[ericvsmith]

Thanks for the info. I'll wait until 0.630 is out and I can verify it.

[huyz]

@ericvsmith What's the status of this issue?

I'm curious because I'm getting a similar error with py3.6.7, mypy 0.910, and dataclasses 0.8

error: Library stubs not installed for "dataclasses" (or incompatible with Python 3.6)

[ericvsmith]

@huyz : It seems some of the above information is contradictory. I'm not a big mypy user. but I'm willing to look at a PR.

[huyz]

@ericvsmith Never mind, mypy has recently changed and we now have to do mypy --install-types to get the types.

Separate question: the download types is version 0.6.1, while dataclasses is at 0.8. I guess you don't maintain these types then?

[ericvsmith]

I'm glad you figured it out.

Correct: I don't maintain the types.

I'm going to close this issue, since it seems to work with recent enough versions of everything.