Open vltr opened 6 years ago
Hey @vltr, thanks for giving my tool a try! It's a huge motivator to see this kind of constructive feedback. Lets go point by point.
Correct, your case won't work with static generation. There is no way to generally handle such cases using static analysis without simulating the code, and in that case why bother with static analysis?
However, note that mkinit does have a dynamic backend, although it isn't fully hooked up yet. However, it wouldn't be out of the question to put a regular expression attribute name matcher that could be passed to the command line tool. Perhaps there could even be a __mkinit_export__
directive to explicitly state what should be exported. I'm planning on hooking up / documenting how to use the dynamic backend, but I'm not sure when I'll be able to get to it.
I think the idea is good, but I'm not in love with the using comments to define code that will be executed as a directive. I think defining a regex and using a pattern matching solution might do the trick. It is going to take some thought to find the right implementation.
Yes, there does need to be a mechanism for this preference. This should actually be pretty easy to do in the current architecture. If you want to try tackling this in a PR let me know, otherwise I'll get to it once I have some time.
This feature is definitely on my want list. This should be another easy one implementation wise. Probably just a few line changes and passing down a flag variable. Again, PRs are welcome, but I do want this feature in a 0.1.0 release.
Honestly, I didn't know that you could silence the linters by having an __all__
variable in your init file. It makes me happy to know I'll never need to write another # flake8: noqa
again. It should be simple to add this feature by modifying mkinit.static_mkinit._initstr
.
I'm glad to help, @Erotemic !
Just like black
, I think this tool can make a lot of sense in Python development. I'll go by each point as you did:
__all__
variable of each file could be an extra feature of this tool? I mean, __all__
variables directly influence the result of the generated __init__.py
file.Anyway, one thing that came to mind now is that using AUTOGEN_INIT_
variables could raise some linter noises, which leads us to use # noqa
on them ... Or use them inside comments, just like most linters works (including # noqa
). I think it's just a case of finding a balance :muscle:
I wasn't aware of black
(I usually use autopep8). I'll check it out.
I would like to eventually make it so you can ask mkinit to run dynamically when running it from the command line. Currently there exists a way to run dynamically from within python, but to autogenerate anything you need to use a command line flag (when running your program), which isn't very clean. Now that I've separated out the logic for output formatting in the static version, the next step for this point is to hook it into the dynamic version.
Its funny that you link to timeit. I made my Timerit tool as an alternative exactly because timeit requires code represented as strings to run. I've been planning on separating it from ubelt and making it a standalone module from some time now, but writing this comment gave me the motivation to actually do it. :smile:
Understood, I think I actually handled points 3, 4, and 5, in #3
My earlier point was not about # NOQA
on a single line, I'm ok with that. I'm much less ok with # flake8: noqa
because it suppresses linting on an entire file.
For the remaining point 1 and 2, I'm going to make new issues. I'll leave this issue open as a milestone until the all points are satisfied.
Sorry for the delay! I was using autopep8 as well, but black
provides such concise code formatting that's almost a pity not to use it. Of course, it's still on beta and you have to add one or two warnings into your flake8 ignore list, but for now that's okay. You can include it in your Makefile
or git hook, well, it's an awesome tool. Just a tip: use the --safe
flag :smile:
I understood what you told about # flake8: noqa
... I was just figuring out if using variables inside __init__.py
like AUTOGEN_INIT_ALL = True # noqa
was a good idea (I mean, it was my suggestion (quite vague actually), so I just wanted to give something else to think about). For now, this looks really good! Thanks for your time, @Erotemic !
@vltr,
The newest 0.2.0
release begins to handle points 1 and 2. You can now specify variables in the __init__.py
that modify the behavior of autogeneration.
While this doesn't implement pattern matching, you can use the __explicit__
, __private__
, and __protected__
variables to define lists of variable / module names. This is what they do:
__all__
variable. (useful when you have custom and auto-generated code). If you think pattern matching is absolutely necessary, it may be reasonable to allow glob-style strings as items in these lists. I think if that feature lands, then all 5 points will be checked off correct?
Note, that the first feature can be handled by specifying an __all__
variable in the module where you only want to export very specific things. By default mkinit
will respect a submodule's __all__
variable.
Hello @Erotemic ! Thanks a lot for the heads up, I had no time to add mkinit
to any script or automatic process (yet), but I'm working on some tools that may lead me to do so (I hope).
If you think pattern matching is absolutely necessary, it may be reasonable to allow glob-style strings as items in these lists. I think if that feature lands, then all 5 points will be checked off correct?
Well, yes. I really don't mind (myself) about pattern matching, but I've seen countless Python source codes to know that this is in fact a requirement, but I think a glob matching would do almost all the job. Example glob matching patterns I have found in the wild (but I don't recall now where (exactly)):
_*
: generally not available in __init__.py
files, let's say, _Base
, _Container
, _Posix
, etc;*Models
: to be available, like I mentioned earlier in one of the comments;Abstract*
, *ABC
or *Enum
to be available;py*
to not be available (like py3
, py2
, etc);Other cases would require a more complex pattern matching, like any stdlib import, some internal flags (like DEBUG
, even though it can be part of __private__
) or any "full underscore" variable names, such as py3
or py2
, that sometimes are used in the source code specially when you're dealing with the presence of one lib and, if not present, fallback to another, like this snippet from the Sanic framework :wink:
But, as always, good work on mkinit
, it is really an exceptional tool for any Python utility tool belt IMHO :muscle:
Hi! First of all, thanks for this nice tool! I think this tool is something rarely perceived but a very common task for Python programmers and it is super welcome!
Anyway, not that this is an issue, I assume that there are plans to implement this (for linter happiness), but I could also give some thoughts about the
__all__
variable because not every single variable from a certain module is needed in the__init__.py
file, like an import from another module. So, I can define what I want inside__all__
and that's what will shows up in the__init__.py
file for that module, right?Well, I want to share some situations ... I'm not here to say "hey, this tool is worthless because it doesn't check this or that", but rather give my use case so you can evolve it in the right direction :wink: I really appreciate your effort!
models.py
file), and I don't want to export anything but my model classes, so I'll do something like this:for k in dir(): if k.startswith("_"): continue if k in locals() and inspect.isclass(locals().get(k)): if issubclass(locals().get(k), BaseModel): _all_locals.append(k)
all = tuple(_all_locals)
Module level imports: I think this should be optional, I don't find very useful to have a
__init__.py
file like this (see comments in code):Another nice option / feature would be to generate relative imports instead of the full module path, example:
For last (but not least important), to make linters and / or code style checkers happy, I think
mkinit
could (optionally) create the__all__
variable at the end of the file /AUTOGEN_INIT
block ... Assuming the above example:all = ( "Base", # from .core "BaseModel", # from .core "metadata", # from .core "Address", # from .distributed "Contact", # from .distributed "Device", # from .distributed )