pyjanitor-devs / pandas_flavor

The easy way to write your own flavor of Pandas
https://zsailer.github.io/software/pandas-flavor/
MIT License
301 stars 17 forks source link

lazy-loader breaks compatibility with PyInstaller #24

Closed bwoodsend closed 1 year ago

bwoodsend commented 2 years ago

The new lazy submodule loader breaks the dependency analysis of PyInstaller. Normally, we'd manually patch this up on PyInstaller's side but, given that pandas_flavours only has two submodules (minus the one which holds the package's version), one imports only pandas and functools, the other imports the first and functools and neither of them do any processing at all on initialisation, what actually is the point in using lazy loading here anyway? Asides from breaking PyInstaller's analysis (and just looking ugly), it screws with IDE completion and static analysis tools.

ericmjl commented 1 year ago

@bwoodsend apologies for responding late here.

I'm struggling to remember the exact reason for doing lazy loading, but the general reason was motivated by startup execution time when doing imports. One of my colleagues at work noticed that our CLI tools, which used pyjanitor as a dependency, was executing really slowly and through profiling, we might have traced performance issues all the way back to pandas_flavor. My apologies for any inconvenience here.

I think we'll need to rethink some of the underlying code organization of both pandas_flavor and pyjanitor to see whether there's a more efficient pattern. I've not had the time to think about the problem deeply, though, because of other life priorities. That said, we'll keep this issue open on the off chance that others may be able to take some time to think about this programming puzzle.

samukweku commented 1 year ago

@bwoodsend closing this issue as lazy loading has been removed. Thanks for the feedback