search

freeman-lab / pim

friendly command-line tool for initializing python packages
MIT License
18 stars 2 forks source link

Comparison to existing library #1

Open freeman-lab opened 8 years ago

freeman-lab commented 8 years ago

There's a python package by @emallson called nppm with similar interests but a different approach.

https://github.com/emallson/nppm

That library introduces a npm-like package.json, and seems to focus on dependency / environment management, as opposed to streamlining the process of creating and publishing, which was the primary interest here. Still, definitely worth exploring overlap / opportunities.

danielballan commented 8 years ago

See also https://pypi.python.org/pypi/flit

emallson commented 8 years ago

Hey! I got the notification about this issue and thought I'd chime in.

You got my focus right: my motivation was frustrations with keeping the requirements.txt up-to-date. This is a continual problem that I have with python projects that I rarely encounter in node because of how simple npm makes it. I had several instances where I was given a requirements.txt that was incomplete, and had to spend time digging through the code to determine the complete dependency list. This was doubly frustrating for dependencies that were only imported late in a long-running process (e.g. joblib being imported to save the output of an algorithm that took hours to run)

At this point, nppm is sort-of abandonware. I am not actively using or updating it, but if any issues come up I'll fix them.

I'm definitely interested in the overlap of these two projects because in some sense they are two sides of the same coin. Dependency management is crucial for publishing working modules, and having published working modules is necessary for dependency management to be useful. One idea I explored but never wrote code for was the possibility of having a generic setup.py that would read relevant information from my package.json to be able to publish a package.

One limitation of my approach was that by building on top of pip, I could not introduce the strict version-checking that I wanted. For example, with pip if you specify a dependency on A that depends on B-1.2 and later specify a dependency on B-2.4, the version of the second dependency will be silently ignored (if memory serves) instead of producing a warning or error. Ideally, this would produce a warning. I'm sure there are other problems (like it breaking with every pip version bump because I (ab)use some internal APIs), but that one sticks out in my memory as one that couldn't be fixed without dropping or modifying pip.