Open fdev31 opened 13 years ago
rubik
commented
13 years ago Argh... Thank you for pointing this out. I get the same error both on Python2.6 and Python2.7...
P.S. This does not concern the problem, but you can install packages from launchpad using the bzr+url syntax:
$ pyg install bzr+lp:dreampie
....I came to the conclusion that it's a distutils bug.
When we use --single-version-externally-managed setuptools wants a --record option too. But when we set the record it fails (even if we are launching a simple python setup.py install --record <path>). Here is dreampie setup:
$ python setup.py install --record "pyg-record-test.rec"
running install
running build
running build_py
running build_scripts
running build_subp_lib
running install_lib
running install_scripts
changing mode of /home/miki/env2/bin/dreampie to 755
running install_data
running install_egg_info
Removing /home/miki/env2/lib/python2.6/site-packages/dreampie-1.1-py2.6.egg-info
Writing /home/miki/env2/lib/python2.6/site-packages/dreampie-1.1-py2.6.egg-info
running install_subp_lib
writing list of installed files to 'pyg-record-test.rec'
Traceback (most recent call last):
File "setup.py", line 161, in <module>
setup(**setup_args)
File "/usr/lib/python2.6/distutils/core.py", line 152, in setup
dist.run_commands()
File "/usr/lib/python2.6/distutils/dist.py", line 975, in run_commands
self.run_command(cmd)
File "/usr/lib/python2.6/distutils/dist.py", line 995, in run_command
cmd_obj.run()
File "/usr/lib/python2.6/distutils/command/install.py", line 642, in run
self.record)
File "/usr/lib/python2.6/distutils/cmd.py", line 358, in execute
util.execute(func, args, msg, dry_run=self.dry_run)
File "/usr/lib/python2.6/distutils/util.py", line 408, in execute
func(*args)
File "/usr/lib/python2.6/distutils/file_util.py", line 252, in write_file
f.write(line + "\n")
TypeError: can only concatenate list (not "str") to listdreampie uses distutils. If we manage to replace that function with setuptools' one it could work.
fdev31
commented
13 years ago Did you spot the function we have to replace ? To me it might be a complex use-case with custom Command defined, don't know if setuptools can override that easily, we will have to check :P
rubik
commented
13 years ago With the changes of the above commit it works, but we shouldn't remove the --single-version-externally-managed option.
As you can see, setuptools automatically redirects to distutils:
File "/usr/local/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
--> File "/usr/local/lib/python2.7/site-packages/setuptools/command/install.py", line 53, in run
--> return _install.run(self)
File "/usr/local/lib/python2.7/distutils/command/install.py", line 590, in run
self.record)_install refers to the distutils' install command.
Maybe we can override the write_file function in distutils/file_util.py...
fdev31
commented
13 years ago I think distutils hacks / special cases handling could wait for a next release so we can try to provide a clean "exceptions" handling... did you manage to get a compatibility layer with other frameworks like distutils2 ?
rubik
commented
13 years ago Yes, you're right, I'll change the milestone. As for distutils2, I think we can improve pkgtools, maybe creating a new version (like pkgtools2 or something), that handles both frameworks.
fdev31
commented
13 years ago Short answer: Oh nice, would you be interested in some pkgtools.egg submodule ? Long answer: Oh, I didn't realize you own pkgtools as well... I don't know if you remember but I talked about an alternative packaging format (a zip containing an .egg plus a .py file to run the application...). You can read some bits of it here: http://zicbee.gnux.info/hg/zicbee-workshop/file/a70dbb7649ae/manage (The the Installer class), that code started from a script-like thing but now it's too limited for this model, an integration + improvement into a wider-target library could be nice... don't you think ?
rubik
commented
13 years ago Sorry for the delay but I wasn't at home.
I checked zicbee (I didn't know it was yours! LOL). That's really interesting! bee.egg looks like a bundle with the EGG-INFO file, so we could simply create a bundle with all the required packages and then add metadata.
fdev31
commented
13 years ago Don't apologies, I can't tell I'm very reactive at the moment :P Heh, did you really know about zicbee before checking this code ? I wasn't expecting real users/testers ;) You got the idea, this packaging technique is more or less a bundle, the only tricky part is to "merge" EGG-INFO (and allow the user to customize the way it's merged, there is some hard-coded structure for that in zicbee....). I would be happy to be able to remove my ad-hoc code & use something else to distribute zicbee & others :P
rubik
commented
13 years ago Heh, did you really know about zicbee before checking this code ? I wasn't expecting real users/testers ;)
No, I didn't. But that's probably me, I don't have big playlists and I dont't listen to music very often!
And regarding that packaging technique, do you want it to be included in this release or in the next ones? I think we can extend a little the bundle method, it shouldn't be so difficult.
fdev31
commented
13 years ago I would prefer "as soon as possible" but I can easily wait for the next release... if you have some time, just feel free ;) I think it's a nice feature but there is no emergency.
rubik
commented
13 years ago Ok, since I find it very interesting I started working on it. I have modified the Bundler, made it more flexible.
fdev31
commented
13 years ago Oh great, thanks ! I was playing with features I rarely use an find some behavior a little odd (this is the novice eyes ;)), Maybe you could explain me how it's supposed to work so we can spot the improvement points:
Unknown error occurred: CalledProcessError()
unlike scm+URL format, install will accept repositories without the #egg=PKGNAME parameter when given a plain url to a tarball, the package is correctly installed but:
-e is not working (as for local folder install, this is almost the same...)
See this session:(standard)fab@fab-1015PN:/tmp% pyg install -A https://github.com/rubik/pyg/tarball/master
Installing master from https://github.com/rubik/pyg/tarball/master
Running setup.py egg_info for master
Running setup.py install for master
master installed successfully
(standard)fab@fab-1015PN:/tmp% pyg install -A -e https://github.com/rubik/pyg/tarball/master
Error: URL should start with one of these schemes:
('git+', 'hg+', 'bzr+', 'svn+', 'dir+')Ah yes git is a bit complicated. To make it work I had to copy my .ssh folder from my home to the root (because I run pyg install through sudo), but I never stumbled into that error...
As for the other problem, install accepts, among the others, two inputs:
Now, in the first example you are installing Pyg from an URL (that points to the tarball), but that installs only the specified package, not its dependecies (I don't know whether this is a bug or not). In the latter example you are using the -e option, that requires the scm prefix. Yeah it is not clear! :) But at the moment it is just a copy of the -e option in Pip, we could make it accepts all install's argument in the future.
rubik
commented
13 years ago I'm going to rewrite (in part) the Bundler: I don't like it now because it is not very usable...
fdev31
commented
13 years ago Ok, could you think about some feature at the same time ? :) Still in the goal of packaging python programs for non-python developers (or non-developers): When we use "pyg bundle" it will fetch all sources etc... not using currently installed sources.
If I "python setup.py install" a development package, I expect it to be bundled instead of the released one, don't you agree ?
I think pip have some ways to workaround it (with a cache folder) but it's not very easy as well.
Another remark almost linked to bundle: the freeze command have the same name as in pip, but I think it's very badly named (it's more something like "packages" or "show" or something like that, in ONE world it's not obvious but I think it's possible to find a better name :))
rubik
commented
13 years ago I managed to write something, but I'm going to open a new issue for that. How do you want to call that bundle/egg/pack? The file-extension I mean. I saw you called yours "Zicbee Pack", but on my system the .pack extension is already linked to Pack200 Java archives, I don't know what they are. So in the example I called it pypack, but we should determine the extension.
For the other bundle feature, we can leave the default process as it is, and add an option for that. It is tricky, though... Regarding the freeze command, we can easily replace it with another name. I think "show" and "packages" are good alternatives as well, what do you think?
fdev31
commented
13 years ago About the names, I like "show" since it's short... or "installed" ? (people type that fast since it's a common word)
pyg siteShowing some site informations (preceded by '#') + package list can be great as well (in fact that's my favourite)... are comment allowed in requirements file ? If not, let's go for "show" but I would naturally switch "list" & "show" (as in "list packages" and "show versions").
rubik
commented
13 years ago Yes, comments are allowed in requirements file. So let's go for pyg site, shall I open a new issue?
fdev31
commented
13 years ago Yep, using the bugtracker to keep track of the ideas is not a bad idea :)
rubik
commented
13 years ago From tomorrow I'll work on the new bundle feature you pointed out! :)
fdev31
commented
13 years ago Great ! but which ones ? :) I don't know if I'll be free enough but I'll try to improve the pack command, especially the generated script which is just a place holder for now (thanks for that :)).
rubik
commented
13 years ago I'll start from pyg site and then I'll work on the other one.
I think there is some mangling between python 2.x & 3.x here, need to analyse...