search

nodesign / weio

weio
http://we-io.net
128 stars 35 forks source link

`pip` not working #132

Closed drasko closed 8 years ago

drasko commented 9 years ago 
root@WEIO:/# pip install ctypes
Downloading/unpacking ctypes
  You are installing an externally hosted file. Future versions of pip will default to disallowing externally hosted files.
  You are installing a potentially insecure and unverifiable file. Future versions of pip will default to disallowing insecure files.
  Downloading ctypes-1.0.2.tar.gz (385kB): 385kB downloaded
  Running setup.py egg_info for package ctypes
    Traceback (most recent call last):
      File "<string>", line 16, in <module>
      File "/tmp/pip_build_root/ctypes/setup.py", line 33, in <module>
        from distutils.cygwinccompiler import Mingw32CCompiler
    ImportError: No module named cygwinccompiler
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 16, in <module>

  File "/tmp/pip_build_root/ctypes/setup.py", line 33, in <module>

    from distutils.cygwinccompiler import Mingw32CCompiler

ImportError: No module named cygwinccompiler

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_root/ctypes
Storing complete log in /root/.pip/pip.log
root@WEIO:/# cat /root/.pip/pip.log
Downloading/unpacking ctypes

  Getting page https://pypi.python.org/simple/ctypes/
  URLs to search for versions for ctypes:
  * https://pypi.python.org/simple/ctypes/
  Analyzing links from page https://pypi.python.org/simple/ctypes/
    Found link http://downloads.sourceforge.net/project/ctypes/ctypes/1.0.2/ctypes-1.0.2.tar.gz (from https://pypi.python.org/simple/ctypes/), version: 1.0.2
    Skipping link http://sourceforge.net/project/showfiles.php?group_id=71702 (from https://pypi.python.org/simple/ctypes/); unknown archive format: .php
    Skipping link http://starship.python.net/crew/theller/ctypes.html (from https://pypi.python.org/simple/ctypes/); unknown archive format: .html
    Skipping link http://starship.python.net/crew/theller/ctypes/ (from https://pypi.python.org/simple/ctypes/); not a file
  You are installing an externally hosted file. Future versions of pip will default to disallowing externally hosted files.

  You are installing a potentially insecure and unverifiable file. Future versions of pip will default to disallowing insecure files.

  Downloading from URL http://downloads.sourceforge.net/project/ctypes/ctypes/1.0.2/ctypes-1.0.2.tar.gz (from https://pypi.python.org/simple/ctypes/)
  Running setup.py egg_info for package ctypes

    Traceback (most recent call last):

      File "<string>", line 16, in <module>

      File "/tmp/pip_build_root/ctypes/setup.py", line 33, in <module>

        from distutils.cygwinccompiler import Mingw32CCompiler

    ImportError: No module named cygwinccompiler

    Complete output from command python setup.py egg_info:

    Traceback (most recent call last):

  File "<string>", line 16, in <module>

  File "/tmp/pip_build_root/ctypes/setup.py", line 33, in <module>

    from distutils.cygwinccompiler import Mingw32CCompiler

ImportError: No module named cygwinccompiler

----------------------------------------

Cleaning up...

  Removing temporary dir /tmp/pip_build_root...
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_root/ctypes

Exception information:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg/pip/basecommand.py", line 134, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg/pip/commands/install.py", line 236, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/usr/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg/pip/req.py", line 1134, in prepare_files
    req_to_install.run_egg_info()
  File "/usr/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg/pip/req.py", line 259, in run_egg_info
    command_desc='python setup.py egg_info')
  File "/usr/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg/pip/util.py", line 670, in call_subprocess
    % (command_desc, proc.returncode, cwd))
InstallationError: Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_root/ctypes
root@WEIO:/#
nmarcetic commented 9 years ago

I think its not a pip issue. Can you pls try to pass --allow-all-external. pip install ctypes --allow-all-external I dont have a board with me.

drasko commented 9 years ago

pip is broken - both for Paul and me. Different tests.

On Wed, Feb 4, 2015 at 10:48 PM, Nikola notifications@github.com wrote:

I think its not a pip issue. Can you pls try to pass --allow-all-external. pip install ctypes --allow-all-external I dont have a board with me.

— Reply to this email directly or view it on GitHub https://github.com/nodesign/weio/issues/132#issuecomment-72946711.

nmarcetic commented 9 years ago

ok i will take a look.

captainigloo commented 9 years ago

Hi Drasko, I would like install pyephem pip package, but don't work :

cmd : pip install pyephem

My pip log :

Downloading/unpacking pyephem

Getting page https://pypi.python.org/simple/pyephem/ URLs to search for versions for pyephem:

running build

running build_py

creating build

creating build/lib.linux-mips-2.7

creating build/lib.linux-mips-2.7/ephem

copying ephem/cities.py -> build/lib.linux-mips-2.7/ephem

copying ephem/init.py -> build/lib.linux-mips-2.7/ephem

copying ephem/stars.py -> build/lib.linux-mips-2.7/ephem

creating build/lib.linux-mips-2.7/ephem/tests

copying ephem/tests/test_usno.py -> build/lib.linux-mips-2.7/ephem/tests

copying ephem/tests/test_dates.py -> build/lib.linux-mips-2.7/ephem/tests

copying ephem/tests/test_stars.py -> build/lib.linux-mips-2.7/ephem/tests

copying ephem/tests/test_launchpad_244811.py -> build/lib.linux-mips-2.7/ephem/tests

copying ephem/tests/test_github_issues.py -> build/lib.linux-mips-2.7/ephem/tests

copying ephem/tests/init.py -> build/lib.linux-mips-2.7/ephem/tests

copying ephem/tests/test_satellite.py -> build/lib.linux-mips-2.7/ephem/tests

copying ephem/tests/test_usno_equinoxes.py -> build/lib.linux-mips-2.7/ephem/tests

copying ephem/tests/test_rst.py -> build/lib.linux-mips-2.7/ephem/tests

copying ephem/tests/test_constants.py -> build/lib.linux-mips-2.7/ephem/tests

copying ephem/tests/test_bodies.py -> build/lib.linux-mips-2.7/ephem/tests

copying ephem/tests/test_jpl.py -> build/lib.linux-mips-2.7/ephem/tests

copying ephem/tests/test_launchpad_236872.py -> build/lib.linux-mips-2.7/ephem/tests

copying ephem/tests/test_cities.py -> build/lib.linux-mips-2.7/ephem/tests

copying ephem/tests/test_locales.py -> build/lib.linux-mips-2.7/ephem/tests

copying ephem/tests/test_observers.py -> build/lib.linux-mips-2.7/ephem/tests

copying ephem/tests/test_angles.py -> build/lib.linux-mips-2.7/ephem/tests

creating build/lib.linux-mips-2.7/ephem/doc

copying ephem/doc/quick.rst -> build/lib.linux-mips-2.7/ephem/doc

copying ephem/doc/examples.rst -> build/lib.linux-mips-2.7/ephem/doc

copying ephem/doc/radec.rst -> build/lib.linux-mips-2.7/ephem/doc

copying ephem/doc/index.rst -> build/lib.linux-mips-2.7/ephem/doc

copying ephem/doc/newton.rst -> build/lib.linux-mips-2.7/ephem/doc

copying ephem/doc/coordinates.rst -> build/lib.linux-mips-2.7/ephem/doc

copying ephem/doc/rise-set.rst -> build/lib.linux-mips-2.7/ephem/doc

copying ephem/doc/date.rst -> build/lib.linux-mips-2.7/ephem/doc

copying ephem/doc/catalogs.rst -> build/lib.linux-mips-2.7/ephem/doc

copying ephem/doc/angle.rst -> build/lib.linux-mips-2.7/ephem/doc

copying ephem/doc/reference.rst -> build/lib.linux-mips-2.7/ephem/doc

copying ephem/doc/tutorial.rst -> build/lib.linux-mips-2.7/ephem/doc

copying ephem/doc/CHANGELOG.rst -> build/lib.linux-mips-2.7/ephem/doc

copying ephem/doc/faq.rst -> build/lib.linux-mips-2.7/ephem/doc

creating build/lib.linux-mips-2.7/ephem/tests/jpl

copying ephem/tests/jpl/oberon.txt -> build/lib.linux-mips-2.7/ephem/tests/jpl

copying ephem/tests/jpl/uranus.txt -> build/lib.linux-mips-2.7/ephem/tests/jpl

copying ephem/tests/jpl/saturn.txt -> build/lib.linux-mips-2.7/ephem/tests/jpl

copying ephem/tests/jpl/neptune.txt -> build/lib.linux-mips-2.7/ephem/tests/jpl

copying ephem/tests/jpl/jupiter.txt -> build/lib.linux-mips-2.7/ephem/tests/jpl

copying ephem/tests/jpl/phobos.txt -> build/lib.linux-mips-2.7/ephem/tests/jpl

copying ephem/tests/jpl/europa.txt -> build/lib.linux-mips-2.7/ephem/tests/jpl

copying ephem/tests/jpl/hyperion.txt -> build/lib.linux-mips-2.7/ephem/tests/jpl

copying ephem/tests/jpl/mars.txt -> build/lib.linux-mips-2.7/ephem/tests/jpl

creating build/lib.linux-mips-2.7/ephem/tests/usno

copying ephem/tests/usno/astrom_mercury.txt -> build/lib.linux-mips-2.7/ephem/tests/usno

copying ephem/tests/usno/moon_phases.txt -> build/lib.linux-mips-2.7/ephem/tests/usno

copying ephem/tests/usno/riset_sun.txt -> build/lib.linux-mips-2.7/ephem/tests/usno

copying ephem/tests/usno/appgeo_moon.txt -> build/lib.linux-mips-2.7/ephem/tests/usno

copying ephem/tests/usno/risettran_rigel.txt -> build/lib.linux-mips-2.7/ephem/tests/usno

copying ephem/tests/usno/apptopo_moon.txt -> build/lib.linux-mips-2.7/ephem/tests/usno

copying ephem/tests/usno/apptopo_deneb.txt -> build/lib.linux-mips-2.7/ephem/tests/usno

copying ephem/tests/usno/astrom_antares.txt -> build/lib.linux-mips-2.7/ephem/tests/usno

copying ephem/tests/usno/riset_moon.txt -> build/lib.linux-mips-2.7/ephem/tests/usno

copying ephem/tests/usno/appgeo_deneb.txt -> build/lib.linux-mips-2.7/ephem/tests/usno

copying ephem/tests/usno/risettran_moon.txt -> build/lib.linux-mips-2.7/ephem/tests/usno

copying ephem/tests/usno/appgeo_sun.txt -> build/lib.linux-mips-2.7/ephem/tests/usno

copying ephem/tests/usno/astrom_neptune.txt -> build/lib.linux-mips-2.7/ephem/tests/usno

copying ephem/tests/usno/apptopo_sun.txt -> build/lib.linux-mips-2.7/ephem/tests/usno

copying ephem/tests/usno/appgeo_jupiter.txt -> build/lib.linux-mips-2.7/ephem/tests/usno

running build_ext

building 'ephem._libastro' extension

creating build/temp.linux-mips-2.7

creating build/temp.linux-mips-2.7/extensions

creating build/temp.linux-mips-2.7/libastro-3.7.5

creating build/temp.linux-mips-2.7/extensions/data

mips-openwrt-linux-uclibc-gcc -fno-strict-aliasing -Os -pipe -mno-branch-likely -mips32r2 -mtune=34kc -fno-caller-saves -fhonour-copts -Wno-error=unused-but-set-variable -msoft-float -DNDEBUG -fno-inline -DNDEBUG -Os -pipe -mno-branch-likely -mips32r2 -mtune=34kc -fno-caller-saves -fhonour-copts -Wno-error=unused-but-set-variable -msoft-float -fPIC -Ilibastro-3.7.5 -I/usr/include/python2.7 -c extensions/_libastro.c -o build/temp.linux-mips-2.7/extensions/_libastro.o

unable to execute 'mips-openwrt-linux-uclibc-gcc': No such file or directory

error: command 'mips-openwrt-linux-uclibc-gcc' failed with exit status 1

Cleaning up...

Removing temporary dir /tmp/pip_build_root... Command /usr/bin/python2.7 -c "import setuptools;file='/tmp/pip_build_root/pyephem/setup.py';exec(compile(open(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-WFvZOT-record/install-record.txt --single-version-externally-managed failed with error code 1 in /tmp/pip_build_root/pyephem

Exception information: Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg/pip/basecommand.py", line 134, in main status = self.run(options, args) File "/usr/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg/pip/commands/install.py", line 241, in run requirement_set.install(install_options, global_options, root=options.root_path) File "/usr/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg/pip/req.py", line 1298, in install requirement.install(install_options, global_options, _args, *_kwargs) File "/usr/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg/pip/req.py", line 625, in install cwd=self.source_dir, filter_stdout=self._filter_install, show_stdout=False) File "/usr/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg/pip/util.py", line 670, in call_subprocess % (command_desc, proc.returncode, cwd)) InstallationError: Command /usr/bin/python2.7 -c "import setuptools;file='/tmp/pip_build_root/pyephem/setup.py';exec(compile(open(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-WFvZOT-record/install-record.txt --single-version-externally-managed failed with error code 1 in /tmp/pip_build_root/pyephem

captainigloo commented 9 years ago

I tested with option "--allow-all-external" and it's same error.

ukicar commented 9 years ago

hello, The problem is not in pip but in the module that needs to be cross compiled. Pyephem module is not pure Python, it's mixed with some code that was written in C so needs to be compiled. As WeIO is embedded linux platform with limited ressources we don't have compiler tools installed in it so the solution is to cross compile your module on the pc before installing it. Let us try to cross compile this package because we will need this experience in the future...

captainigloo commented 9 years ago

Ok, I await your cross compilation. But how are you to automate the compilation OpenWRT in WEIO?

drasko commented 9 years ago

OpenWrt cross-compiles it's packages using modified buildroot( http://buildroot.uclibc.org/). WeIO has only 16MB flash to hold LInux, rootfs with Python and user-space applications - is an embedded machine (not a little PC like RPi). Gcc is big piece of code, and practically impossible to put in so little space. That's why OpenWrt doe not provide support for holding the complier on PC.

The easiest case for using pip and other Python installation is when we have pure Python packages. Then it just works.

When we have to cross-complie, we have to use something like this: http://wiki.openwrt.org/doc/devel/packages, then cross-comple package, put it on the board and install it using opkg.

This is the biggest levelof automation that we can have (there is no other possibility for non-pure Python packages). This is how OpenWrt works.

On Sun, Mar 1, 2015 at 11:30 AM, Sébastien Joly notifications@github.com wrote:

Ok, I await your cross compilation. But how are you to automate the compilation OpenWRT in WEIO?

— Reply to this email directly or view it on GitHub https://github.com/nodesign/weio/issues/132#issuecomment-76591179.

captainigloo commented 9 years ago

Ok I understand. I bought the WEIO for a solar tracking specific project and measures of insolation with this package pyephem. It is the basis of my development. I'll go back to a Raspberry PI. I'll find another project for WEIO.

drasko commented 9 years ago

We have to take a look at pyephem - maybe it is not so difficult to port.

On Sun, Mar 1, 2015 at 12:10 PM, Sébastien Joly notifications@github.com wrote:

Ok I understand. I bought the WEIO for a solar tracking specific project and measures of insolation with this package pyephem. It is the basis of my development. I'll go back to a Raspberry PI. I'll find another project for WEIO.

— Reply to this email directly or view it on GitHub https://github.com/nodesign/weio/issues/132#issuecomment-76592350.

drasko commented 9 years ago

@captainigloo upon your special request ;) we have crafted a nice script to cross-compile Python packages more easily. Please find the doc here: https://github.com/nodesign/weioWiki/wiki/Cross-Compiling-Py

Now, this might be a little tedious because one must compile OpenWrt first to have cross-compiler.

That's why I propose that we create a repo where we can keep .egg files for WeIO that users request/contribute, so that they can just download them and install directly.

We'll re-post as soon as we have such a doc.

drasko commented 8 years ago

Looks like new OpenWrt version will resolve this one.

ks156 commented 8 years ago

Chaos Calmer fix the problem. pip dependancies must be installed that way : opkg install python-xml python-distutils

And here's the result with python-twitter : 

root@WEIO:/# pip install python-twitter
Downloading/unpacking python-twitter
  Downloading python-twitter-2.2.tar.gz (66kB): 66kB downloaded
  Running setup.py egg_info for package python-twitter

    no previously-included directories found matching '.DS_Store'
Downloading/unpacking requests (from python-twitter)
  Downloading requests-2.8.1.tar.gz (480kB): 480kB downloaded
  Running setup.py egg_info for package requests

Downloading/unpacking requests-oauthlib (from python-twitter)
  Downloading requests-oauthlib-0.5.0.tar.gz (54kB): 54kB downloaded
  Running setup.py egg_info for package requests-oauthlib

Downloading/unpacking oauthlib>=0.6.2 (from requests-oauthlib->python-twitter)
  Downloading oauthlib-1.0.3.tar.gz (109kB): 109kB downloaded
  Running setup.py egg_info for package oauthlib

Installing collected packages: python-twitter, requests, requests-oauthlib, oauthlib
  Running setup.py install for python-twitter
    warning: build_py: byte-compiling is disabled, skipping.

    warning: install_lib: byte-compiling is disabled, skipping.

    no previously-included directories found matching '.DS_Store'
  Running setup.py install for requests

    warning: build_py: byte-compiling is disabled, skipping.

    warning: install_lib: byte-compiling is disabled, skipping.

  Running setup.py install for requests-oauthlib
    warning: build_py: byte-compiling is disabled, skipping.

    warning: install_lib: byte-compiling is disabled, skipping.

  Running setup.py install for oauthlib
    warning: build_py: byte-compiling is disabled, skipping.

    warning: install_lib: byte-compiling is disabled, skipping.

Successfully installed python-twitter requests requests-oauthlib oauthlib
Cleaning up...
root@WEIO:/# python
Python 2.7.9 (default, Oct 19 2015, 13:56:16)
[GCC 4.8.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import twitter
>>> dir(twitter)
['Api', 'DirectMessage', 'Hashtag', 'List', 'ParseTweet', 'Status', 'Trend', 'TwitterError', 'Url', 'User', 'UserStatus', '_FileCache', '__author__', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', '__version__', '_file_cache', 'api', 'direct_message', 'error', 'hashtag', 'list', 'md5', 'parse_tweet', 'simplejson', 'status', 'trend', 'url', 'user']
>>>
drasko commented 8 years ago

@ks156 this one is interesting. Note that python-pip is present via CC feeds. However, when I included it in menuconfig, it selected all Python packages as a dependencies. Now I see why - looks like it needs python-xml, python-distutils and maybe some other stuff.

I propose thus:

Opinions?

ks156 commented 8 years ago

Agree 100%. You handle it ?

drasko commented 8 years ago

I was hoping that you would take it :). I would like to do some testing.

ks156 commented 8 years ago

Ok, I'll do it. :D

drasko commented 8 years ago

Not urgent. We still have a few days before release ;).

ks156 commented 8 years ago

No problem. I'll do it tonight. After that, I'll recompile all from scratch, to be sure all is OK. So the result will be for tomorrow

ks156 commented 8 years ago

Fixed by v1.2. pip is compiled with openwrt.