search

hobuinc / mgrs

Python MGRS library
https://pypi.python.org/pypi/mgrs
MIT License
111 stars 36 forks source link

Fix how libmgrs.so is imported if installing with Python 3.5. #10

Closed rguerra closed 8 years ago

rguerra commented 9 years ago

I tried installing the original mgrs from hobu's repo with Python 3.5 but it couldn't find the libmgrs.so library. I later realized that I had broken the installation for Python2.7, but just fixed it in the latest commit of this branch.

hobu commented 9 years ago

Is SOABI universal across all *nix?

rguerra commented 9 years ago

Hi,

To be honest, I don't know. I just assumed it was.

Raul

On Sun, Oct 18, 2015 at 8:22 PM, Howard Butler notifications@github.com wrote:

Is SOABI universal across all *nix?

— Reply to this email directly or view it on GitHub https://github.com/hobu/mgrs/pull/10#issuecomment-149036602.

hobu commented 9 years ago

Some googling shows this isn't universal. We would need a patch that detected the SOEXT reliably for all *nix before implementing this approach.

proximous commented 8 years ago

This pull request worked with Python 3.4 on Ubuntu 14.04. I'm confused why this pull isn't accepted? Am I reading the code wrong? If SOABI is not supported, won't this pull behave like the master? So even if this pull doesn't fix "all *unix" environments, it doesn't break anything that works with the master, and does fix a very popular distribution. So this pull makes this better (though not perfect) without causing harm. I would love to see it accepted!

hobu commented 8 years ago

I'm confused why this pull isn't accepted?

For the patch to be accepted, it must reliably detect the SOEXT for all unix flavors. A specific fix for a single distribution isn't good enough.

proximous commented 8 years ago

Sorry, I'm still confused. The current master doesn't work for all distributions. So isn't a fix that makes it work in more distributions a good thing? I agree all is best, but isn't more better than less? Do you believe this fix breaks or otherwise degrades operation where the master currently works?

hobu commented 8 years ago

When I test the following with Ubuntu 15.04, it all works:

git clone https://github.com/hobu/mgrs.git
cd mgrs
python3 setup.py build
python3 setup.py test
Doctest: test.txt ... ok

----------------------------------------------------------------------
Ran 1 test in 0.013s

OK

I don't understand why this patch is needed. What is producing the incorrect libmgrs.so filename in the first place?

proximous commented 8 years ago

Thanks for taking the time to look into this and for posting your test procedure. That really helped. I get a different result with Ubuntu 14.04:

git clone https://github.com/hobu/mgrs.git  
cd mgrs
python3 setup.py build
python3 setup.py test
Doctest: test.txt ... FAIL

======================================================================
FAIL: /home/jerry/mgrs/tests/test.txt
Doctest: test.txt
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.4/doctest.py", line 2187, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for test.txt
  File "/home/jerry/mgrs/tests/test.txt", line 0

----------------------------------------------------------------------
File "/home/jerry/mgrs/tests/test.txt", line 3, in test.txt
Failed example:
    import mgrs
Exception raised:
    Traceback (most recent call last):
      File "/usr/lib/python3.4/doctest.py", line 1318, in __run
        compileflags, 1), test.globs)
      File "<doctest test.txt[0]>", line 1, in <module>
        import mgrs
      File "/home/jerry/mgrs/mgrs/__init__.py", line 1, in <module>
        from . core import rt
      File "/home/jerry/mgrs/mgrs/core.py", line 35, in <module>
        rt = ctypes.CDLL(os.path.join(local_library_path, lib_name))
      File "/usr/lib/python3.4/ctypes/__init__.py", line 351, in __init__
        self._handle = _dlopen(self._name, mode)
    OSError: /home/jerry/mgrs/libmgrs.so: cannot open shared object file: No such file or directory

----------------------------------------------------------------------
Ran 1 test in 0.089s

FAILED (failures=1)
$ python3 --version
Python 3.4.3
$ python3 setup.py build
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.4
creating build/lib.linux-x86_64-3.4/mgrs
copying mgrs/core.py -> build/lib.linux-x86_64-3.4/mgrs
copying mgrs/__init__.py -> build/lib.linux-x86_64-3.4/mgrs
running build_ext
building 'libmgrs' extension
creating build/temp.linux-x86_64-3.4
creating build/temp.linux-x86_64-3.4/libmgrs
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I./libmgrs -I/usr/include/python3.4m -c libmgrs/mgrs.c -o build/temp.linux-x86_64-3.4/libmgrs/mgrs.o
In file included from libmgrs/mgrs.c:86:0:
libmgrs/mgrs.h:255:3: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
   void initlibmgrs();
   ^
libmgrs/mgrs.h:256:3: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
   void PyInit_libmgrs();
   ^
libmgrs/mgrs.c: In function ‘Make_MGRS_String’:
libmgrs/mgrs.c:412:3: warning: field width specifier ‘*’ expects argument of type ‘int’, but argument 3 has type ‘long int’ [-Wformat=]
   i += sprintf (MGRS+i, "%*.*ld", Precision, Precision, east);
   ^
libmgrs/mgrs.c:412:3: warning: field precision specifier ‘.*’ expects argument of type ‘int’, but argument 4 has type ‘long int’ [-Wformat=]
libmgrs/mgrs.c:417:3: warning: field width specifier ‘*’ expects argument of type ‘int’, but argument 3 has type ‘long int’ [-Wformat=]
   i += sprintf (MGRS+i, "%*.*ld", Precision, Precision, north);
   ^
libmgrs/mgrs.c:417:3: warning: field precision specifier ‘.*’ expects argument of type ‘int’, but argument 4 has type ‘long int’ [-Wformat=]
libmgrs/mgrs.c: In function ‘Convert_MGRS_To_Geodetic’:
libmgrs/mgrs.c:879:6: warning: suggest explicit braces to avoid ambiguous ‘else’ [-Wparentheses]
   if (!error_code)
      ^
libmgrs/mgrs.c: At top level:
libmgrs/mgrs.c:1344:6: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
 void initlibmgrs() {}
      ^
libmgrs/mgrs.c:1345:6: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
 void PyInit_libmgrs() {}
      ^
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I./libmgrs -I/usr/include/python3.4m -c libmgrs/utm.c -o build/temp.linux-x86_64-3.4/libmgrs/utm.o
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I./libmgrs -I/usr/include/python3.4m -c libmgrs/ups.c -o build/temp.linux-x86_64-3.4/libmgrs/ups.o
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I./libmgrs -I/usr/include/python3.4m -c libmgrs/tranmerc.c -o build/temp.linux-x86_64-3.4/libmgrs/tranmerc.o
libmgrs/tranmerc.c: In function ‘Convert_Transverse_Mercator_To_Geodetic’:
libmgrs/tranmerc.c:487:10: warning: variable ‘s’ set but not used [-Wunused-but-set-variable]
   double s;       /* Sine of latitude                        */
          ^
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I./libmgrs -I/usr/include/python3.4m -c libmgrs/polarst.c -o build/temp.linux-x86_64-3.4/libmgrs/polarst.o
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.4/libmgrs/mgrs.o build/temp.linux-x86_64-3.4/libmgrs/utm.o build/temp.linux-x86_64-3.4/libmgrs/ups.o build/temp.linux-x86_64-3.4/libmgrs/tranmerc.o build/temp.linux-x86_64-3.4/libmgrs/polarst.o -o build/lib.linux-x86_64-3.4/libmgrs.cpython-34m.so
$ python3 setup.py test                              
running test
running egg_info
creating mgrs.egg-info
writing requirements to mgrs.egg-info/requires.txt
writing mgrs.egg-info/PKG-INFO
writing dependency_links to mgrs.egg-info/dependency_links.txt
writing top-level names to mgrs.egg-info/top_level.txt
writing manifest file 'mgrs.egg-info/SOURCES.txt'
reading manifest file 'mgrs.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'mgrs.egg-info/SOURCES.txt'
running build_ext
copying build/lib.linux-x86_64-3.4/libmgrs.cpython-34m.so -> 
/home/jerry/mgrs/tests/test.txt
Doctest: test.txt ... FAIL

======================================================================
FAIL: /home/jerry/mgrs/tests/test.txt
Doctest: test.txt
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.4/doctest.py", line 2187, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for test.txt
  File "/home/jerry/mgrs/tests/test.txt", line 0

----------------------------------------------------------------------
File "/home/jerry/mgrs/tests/test.txt", line 3, in test.txt
Failed example:
    import mgrs
Exception raised:
    Traceback (most recent call last):
      File "/usr/lib/python3.4/doctest.py", line 1318, in __run
        compileflags, 1), test.globs)
      File "<doctest test.txt[0]>", line 1, in <module>
        import mgrs
      File "/home/jerry/mgrs/mgrs/__init__.py", line 1, in <module>
        from . core import rt
      File "/home/jerry/mgrs/mgrs/core.py", line 35, in <module>
        rt = ctypes.CDLL(os.path.join(local_library_path, lib_name))
      File "/usr/lib/python3.4/ctypes/__init__.py", line 351, in __init__
        self._handle = _dlopen(self._name, mode)
    OSError: /home/jerry/mgrs/libmgrs.so: cannot open shared object file: No such file or directory

----------------------------------------------------------------------
Ran 1 test in 0.089s

FAILED (failures=1)
proximous commented 8 years ago

I also have an Ubuntu 15.10 system with an ARM processor. I tried it there...same failure:

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 15.10
Release:    15.10
Codename:   wily
$ python3 --version                     
Python 3.4.3+
$ python3 setup.py build
running build
running build_py
creating build
creating build/lib.linux-armv7l-3.4
creating build/lib.linux-armv7l-3.4/mgrs
copying mgrs/core.py -> build/lib.linux-armv7l-3.4/mgrs
copying mgrs/__init__.py -> build/lib.linux-armv7l-3.4/mgrs
running build_ext
building 'libmgrs' extension
creating build/temp.linux-armv7l-3.4
creating build/temp.linux-armv7l-3.4/libmgrs
arm-linux-gnueabihf-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I./libmgrs -I/usr/include/python3.4m -c libmgrs/mgrs.c -o build/temp.linux-armv7l-3.4/libmgrs/mgrs.o
In file included from libmgrs/mgrs.c:86:0:
libmgrs/mgrs.h:255:3: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
   void initlibmgrs();
   ^
libmgrs/mgrs.h:256:3: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
   void PyInit_libmgrs();
   ^
libmgrs/mgrs.c: In function ‘Make_MGRS_String’:
libmgrs/mgrs.c:412:25: warning: field width specifier ‘*’ expects argument of type ‘int’, but argument 3 has type ‘long int’ [-Wformat=]
   i += sprintf (MGRS+i, "%*.*ld", Precision, Precision, east);
                         ^
libmgrs/mgrs.c:412:25: warning: field precision specifier ‘.*’ expects argument of type ‘int’, but argument 4 has type ‘long int’ [-Wformat=]
libmgrs/mgrs.c:417:25: warning: field width specifier ‘*’ expects argument of type ‘int’, but argument 3 has type ‘long int’ [-Wformat=]
   i += sprintf (MGRS+i, "%*.*ld", Precision, Precision, north);
                         ^
libmgrs/mgrs.c:417:25: warning: field precision specifier ‘.*’ expects argument of type ‘int’, but argument 4 has type ‘long int’ [-Wformat=]
libmgrs/mgrs.c: In function ‘Convert_MGRS_To_Geodetic’:
libmgrs/mgrs.c:879:6: warning: suggest explicit braces to avoid ambiguous ‘else’ [-Wparentheses]
   if (!error_code)
      ^
libmgrs/mgrs.c: At top level:
libmgrs/mgrs.c:1344:6: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
 void initlibmgrs() {}
      ^
libmgrs/mgrs.c:1345:6: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
 void PyInit_libmgrs() {}
      ^
arm-linux-gnueabihf-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I./libmgrs -I/usr/include/python3.4m -c libmgrs/utm.c -o build/temp.linux-armv7l-3.4/libmgrs/utm.o
arm-linux-gnueabihf-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I./libmgrs -I/usr/include/python3.4m -c libmgrs/ups.c -o build/temp.linux-armv7l-3.4/libmgrs/ups.o
arm-linux-gnueabihf-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I./libmgrs -I/usr/include/python3.4m -c libmgrs/tranmerc.c -o build/temp.linux-armv7l-3.4/libmgrs/tranmerc.o
libmgrs/tranmerc.c: In function ‘Convert_Transverse_Mercator_To_Geodetic’:
libmgrs/tranmerc.c:487:10: warning: variable ‘s’ set but not used [-Wunused-but-set-variable]
   double s;       /* Sine of latitude                        */
          ^
arm-linux-gnueabihf-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I./libmgrs -I/usr/include/python3.4m -c libmgrs/polarst.c -o build/temp.linux-armv7l-3.4/libmgrs/polarst.o
arm-linux-gnueabihf-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 build/temp.linux-armv7l-3.4/libmgrs/mgrs.o build/temp.linux-armv7l-3.4/libmgrs/utm.o build/temp.linux-armv7l-3.4/libmgrs/ups.o build/temp.linux-armv7l-3.4/libmgrs/tranmerc.o build/temp.linux-armv7l-3.4/libmgrs/polarst.o -o build/lib.linux-armv7l-3.4/libmgrs.cpython-34m.so
$ python3 setup.py test
running test
running egg_info
creating mgrs.egg-info
writing top-level names to mgrs.egg-info/top_level.txt
writing requirements to mgrs.egg-info/requires.txt
writing mgrs.egg-info/PKG-INFO
writing dependency_links to mgrs.egg-info/dependency_links.txt
writing manifest file 'mgrs.egg-info/SOURCES.txt'
reading manifest file 'mgrs.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'mgrs.egg-info/SOURCES.txt'
running build_ext
copying build/lib.linux-armv7l-3.4/libmgrs.cpython-34m.so -> 
/home/linaro/workspace/mgrs/tests/test.txt
Doctest: test.txt ... FAIL

======================================================================
FAIL: /home/linaro/workspace/mgrs/tests/test.txt
Doctest: test.txt
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.4/doctest.py", line 2187, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for test.txt
  File "/home/linaro/workspace/mgrs/tests/test.txt", line 0

----------------------------------------------------------------------
File "/home/linaro/workspace/mgrs/tests/test.txt", line 3, in test.txt
Failed example:
    import mgrs
Exception raised:
    Traceback (most recent call last):
      File "/usr/lib/python3.4/doctest.py", line 1318, in __run
        compileflags, 1), test.globs)
      File "<doctest test.txt[0]>", line 1, in <module>
        import mgrs
      File "/home/linaro/workspace/mgrs/mgrs/__init__.py", line 1, in <module>
        from . core import rt
      File "/home/linaro/workspace/mgrs/mgrs/core.py", line 35, in <module>
        rt = ctypes.CDLL(os.path.join(local_library_path, lib_name))
      File "/usr/lib/python3.4/ctypes/__init__.py", line 351, in __init__
        self._handle = _dlopen(self._name, mode)
    OSError: /home/linaro/workspace/mgrs/libmgrs.so: cannot open shared object file: No such file or directory

----------------------------------------------------------------------
Ran 1 test in 0.307s

FAILED (failures=1)
hobu commented 8 years ago

Ok, the SOABI is from Python's environment, not Ubuntu's. I was confused. I see https://www.python.org/dev/peps/pep-3149/ says this is the way to do it now.

Thanks for continuing to press.

proximous commented 8 years ago

Thanks for taking the time to review and for making the package available!!

Is this something you will push to pypi? I love the convenience of just doing a pip to install.

hobu commented 8 years ago

Is this something you will push to pypi?

I will clean up the warnings first and then I'll push a new release.

hobu commented 8 years ago

done. https://pypi.python.org/pypi/mgrs

proximous commented 8 years ago

Fantastic! Thanks!