Error Accessing Geosupport File: Grid1

[benjaminsingleton]

Thanks for building this library - it will be extremely valuable to me once I can get it working!

When I try to run the first example in the docs, I get the following error. I am running Ubuntu 16.04.

root@0b3992e9466b:/# python
Python 3.6.6 |Anaconda custom (64-bit)| (default, Jun 28 2018, 17:14:51)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from geosupport import Geosupport
>>> g = Geosupport()
>>> result = g.address(house_number=125, street_name='Worth St', borough_code='Mn')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/anaconda/lib/python3.6/site-packages/geosupport/geosupport.py", line 115, in call
    result
geosupport.error.GeosupportError: ERROR ACCESSING GEOSUPPORT FILE: GRID1. NOTIFY SYSTEM SUPPORT ERROR ACCESSING GEOSUPPORT FILE: PAD. NOTIFY SYSTEM SUPPORT

The relevant part of the Dockerfile where I install Geosupport is as follows:

# Lots of stuff before this point ...
RUN apt-get update && \
    apt-get install -y unzip && \
    wget https://www1.nyc.gov/assets/planning/download/zip/data-maps/open-data/gdelx_18b.zip && \
    unzip gdelx_18b.zip -d /var/geosupport/ && \
    rm gdelx_18b.zip
ENV GEOFILES=/var/geosupport/version-18b_18.2/fls
ENV LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/var/geosupport/version-18b_18.2/lib/
RUN pip install python-geosupport

I took a look at your Dockerfile and so I ran:

git clone https://github.com/ishiland/python-geosupport.git
cd ./python-geosupport
pip install -e .[dev]
invoke test all

The output of the tests was a segmentation fault:

root@0b3992e9466b:/# git clone https://github.com/ishiland/python-geosupport.git
Cloning into 'python-geosupport'...
remote: Enumerating objects: 602, done.
remote: Total 602 (delta 0), reused 0 (delta 0), pack-reused 602
Receiving objects: 100% (602/602), 510.80 KiB | 0 bytes/s, done.
Resolving deltas: 100% (311/311), done.
Checking connectivity... done.
root@0b3992e9466b:/# cd ./python-geosupport
root@0b3992e9466b:/python-geosupport# pip install -e .[dev]
Obtaining file:///python-geosupport
Requirement already satisfied: coverage in /usr/local/anaconda/lib/python3.6/site-packages (from python-geosupport==1.0.3) (4.5.2)
Collecting invoke>=1.1.1 (from python-geosupport==1.0.3)
  Downloading https://files.pythonhosted.org/packages/be/9f/8508712c9cad73ac0c8eeb2c3e51c9ef65136653dda2b512bde64109f023/invoke-1.2.0-py3-none-any.whl (206kB)
    100% |████████████████████████████████| 215kB 8.8MB/s
Requirement already satisfied: nose in /usr/local/anaconda/lib/python3.6/site-packages (from python-geosupport==1.0.3) (1.3.7)
Installing collected packages: invoke, python-geosupport
  Found existing installation: python-geosupport 1.0.3
    Uninstalling python-geosupport-1.0.3:
      Successfully uninstalled python-geosupport-1.0.3
  Running setup.py develop for python-geosupport
Successfully installed invoke-1.2.0 python-geosupport
root@0b3992e9466b:/python-geosupport# invoke test all
EEEEEE.EEEFEFEEEEEEEE......EEEE.Segmentation fault

Pytest gives the following output:

root@0b3992e9466b:/python-geosupport# pytest
======================================================== test session starts ========================================================
platform linux -- Python 3.6.6, pytest-3.8.0, py-1.6.0, pluggy-0.7.1
rootdir: /python-geosupport, inifile:
plugins: remotedata-0.3.0, openfiles-0.3.0, doctestplus-0.1.3, cov-2.5.1, arraydiff-0.2, celery-4.2.1
collected 51 items

tests/functional/test_call.py FFFFFF.FFFFFFFFFFFFFF......FFFF.                                                                [ 62%]
tests/functional/test_call_alternate.py Segmentation fault
root@0b3992e9466b:/python-geosupport#

Any sense what might be the cause of the problem?

[ishiland]

You are running via docker container or on a stand alone Ubuntu machine? I would check the system resources that are available leading up to that error. Is it possible the environment does not have enough memory?

[benjaminsingleton]

I was able to get python-geosupport working in a different (more simple) docker container, so I need to continue to troubleshoot the seg fault. As you suggested, it could be possible that the environment does not have enough memory. I'll report back when I have confidence in the root cause. Thanks for the feedback.

[bbhoss]

Could you share the simple Dockerfile you got to work? I am getting the initial error, trying the exact same thing.

[benjaminsingleton]

@bbhoss This works for me. I had to make a small change to the source code, which you can see in my fork.

FROM python:3.7-alpine

# Change version as necessary
ARG GEOSUPPORT_VERSION="18c"

RUN apk update && \
    apk add bash curl git libgcc unzip && \
    curl https://www1.nyc.gov/assets/planning/download/zip/data-maps/open-data/gdelx_$GEOSUPPORT_VERSION.zip -O -J -L && \
    unzip gdelx_$GEOSUPPORT_VERSION.zip -d /geosupport && \
    export UNZIPPED_FOLDER_NM="$(ls /geosupport/)" && \
    mv /geosupport/$UNZIPPED_FOLDER_NM /geosupport/app && \
    rm gdelx_$GEOSUPPORT_VERSION.zip

ENV GEOFILES="/geosupport/app/fls/" \
    LD_LIBRARY_PATH="/geosupport/app/lib"

RUN pip install git+git://github.com/benjaminsingleton/python-geosupport.git

RUN apk del curl unzip

CMD ["/bin/sh"]

[docmarionum1]

@benjaminsingleton You're running it on python:3.7-alpine? Or are you running on OSX?

I saw the change in your fork and it's checking if the version is in ['linux', 'darwin']. alpine should have worked with sys.platform.startswith('linux') without the change you made. 'darwin' is suppose to be for OSX.

[benjaminsingleton]

@docmarionum1 You're right. You don't need to use my fork when using alpine linux. I forgot that I was trying to get the package to work on my Mac at one point.

This works:

FROM python:3.7-alpine

# Change version as necessary
ARG GEOSUPPORT_VERSION="18c"

RUN apk update && \
    apk add bash curl git libgcc unzip && \
    curl https://www1.nyc.gov/assets/planning/download/zip/data-maps/open-data/gdelx_$GEOSUPPORT_VERSION.zip -O -J -L && \
    unzip gdelx_$GEOSUPPORT_VERSION.zip -d /geosupport && \
    export UNZIPPED_FOLDER_NM="$(ls /geosupport/)" && \
    mv /geosupport/$UNZIPPED_FOLDER_NM /geosupport/app && \
    rm gdelx_$GEOSUPPORT_VERSION.zip

ENV GEOFILES="/geosupport/app/fls/" \
    LD_LIBRARY_PATH="/geosupport/app/lib"

RUN pip install python-geosupport

RUN apk del curl unzip

CMD ["/bin/sh"]

[benjaminsingleton]

@ishiland I was able to get the package to work on a fresh build of my Docker image. I'm honestly not sure what was the root cause of the earlier error, but I'm going to close the issue since it seems to have resolved itself.

[docmarionum1]

@benjaminsingleton Did you get it working on your Mac?