Install through pip fails due to VERSION and REAMDE.rst file reads

[clarkenciel]

Trying to install keen through pip fails. setup.py is unable to find VERSION or README.rst. This seems to be related to how and where pip runs a package's setup.py script.

The line here is the culprit: the VERSION file cannot be found in whatever directory pip is calling the script from.

This line also seems to fail for a similar reason.

Here is the pip error for the VERSION failure:

Traceback (most recent call last):
  File "/tmp/pip-install-test/keen/setup.py", line 29, in <module>
    version_file = open(os.path.join('.', 'VERSION'))
FileNotFoundError: [Errno 2] No such file or directory: './VERSION'

Here is the pip error for the README failure (which occurs after fixing the VERSION failure):

Traceback (most recent call last):
  File "/tmp/pip-install-test/keen/setup.py", line 36, in <module>
    long_description=codecs.open(os.path.join('README.rst'), 'r', encoding='UTF-8').read(),
  File "/usr/lib/python3.6/codecs.py", line 895, in open
    file = builtins.open(filename, mode, buffering)
FileNotFoundError: [Errno 2] No such file or directory: 'README.rst'

[clarkenciel]

fixed as i was writing this!

[dkador]

thanks for the quick report! should be fixed with 0.3.30 now. :) sorry for the hassle!

[dperconti]

Still seeing this error with 0.3.30:

IOError: [Errno 2] No such file or directory: './VERSION'

/lib/python2.7/site-packages/keen/__init__.pyc in add_event(event_collection, body, timestamp)
     47     """
     48     _initialize_client_from_environment()
---> 49     _client.add_event(event_collection, body, timestamp=timestamp)
     50
     51

/lib/python2.7/site-packages/keen/client.pyc in add_event(self, event_collection, event_body, timestamp)
    131         event = Event(self.project_id, event_collection, event_body,
    132                       timestamp=timestamp)
--> 133         self.persistence_strategy.persist(event)
    134
    135     def add_events(self, events):

/lib/python2.7/site-packages/keen/persistence_strategies.pyc in persist(self, event)
     35         :param event: an Event to persist
     36         """
---> 37         self.api.post_event(event)
     38
     39     def batch_persist(self, events):

/lib/python2.7/site-packages/keen/api.pyc in post_event(self, event)
    107                                                        self.project_id,
    108                                                        event.event_collection)
--> 109         headers = utilities.headers(self.write_key)
    110         payload = event.to_json()
    111         response = self.fulfill(HTTPMethods.POST, url, data=payload, headers=headers, timeout=self.post_timeout)

/lib/python2.7/site-packages/keen/utilities.pyc in headers(api_key)
     25       "Content-Type": "application/json",
     26       "Authorization": api_key,
---> 27       "Keen-Sdk": "python-{0}".format(version())
     28     }

/lib/python2.7/site-packages/keen/utilities.pyc in version()
     10     global VERSION
     11     if VERSION is None:
---> 12         version_file = open(os.path.join('.', 'VERSION'))
     13         VERSION = version_file.read().strip()
     14

IOError: [Errno 2] No such file or directory: './VERSION'

pip freeze results in keen==0.3.30

[dkador]

sorry to hear that! is this during pip install? i just did a pip install keen in a fresh virtualenv and it worked for me, so maybe something else is going on. if you can give me some more details on your environment (python version especially), we can track this down.

[dperconti]

Python 2.7.11 pip 9.0.1

[dkador]

hm. i'm having difficult reproducing. to demonstrate what i see:

(keenclient-python-install-test) ➜  /tmp python --version
Python 2.7.11
(keenclient-python-install-test) ➜  /tmp pip --version
pip 9.0.1 from /Users/dkador/virtualenvs/keenclient-python-install-test/lib/python2.7/site-packages (python 2.7)
(keenclient-python-install-test) ➜  /tmp pip freeze
(keenclient-python-install-test) ➜  /tmp pip install keen
Collecting keen
Collecting pycryptodome>=3.4 (from keen)
Collecting requests<2.11.0,>=2.5 (from keen)
  Using cached requests-2.10.0-py2.py3-none-any.whl
Installing collected packages: pycryptodome, requests, keen
Successfully installed keen-0.3.30 pycryptodome-3.4.5 requests-2.10.0
(keenclient-python-install-test) ➜  /tmp

what's the output of your pip freeze?

what OS are you on?

[masojus]

Yeah I too am seeing this after both upgrading and uninstalling/fresh-installing keen to 0.3.30--no VERSION file. @dkador should the setup.py be specifying include_package_data=True to include the files listed in MANIFEST.in or explicitly listing the VERSION file in the package_data so as to not install the readme/license/requirements? My understanding is that MANIFEST.in works for sdist, but package-data is necessary for a bdist. https://packaging.python.org/distributing/#package-data

[dkador]

thanks for the report, @masojus! there's clearly something wrong here. i'd just love to be able to reproduce it myself so i can verify there's a fix.

i think you're on the right path. the VERSION file is definitely the culprit, somehow.

what OS are you on?

[masojus]

I'm on Windows 10, pip 8.1.2, running this test in Python 2.7.10. But the site-packages/keen dir has no VERSION (or any of the files listed in MANIFEST.in) which supports my hypothesis. Also, the distutils docs mentions that only the sdist command processes that manifest. See more here. However, the keen-0.3.30-py2.7.egg-info/SOURCES.txt in my site-packages lists VERSION, but installed-files.txt does not directly mention it.

[dkador]

ok. could you try pulling master here and running python setup.py sdist? i'm curious if this is a windows thing. that's my best guess at this point since it works on my mac. we'll fix it no matter what, but hopefully that could isolate the issue for us.

[dperconti]

MacOS Sierra, 10.12.3

pip freeze:

appdirs==1.4.2
click==6.7
Flask==0.12
gunicorn==19.6.0
honcho==0.7.1
itsdangerous==0.24
Jinja2==2.9.5
keen==0.3.30
MarkupSafe==0.23
packaging==16.8
pycryptodome==3.4.5
pyparsing==2.1.10
requests==2.10.0
six==1.10.0
slacker==0.9.42
Werkzeug==0.11.15
whitenoise==2.0.6

[dperconti]

Here's my reproduction steps:

> mkdir tmp

> cd tmp

> virtualenv tmp
New python executable in /tmp/bin/python2.7
Also creating executable in /tmp/bin/python
Installing setuptools, pip, wheel...done.

> cd tmp

> source bin/activate

> pip install keen
Collecting keen
Collecting requests<2.11.0,>=2.5 (from keen)
  Using cached requests-2.10.0-py2.py3-none-any.whl
Collecting pycryptodome>=3.4 (from keen)
Installing collected packages: requests, pycryptodome, keen
Successfully installed keen-0.3.30 pycryptodome-3.4.5 requests-2.10.0

> python
Python 2.7.11 (default, Jan 22 2016, 08:29:18)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import keen
>>>
>>> keen.project_id = "***"
>>> keen.write_key = "******"
>>> keen.add_event("text", {})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/tmp/lib/python2.7/site-packages/keen/__init__.py", line 49, in add_event
    _client.add_event(event_collection, body, timestamp=timestamp)
  File "/tmp/lib/python2.7/site-packages/keen/client.py", line 133, in add_event
    self.persistence_strategy.persist(event)
  File "/tmp/lib/python2.7/site-packages/keen/persistence_strategies.py", line 37, in persist
    self.api.post_event(event)
  File "/tmp/lib/python2.7/site-packages/keen/api.py", line 109, in post_event
    headers = utilities.headers(self.write_key)
  File "/tmp/lib/python2.7/site-packages/keen/utilities.py", line 27, in headers
    "Keen-Sdk": "python-{0}".format(version())
  File "/tmp/lib/python2.7/site-packages/keen/utilities.py", line 12, in version
    version_file = open(os.path.join('.', 'VERSION'))
IOError: [Errno 2] No such file or directory: './VERSION'
>>>

[dkador]

thank you!

[dkador]

@dperconti - i /believe/ that 0.3.31 should put this to bed. would you mind giving it a shot?

[ajabogdanoff]

👍