write functions that work work when numpy is not installed

santoshphilip commented 10 years ago

There are geometry functions in ./eppy/geometry that use numpy. It can be difficult to install numpy at times (for instance if you want to run epp in rhino http://www.rhino3d.com)

rewrite the those functions to run without numpy.
test if numpy is installed (use <try .. except> block to check)
if it is not installed, use the new functions

santoshphilip commented 9 years ago

have you pushed all changes back ?

with numpy NOT installed, I get the following error with pytest:

==================================== ERRORS ====================================
________ ERROR collecting eppy/tests/geometry_tests/test_volume_zone.py ________
eppy/tests/geometry_tests/test_volume_zone.py:22: in <module>
    import eppy.geometry.volume_zone as volume_zone
eppy/geometry/volume_zone.py:25: in <module>
    import numpy as np
E   ImportError: No module named numpy

eayoungs commented 9 years ago

Yes, I think that is something to do with the error handling in the try/except statement. Without that statement, they all pass; without numpy. With numpy, they still all pass as well, but the exception seems to be picked up by pytest. It's not ready to be merged back to the master branch but the unit tests, by themselves, are passing. It will likely ready for a merge by early next week. On Jul 9, 2015 5:19 AM, "santoshphilip" notifications@github.com wrote:

have you pushed all changes back ?

with numpy NOT installed, I get the following error with pytest:

==================================== ERRORS ==================================== ____ ERROR collecting eppy/tests/geometry_tests/test_volume_zone.py ____ eppy/tests/geometry_tests/test_volume_zone.py:22: in import eppy.geometry.volume_zone as volume_zone eppy/geometry/volume_zone.py:25: in import numpy as np E ImportError: No module named numpy

— Reply to this email directly or view it on GitHub https://github.com/santoshphilip/eppy/issues/30#issuecomment-119938147.

eayoungs commented 9 years ago

OK, now it looks like all tests are passing with and without numpy installed. Turns out there were some simple math functions (add, subtract, divide) that involved the ndarray type from numpy. I'm not sure how I overlooked that during the previous testing but everything seems to run and pass now, so go ahead and take another look. I'm still planning on cleaning things up a bit this weekend before merging but it should be functional.

Cheers

Eric

santoshphilip commented 9 years ago

let me do the final merge into develop Just let me know when.

Santosh

eayoungs commented 9 years ago

OK, Santosh; I think the branch is ready for a merge. Feel free to let me know if there are any changes you'd like me to make.

Cheers

Eric

santoshphilip commented 9 years ago

I'll do this merge after I do the release with the MIT license. (trying to do one thing at t time :-)

eayoungs commented 9 years ago

Sure, no worries. It's nice to see so much activity.

santoshphilip commented 9 years ago

all tests are passing on my home computer

with numpy and
without numpy.

On my work computer one of the tests is for tinynumpy is failing. This is very curious.

I'll investigate further and report back

eayoungs commented 9 years ago

On my machine there are 2 tests that are skipped in tinynumpy. One is skipped both with and without numpy installed, the other is skipped when numpy is not installed.

santoshphilip commented 9 years ago

I did notice that. Something on my work machine is different that is triggering the failure. maybe an older version of pytest

I have succeeded in merging your branch into a copy of the develop branch. It had a couple of merge conflicts that had to be hand edited. The merged branch passes all tests

I'll now run pylint and clean up some stuff. (pylint sends to be very strict, and I tend to ignore some it's complaints)

Then I'll generate the python3 files and run tests in python3 too.

santoshphilip commented 9 years ago

i had not seen skip used in pytest before. Learnt something new here

santoshphilip commented 9 years ago

Eric,

Can you check if tinynumpy works under python3. The github page claims that it does. I want to make sure since eppy should run under python3.

tinynumpy is not on pypi. It would be nice to be able to do

pip install tinynumpy

and then not have to worry about it, especially if it will install either python2 or python3 as appropriate In any case check into this and if things are not working we would expect, we'll find a workaround. Maybe we can persuade the developers to upload to pypi

eayoungs commented 9 years ago

Sure I will test it for 3 compatibility and ask Wade if he can get the project set up on pypi. I was considering whether to try to import tinynumpy into eppy rather than (as I actually did) simply embed the modules inside eppy. I figured for right now it's faster to just to embed tinynumpy as it's a small project, early in it's lifecycle and so not changing very quickly but in the long term I'd like to move toward an externally imported module. I've also committed back the functions I created to tinynumpy. This could be a useful relationship between projects.

Eric ToDo:

[ ] Test tinynumpy under Python3
[ ] Determine a path for tinynumpy to be available through pip

eayoungs commented 9 years ago

As for the tests, something to keep in mind is that tinynumpy's tests were written with unittest. I did make a simple change to get it to work with py.test (I don't remember exactly what it was).

Glad I wasn't the only one who hadn't heard of skip; I looked it up too.

santoshphilip commented 9 years ago

if tinynumpy does not work under python3
- the eppy team can try to make that happen.
- or
- we can wait for Wade to do it.
- in the meantime, eppy release for python3 will have numpy as a dependency.

In essence, tinynumpy not working under python3 is not a deal breaker. It does not stop our release cycle

santoshphilip commented 9 years ago

Eric.

I started to merge branch 'i30_pythongeometry' into a copy of develop. A number of additional steps had to be taken:

fix some merge conflicts by hand. (most of them were done in the merge tool)
add MIT license to the new files
get files ready for python3 transition
run pylint to come close to PEP 0008 style
use 2to3 to transition to python3
unit test for python3 code

A number of tools have to be set up in your development environment to do this. My thinking on this is this:

I complete all the steps to do the merge. (into a copy of develop)
I commit at all important stages, with a clear commit message
If you look at the log, you can see the steps I have taken.
You can also do a diff between the commits to see what has changed.
Now you can do the merge on your own, by looking at my steps
this may help to shorten the learning curve, and to set up the tools in your environment.
some of this may go into the developer documentation to help others do this (Leora and myself will take responsibility for this)

Would you like me to do this ? So you have any other ideas ?

eayoungs commented 9 years ago

You know, it just occurs to me that I've been working from the master branch rather than pulling changes from the develop branch. I wonder if this has caused any issues?

Whatever setup is required to do the job, I'm happy to set it up on my system. As far as PEP 8 is concerned, I was checking in for quite a while but recently stopped just to concentrate on getting the tests to pass.

I think setting up this branch for Python 3 would be especially useful for me, if I'm going to work on the release.

santoshphilip commented 9 years ago

Right now develop and master are identical. In general

branch from master and work on an issue
merge into develop when you are done.
I will merge develop into master before a release

Regarding PEP 8. Be lazy about it. Code in any way you want. Let pylint do the hard work of getting you to PEP 8

eayoungs commented 9 years ago

Santosh, All sounds good. Do you think it would be useful for me to start documenting the contribution process while it's still fresh in my mind? I'm thinking I could just add some markdown to the repo.

santoshphilip commented 9 years ago

It may be so. Why don't you make some notes

eayoungs commented 9 years ago

Santosh, Tinynumpy does work with Python 3. I ran the tests with my additions and found a small change in my code needed for Python 3 compatibility. I've made the change in this issue branch (_i30pythongeometry) and pushed the branch back to the repo. Also, I pulled all changes (quite a few) from the master branch into _i30pythongeometry and made a few merges by hand. I hope this will make the merging into the development branch easier.

Hope this helps!

santoshphilip commented 9 years ago

Done …. Your manual merges made things easier. I have merged it into develop. I did the following things

Put MIT license in all new files
added some imports from __future__ (needed for python3 transition)
added some test code
ran pylint on all new files (except tinynumpy.py)
- at some point I would like you to run pylint on test_tinydnarray.py again
- it is scoring a little low now
ran 2to3 conversions

All tests are passing in:

python2 with and without numpy
python3 with and without numpy

I am closing this issue. Good job! Eric.

eayoungs commented 9 years ago

Great! Will do, Santosh. Thanks for all the help and encouragement.

eayoungs commented 9 years ago

Santosh, Can you try installing tinynumpy from pip again? I believe I've got it working now.

Cheers

Eric

santoshphilip commented 9 years ago

Yup. pip install tinynumpy is working fine. This will ease the 2to3 transiton

santoshphilip commented 9 years ago

Can you make changes in your branch so that

tinynumpy is in requirements.txt
remove tinynumpy from the eppy codebase

Let me know when all is well and I'll merge into develop branch

eayoungs commented 9 years ago

Sure; I've got those changes mostly completed on the eayoungs branch. Just want to verify some things. I will probably be able to complete it this week.

eayoungs commented 9 years ago

Hey, sorry for the delay. I got caught up in work last week. I ran into some issues with tinynumpy again. It's not importing into eppy, even though it appears to be installed in my virtualenv. I'll have to dig into PyPI again. Every time I google issues with PyPI, I run into blog posts that wonder why it's so difficult to get packages working with PyPI, so at least I'm not alone!

Eric

Santosh, OK! I have tinynumpy working on PyPI, it installs with pip and all tests pass. Please see the eayoungs branch and let me know if there are any changes you'd like me to make.

Cheers

Eric

santoshphilip commented 9 years ago

for some reason, I did not see the second half of your message. Will test and respond

santoshphilip commented 9 years ago

I am still not getting an install for tinynumpy

(eppy_nonumpy)Santoshs-Air:~ santoshphilip$ pip install tinynumpy
Downloading/unpacking tinynumpy
  Could not find a version that satisfies the requirement tinynumpy (from versions: 1.2.0.dev3)
Cleaning up...
No distributions matching the version for tinynumpy
Storing debug log for failure in /Users/santoshphilip/.pip/pip.log

shall I try to set up a temporary pipy account for tinynumpy (I'll call it tinynumpy_test or something). Just to see if I can get it to work.

eayoungs commented 9 years ago

Ugh! I thought I tested it on my Ubuntu machine, too. Let me check into it. This week is a little tight but I should be able to get to it in the next day or two. I apologize for all the back & forth but I think it's key that I get this right, especially since my next task is to get the Python3 package working.

Thanks for staying in touch about this issue!

Eric

eayoungs commented 9 years ago

Santosh, Here's what I get on my Ubuntu machine:

_(test_tnp)eayoungs@pinky:~$ pip install tinynumpy Collecting tinynumpy Downloading tinynumpy-1.2.0.dev3.tar.gz Building wheels for collected packages: tinynumpy Running setup.py bdistwheel for tinynumpy Stored in directory: /home/eayoungs/.cache/pip/wheels/9e/93/24/ade532b9bd77231fb222c4c1f6e62947014e568429bd76fcf3 Successfully built tinynumpy Installing collected packages: tinynumpy Successfully installed tinynumpy-1.2.0.dev3

All tests pass on Ubuntu with both setups, too. Can someone else test this? @jamiebull1 @vinc85

santoshphilip commented 9 years ago

Eric, I tracked down the problem.

From my error log:

https://pypi.python.org/simple/tinynumpy/), version 1.2.0.dev3 is a pre-release (use --pre to allow).

It installs if I do:

pip install --pre tinynumpy

For eppy to install it, it should work without the "--pre". If you are comfortable enough with the release, can you make a release that is not a pre-release ?

eayoungs commented 9 years ago

Interesting. I didn't use that flag, but I'll check with @wadetb (tinynumpy) and see if I can get this updated on PyPI. It may be late this week or early next.

santoshphilip commented 9 years ago

I am attaching my error log, if it helps. pip.txt

santoshphilip commented 9 years ago

Maybe this is happening because the word "dev" is in the version number.

the DOAP record: tinynumpy-1.2.0.dev3.xml says

<?xml version="1.0" encoding="UTF-8" ?>
<rdf:RDF xmlns="http://usefulinc.com/ns/doap#" xmlns:foaf="http://xmlns.com/foaf/0.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"><Project><name>tinynumpy</name>
<shortdesc>A lightweight, pure Python, numpy compliant ndarray class</shortdesc>
<homepage rdf:resource="https://github.com/wadetb/tinynumpy" />
<maintainer><foaf:Person><foaf:name>Wade Brainerd, Almar Klein</foaf:name>
<foaf:mbox_sha1sum>2d85bb5e3da728bde551ac136e647bd44cd904fa</foaf:mbox_sha1sum></foaf:Person></maintainer>
<release><Version><revision>1.2.0.dev3</revision></Version></release>
</Project></rdf:RDF>

try without the word "dev" in the version number. It might fix the issue

eayoungs commented 9 years ago

Sorry for the delay; I'll give it a try today.

eayoungs commented 9 years ago

Give it a try now; I've removed the "dev3" from the package name.

santoshphilip commented 9 years ago

Yes ! It works

santoshphilip commented 9 years ago

in branch i30_pythongeometry all tests pass

with tinynumpy installed and
without numpy installed

I have lost track of what to merge, since this is an old issue. My recollection is that this was already merged into branch develop. I'll try to spend some time update myself on the state of this issue

eayoungs commented 9 years ago

Great! I think I've got all the changes in the eayoungs branch. If the current state of that branch has been merged, then we're likely up to date.

Now I can move on to getting the Python 3 version of eppy available on PyPI

santoshphilip commented 9 years ago

Eric, I was looking thru branch eayoungs and some thoughts:

requirements.txt should have tinynumpy in it
tinnumpy should be removed from eppy (and any associated tests for tinynumpy)
the import statements for tinynumpy will need an update (or pytest will fail)
after you do this, all pytest should pass

I tested the merge and I am not seeing conflicts arising from doing the merge into develop

eayoungs commented 9 years ago

Santosh, I had made most of those changes previously but somehow stashed the changes when I undid the changes to the develop branch. Anyway, I reapplied the changes and added tinynumpy>=1.2.0 to requirements.txt. All tests still pass with and without numpy. All commits have been pushed to eayoungs branch should now be available on Github.

Cheers

Eric

santoshphilip commented 9 years ago

Thanks. I'll test and then merge into develop. will close this issue once that is done

santoshphilip commented 9 years ago

Some import statements need to be updated. I have tinynumpy installed:

(tinynumpy)Santoshs-MacBook-Air:eppy santoshphilip$ python 
Python 2.7.6 (default, Apr  9 2014, 11:48:52) 
[GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.38)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import tinynumpy
>>>

py.test gives the following errors

(tinynumpy)Santoshs-MacBook-Air:eppy santoshphilip$ py.test
============================= test session starts ==============================
platform darwin -- Python 2.7.6, pytest-2.8.2, py-1.4.30, pluggy-0.3.1
rootdir: /Users/santoshphilip/Documents/coolshadow/eppy, inifile: setup.cfg
collected 12 items / 12 errors 

eppy/tests/test_bunchhelpers.py ....
eppy/tests/test_iddgaps.py .
eppy/tests/test_readhtml.py ......
eppy/tests/EPlusInterfaceFunctions_tests/test_parse_idd.py .

==================================== ERRORS ====================================
___________________ ERROR collecting eppy/tests/test_IDF.py ____________________
eppy/tests/test_IDF.py:18: in <module>
    from eppy.modeleditor import IDF0
eppy/modeleditor.py:17: in <module>
    from eppy.idfreader import idfreader1
eppy/idfreader.py:20: in <module>
    import eppy.function_helpers as fh
eppy/function_helpers.py:17: in <module>
    from eppy.geometry import surface as g_surface
eppy/geometry/surface.py:27: in <module>
    from . import tinynumpy as np
E   ImportError: cannot import name tinynumpy
______________ ERROR collecting eppy/tests/test_bunch_subclass.py ______________
eppy/tests/test_bunch_subclass.py:682: in <module>
    import eppy.idfreader as idfreader
eppy/idfreader.py:20: in <module>
    import eppy.function_helpers as fh
eppy/function_helpers.py:17: in <module>
    from eppy.geometry import surface as g_surface
eppy/geometry/surface.py:27: in <module>
    from . import tinynumpy as np
E   ImportError: cannot import name tinynumpy
_________________ ERROR collecting eppy/tests/test_examples.py _________________
eppy/tests/test_examples.py:15: in <module>
    from eppy.idfreader import idfreader
eppy/idfreader.py:20: in <module>
    import eppy.function_helpers as fh
eppy/function_helpers.py:17: in <module>
    from eppy.geometry import surface as g_surface
eppy/geometry/surface.py:27: in <module>
    from . import tinynumpy as np
E   ImportError: cannot import name tinynumpy
_______________ ERROR collecting eppy/tests/test_hvacbuilder.py ________________
eppy/tests/test_hvacbuilder.py:15: in <module>
    import eppy.hvacbuilder as hvacbuilder
eppy/hvacbuilder.py:17: in <module>
    import eppy.modeleditor as modeleditor
eppy/modeleditor.py:17: in <module>
    from eppy.idfreader import idfreader1
eppy/idfreader.py:20: in <module>
    import eppy.function_helpers as fh
eppy/function_helpers.py:17: in <module>
    from eppy.geometry import surface as g_surface
eppy/geometry/surface.py:27: in <module>
    from . import tinynumpy as np
E   ImportError: cannot import name tinynumpy
________________ ERROR collecting eppy/tests/test_idfreader.py _________________
eppy/tests/test_idfreader.py:15: in <module>
    import eppy.idfreader as idfreader
eppy/idfreader.py:20: in <module>
    import eppy.function_helpers as fh
eppy/function_helpers.py:17: in <module>
    from eppy.geometry import surface as g_surface
eppy/geometry/surface.py:27: in <module>
    from . import tinynumpy as np
E   ImportError: cannot import name tinynumpy
_______________ ERROR collecting eppy/tests/test_modeleditor.py ________________
eppy/tests/test_modeleditor.py:18: in <module>
    from eppy.modeleditor import IDF
eppy/modeleditor.py:17: in <module>
    from eppy.idfreader import idfreader1
eppy/idfreader.py:20: in <module>
    import eppy.function_helpers as fh
eppy/function_helpers.py:17: in <module>
    from eppy.geometry import surface as g_surface
eppy/geometry/surface.py:27: in <module>
    from . import tinynumpy as np
E   ImportError: cannot import name tinynumpy
________________ ERROR collecting eppy/tests/test_simpleread.py ________________
eppy/tests/test_simpleread.py:14: in <module>
    import eppy.simpleread as simpleread
eppy/simpleread.py:14: in <module>
    import eppy.modeleditor as modeleditor
eppy/modeleditor.py:17: in <module>
    from eppy.idfreader import idfreader1
eppy/idfreader.py:20: in <module>
    import eppy.function_helpers as fh
eppy/function_helpers.py:17: in <module>
    from eppy.geometry import surface as g_surface
eppy/geometry/surface.py:27: in <module>
    from . import tinynumpy as np
E   ImportError: cannot import name tinynumpy
_________ ERROR collecting eppy/tests/geometry_tests/test_area_zone.py _________
eppy/tests/geometry_tests/test_area_zone.py:12: in <module>
    import eppy.geometry.area_zone as area_zone
eppy/geometry/area_zone.py:18: in <module>
    import eppy.geometry.surface as surface
eppy/geometry/surface.py:27: in <module>
    from . import tinynumpy as np
E   ImportError: cannot import name tinynumpy
__________ ERROR collecting eppy/tests/geometry_tests/test_surface.py __________
eppy/tests/geometry_tests/test_surface.py:12: in <module>
    import eppy.geometry.surface as surface
eppy/geometry/surface.py:27: in <module>
    from . import tinynumpy as np
E   ImportError: cannot import name tinynumpy
________ ERROR collecting eppy/tests/geometry_tests/test_volume_zone.py ________
eppy/tests/geometry_tests/test_volume_zone.py:17: in <module>
    import eppy.geometry.volume_zone as volume_zone
eppy/geometry/volume_zone.py:25: in <module>
    from eppy.geometry.tinynumpy import dot as dot
E   ImportError: No module named tinynumpy
_______ ERROR collecting eppy/tests/tinynumpy_tests/test_tinyndarray.py ________
eppy/tests/tinynumpy_tests/test_tinyndarray.py:13: in <module>
    import eppy.geometry.tinynumpy as tinynumpy
E   ImportError: No module named tinynumpy
________ ERROR collecting eppy/tests/tinynumpy_tests/test_tinynumpy.py _________
eppy/tests/tinynumpy_tests/test_tinynumpy.py:16: in <module>
    import eppy.geometry.tinynumpy as tnp
E   ImportError: No module named tinynumpy
===================== 12 passed, 12 error in 0.87 seconds ======================
(tinynumpy)Santoshs-MacBook-Air:eppy santoshphilip$

eayoungs commented 9 years ago

That's odd. First off, there shouldn't be any tinynumpy tests in the project anymore. I just checked quickly and there are commits on my machine that weren't pushed out. I've done that now and will check more closely this afternoon.

eayoungs commented 8 years ago

Santosh, I've applied the changes to import directly from the pip installed module, but I'm getting an error related to the linalg module in the tinynumpy project. It is a separate file to mimic the structure of numpy and maximize the reuse of code with either numpy or tinynumpy. Unfortunately I seem to have named the module tinylinalg instead of linalg, which is causing an error of it's own. I will have to change this in the tinynumpy project and revise the project on PyPI.

santoshphilip commented 8 years ago

give ma a heads up when you are done with that. I'll run the tests again.

I am traveling most of December and may be out of contact. We should get some momentum on the python3 stuff before that. (completing the tinynumpy stuff is a precursor to that) I'll put together some notes in the developer documentation about the present state of python3 version

eayoungs commented 8 years ago

I've been thinking about it and it should be just a matter of adjusting the import statement in eppy, hopefully without modifying tinynumpy Well, that will give me some motivation to wrap this up and get started on the Python 3 issues by the end of the month. I've been distracted by work, but the good news is I've been using Python & R at work so my programming skills should have at least held, rather than degrading. ;-)

Eric

eayoungs commented 8 years ago

OK. I just made the changes to a few import statements and all tests path in both virtual environments (with numpy only, and with tinynumpy only). Let's try this one more time!

Happy Thanksgiving!

santoshphilip / eppy

write functions that work work when numpy is not installed #30