search

ggaughan / pipe2py

A project to compile Yahoo! Pipes into Python (see it hosted on Google App Engine: http://pipes-engine.appspot.com)
http://wiki.github.com/ggaughan/pipe2py
GNU General Public License v2.0
317 stars 51 forks source link

ImportError on OS X install #14

Open j2l opened 12 years ago

j2l commented 12 years ago

Hi,

After pipe2py install (feedparser and simplejson are already installed), testing or compiling throws: python testbasics.py Traceback (most recent call last): File "testbasics.py", line 9, in from pipe2py import Context ImportError: No module named pipe2py

on Mac OS X 10.5

Any ideas?

ggaughan commented 12 years ago

Do you have your PYTHONPATH environment variable set to the pipe2py directory? In Linux this would be something like:

export PYTHONPATH=/home/mypath/pipe2py

But perhaps setenv for Max OS X?

j2l commented 12 years ago

Thanks, I checked on http://docs.python.org/using/mac.html path seems to be ok with: export PYTHONPATH=$PYTHONPATH:/Applications/XAMPP/htdocs/pipeline/pipe2py

but no luck so far, I also tried on a pure linux box (my shared web server) and I have the same problem so I guess I missed something on both installs.

ggaughan commented 12 years ago

Ah - yes. the pipe2py directory with the __init__.py is the package directory, so you'll want:

export PYTHONPATH=$PYTHONPATH:/Applications/XAMPP/htdocs/pipeline
j2l commented 12 years ago

Yeah, I did that too, no luck, it's stuck ... Any other idea?

On Sun, Nov 6, 2011 at 9:35 PM, Greg Gaughan < reply@reply.github.com>wrote:

Ah - yes. the pipe2py directory with the init.py is the package directory, so you'll want:

export PYTHONPATH=$PYTHONPATH:/Applications/XAMPP/htdocs/pipeline

Reply to this email directly or view it on GitHub: https://github.com/ggaughan/pipe2py/issues/14#issuecomment-2647519

ggaughan commented 12 years ago

What do you get when you

$ python
>>> import sys
>>> print sys.path
j2l commented 12 years ago

I get: ['', '/Library/Python/2.5/site-packages/feedparser-5.0.1-py2.5.egg', '/Library/Python/2.5/site-packages/simplejson-2.2.1-py2.5-macosx-10.5-i386.egg', '/Applications/XAMPP/xamppfiles/htdocs/pipeline/test/pipeline', '/Applications/XAMPP/htdocs/pipeline/pipe2py', '/Applications/XAMPP/htdocs/pipeline', '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python25.zip', '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5', '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-darwin', '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-mac', '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-mac/lib-scriptpackages', '/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python', '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-tk', '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-dynload', '/Library/Python/2.5/site-packages', '/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/PyObjC']

On Mon, Nov 7, 2011 at 6:08 PM, Greg Gaughan < reply@reply.github.com>wrote:

What do you get when you

$ python

import sys print sys.path

Reply to this email directly or view it on GitHub: https://github.com/ggaughan/pipe2py/issues/14#issuecomment-2656214

ggaughan commented 12 years ago

Hmm. Well presumably one of those paths should have the package files in it. Though I wonder what the difference is between the contents of /Applications/XAMPP/xamppfiles/htdocs/pipeline/test/pipeline and /Applications/XAMPP/htdocs/pipeline.

I don't use a Mac, so I'm not sure if you need any extra steps. Perhaps Python doesn't have permission to read the package directory or files?

What happens if you try to access the package __init__.py file from within Python:

$python
>>> open('/Applications/XAMPP/htdocs/pipeline/pipe2py/__init__.py').readlines()[:5]
j2l commented 12 years ago

open('/Applications/XAMPP/htdocs/pipeline/pipe2py/init.py').readlines()[:5] Traceback (most recent call last): File "", line 1, in IOError: [Errno 2] No such file or directory: '/Applications/XAMPP/htdocs/pipeline/pipe2py/init.py' open('/Applications/XAMPP/htdocs/pipeline/init.py').readlines()[:5] ['#pipe2py package\n', '#Author: Greg Gaughan\n', '\n', '#See LICENCE file for licence details\n', '\n']

On Mon, Nov 7, 2011 at 8:54 PM, Greg Gaughan < reply@reply.github.com>wrote:

Hmm. Well presumably one of those paths should have the package files in it. Though I wonder what the difference is between the contents of /Applications/XAMPP/xamppfiles/htdocs/pipeline/test/pipeline and /Applications/XAMPP/htdocs/pipeline.

I don't use a Mac, so I'm not sure if you need any extra steps. Perhaps Python doesn't have permission to read the package directory or files?

What happens if you try to access the package __init__.py file from within Python:

$python

open('/Applications/XAMPP/htdocs/pipeline/pipe2py/init.py').readlines()[:5]

Reply to this email directly or view it on GitHub: https://github.com/ggaughan/pipe2py/issues/14#issuecomment-2658661

ggaughan commented 12 years ago

So the files seem to be there but they need to be in a package directory named pipe2py for the import pipe2py to work. So it looks like the files in /Applications/XAMPP/htdocs/pipeline/ need to be moved into the /Applications/XAMPP/htdocs/pipeline/pipe2py/ sub-directory. Having /Applications/XAMPP/htdocs/pipeline in sys.path should then be enough to locate the pipe2py package.

j2l commented 12 years ago

Right! It works now. Thanks for your help

I think I misunderstood this phrase in README:

Setting up the environment

If using a Python version before 2.6 then simplejson is needed:

At first I put simplejson source code in pipeline/pipe2py and pipe2py in pipeline/

FIXED!

On Mon, Nov 7, 2011 at 11:06 PM, Greg Gaughan < reply@reply.github.com>wrote:

So the files seem to be there but they need to be in a package directory named pipe2py for the import pipe2py to work. So it looks like the files in /Applications/XAMPP/htdocs/pipeline/ need to be moved into the /Applications/XAMPP/htdocs/pipeline/pipe2py/ sub-directory. Having /Applications/XAMPP/htdocs/pipeline in sys.path should then be enough to locate the pipe2py package.

Reply to this email directly or view it on GitHub: https://github.com/ggaughan/pipe2py/issues/14#issuecomment-2660563

j2l commented 12 years ago

Ok, I can compile but I cannot execute it.

My compiled pipe throws: $ python pipe_0373e6dbc0f888a2f5af7d342e56eaba.py

Traceback (most recent call last): File "pipe_0373e6dbc0f888a2f5af7d342e56eaba.py", line 38, in for i in p: File "/Applications/XAMPP/htdocs/pipeline/pipe2py/modules/pipeoutput.py", line 18, in pipe_output for item in _INPUT: File "/Applications/XAMPP/htdocs/pipeline/pipe2py/modules/pipeloop.py", line 35, in pipe_loop for item in _INPUT: File "/Applications/XAMPP/htdocs/pipeline/pipe2py/modules/piperegex.py", line 30, in pipe_regex matchc = re.compile(match, re.DOTALL) #compile for speed and we need to pass flags File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/re.py", line 180, in compile return _compile(pattern, flags) File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/re.py", line 229, in _compile raise TypeError, "first argument must be string or compiled pattern" TypeError: first argument must be string or compiled pattern

It's not the real pipe cause I want to keep it private for now.

There's no argument in this pipe, everything is hardcoded within it, why does it ask for first argument? what (and how) should I put as argument?

Thanks for your help, again ;-)

On Tue, Nov 8, 2011 at 7:56 AM, j2l jeanllegros@yahoo.fr wrote:

Right! It works now. Thanks for your help

I think I misunderstood this phrase in README:

Setting up the environment

If using a Python version before 2.6 then simplejson is needed:

At first I put simplejson source code in pipeline/pipe2py and pipe2py in pipeline/

FIXED!

On Mon, Nov 7, 2011 at 11:06 PM, Greg Gaughan < reply@reply.github.com>wrote:

So the files seem to be there but they need to be in a package directory named pipe2py for the import pipe2py to work. So it looks like the files in /Applications/XAMPP/htdocs/pipeline/ need to be moved into the /Applications/XAMPP/htdocs/pipeline/pipe2py/ sub-directory. Having /Applications/XAMPP/htdocs/pipeline in sys.path should then be enough to locate the pipe2py package.

Reply to this email directly or view it on GitHub: https://github.com/ggaughan/pipe2py/issues/14#issuecomment-2660563

ggaughan commented 12 years ago

I'll make a note to improve the README text - I can see how it could be confusing.

Sounds like there could something strange with the pipe definition. I can't seem to access it via pipes or pipe2py for some reason - could you send me the definition please? Either a link that works, or using -j when compiling to get a .json representation.

Thanks, Greg

ggaughan commented 12 years ago

Sorry - just saw the bit about it being private. It sounds like a regex module parameter is missing - perhaps the loop failed to pass it something to work with. I'll need the .json file to investigate further though.

j2l commented 12 years ago

When trying to compile with Json: python compile.py pipe_0373e6dbc0f888a2f5af7d342e56eaba.py Traceback (most recent call last): File "compile.py", line 356, in pipe_def = json.loads(pjson) File "/Library/Python/2.5/site-packages/simplejson-2.2.1-py2.5-macosx-10.5-i386.egg/simplejson/init.py", line 413, in loads return _default_decoder.decode(s) File "/Library/Python/2.5/site-packages/simplejson-2.2.1-py2.5-macosx-10.5-i386.egg/simplejson/decoder.py", line 402, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/Library/Python/2.5/site-packages/simplejson-2.2.1-py2.5-macosx-10.5-i386.egg/simplejson/decoder.py", line 420, in raw_decode raise JSONDecodeError("No JSON object could be decoded", s, idx) simplejson.decoder.JSONDecodeError: No JSON object could be decoded: line 1 column 0 (char 0)

Not sure if it's related but this pipe is heavy on computing, it takes 20-40s to execute in yahoo pipes and sometimes fails. Regex is used with no errors but result can be cleaner so I get some warnings at execution. Result is a list with locations so I can get the map result too.

On Wed, Nov 9, 2011 at 1:37 PM, Greg Gaughan < reply@reply.github.com>wrote:

Sorry - just saw the bit about it being private. It sounds like a regex module parameter is missing - perhaps the loop failed to pass it something to work with. I'll need the .json file to investigate further though.

Reply to this email directly or view it on GitHub: https://github.com/ggaughan/pipe2py/issues/14#issuecomment-2680651

j2l commented 12 years ago

Sorry for my simple question: What option -j? What's the full command line to get for instance pipe_2de0e4517ed76082dcddf66f7b218057.json?

I tracked another problem with a fetch page loop in the same pipe ( I truncated the original pipe to track the bugs): $ python pipe_XX.py Traceback (most recent call last): File "pipe_XX.py", line 33, in for i in p: File "/Applications/XAMPP/htdocs/pipeline/pipe2py/modules/pipeoutput.py", line 18, in pipe_output for item in _INPUT: File "/Applications/XAMPP/htdocs/pipeline/pipe2py/modules/pipeloop.py", line 50, in pipe_loop for i in p: File "/Applications/XAMPP/htdocs/pipeline/pipe2py/modules/pipefetchpage.py", line 40, in pipe_fetchpage request = urllib2.Request(url) File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/urllib2.py", line 190, in init self.__original = unwrap(url) File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/urllib.py", line 1009, in unwrap url = url.strip() AttributeError: 'NoneType' object has no attribute 'strip'

I don't get what it means.

In Yahoo pipes, result of this fetch page loop section is in the form: 0 content html link http://... loop:fetchpage content html 1...

Thanks again,

On Fri, Nov 11, 2011 at 2:59 PM, j2l jeanllegros@yahoo.fr wrote:

When trying to compile with Json: python compile.py pipe_0373e6dbc0f888a2f5af7d342e56eaba.py Traceback (most recent call last): File "compile.py", line 356, in pipe_def = json.loads(pjson) File "/Library/Python/2.5/site-packages/simplejson-2.2.1-py2.5-macosx-10.5-i386.egg/simplejson/init.py", line 413, in loads return _default_decoder.decode(s) File "/Library/Python/2.5/site-packages/simplejson-2.2.1-py2.5-macosx-10.5-i386.egg/simplejson/decoder.py", line 402, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/Library/Python/2.5/site-packages/simplejson-2.2.1-py2.5-macosx-10.5-i386.egg/simplejson/decoder.py", line 420, in raw_decode raise JSONDecodeError("No JSON object could be decoded", s, idx) simplejson.decoder.JSONDecodeError: No JSON object could be decoded: line 1 column 0 (char 0)

Not sure if it's related but this pipe is heavy on computing, it takes 20-40s to execute in yahoo pipes and sometimes fails. Regex is used with no errors but result can be cleaner so I get some warnings at execution. Result is a list with locations so I can get the map result too.

On Wed, Nov 9, 2011 at 1:37 PM, Greg Gaughan < reply@reply.github.com>wrote:

Sorry - just saw the bit about it being private. It sounds like a regex module parameter is missing - perhaps the loop failed to pass it something to work with. I'll need the .json file to investigate further though.

Reply to this email directly or view it on GitHub: https://github.com/ggaughan/pipe2py/issues/14#issuecomment-2680651

ggaughan commented 12 years ago

Use the -s option to create a .json file created (sorry, not -j), e.g.

python compile.py -s -p 2de0e4517ed76082dcddf66f7b218057

(Your earlier python compile.py pipe_0373e6dbc0f888a2f5af7d342e56eaba.py will fail: compile.py expects a json file, not the pipe module itself.)

j2l commented 12 years ago

ok, thanks. I can produce .json file. I don't know how to help debug this. I cannot give access to this pipe or json file.

On Fri, Nov 11, 2011 at 5:27 PM, Greg Gaughan < reply@reply.github.com>wrote:

Use the -s option to create a .json file created (sorry, not -j), e.g.

python compile.py -s -p 2de0e4517ed76082dcddf66f7b218057

(Your earlier python compile.py pipe_0373e6dbc0f888a2f5af7d342e56eaba.py will fail: compile.py expects a json file, not the pipe module itself.)

Reply to this email directly or view it on GitHub: https://github.com/ggaughan/pipe2py/issues/14#issuecomment-2709918

ggaughan commented 12 years ago

Could you find out the value of rule just before line 30 in piperegex.py (the point where it fails).

My guess is that the regular expression is missing for some reason. What is the loop module looping over?

j2l commented 12 years ago

how to do that? I tried: print "%s" %(match) in piperegex.py but no luck

ggaughan commented 12 years ago 
print rule

would be better. Or trace into the exception and and inspect the value of rule.

I'm not sure how far we'll get with this indirect debugging ;)

reubano commented 9 years ago

I just made a bunch of updates, can you check and see if the issue is still there?