Closed GoogleCodeExporter closed 8 years ago
GoogleCodeExporter
commented
8 years ago One solution to it will be to move the whole setup/install process to paster
created
script. Generate skeleton using "paster create", then move over the commands, to
paster style. Benefit will be that the installation will go much smoother for
any
platform that supports python egg installation. Original comment by szybal...@gmail.com on 23 Jun 2008 at 6:06
GoogleCodeExporter
commented
8 years ago Original comment by szybal...@gmail.com on 25 Jun 2008 at 1:31
GoogleCodeExporter
commented
8 years ago What do you use sgmlop for?
sgmlop.c:377: error: ‘FastParserObject’ has no member named
‘resolve_entityref’
I assume I am missing some kind of package. I'll have to play around with it.
Do we use sgmlop system wide? Original comment by szybal...@gmail.com on 1 Jul 2008 at 12:55
GoogleCodeExporter
commented
8 years ago I am using sgmlop as an optional fast HTML parser. This is used to parse those
pages
which have bad HTML since the pure Python parser often fails with bad HTML, thus
rendering such pages uncrawlable. If we find a page fails with regular parser,
we
switch to the sgmlop based parser and reparse it. It succeeds 90% of the time
and
gives us links, since sgmlop is tolerant of bad HTML/XHTML.
So this is required in HarvestMan. It is used in the module crawler.py and the
sgmlop
based HTML parser is part of module pageparser.py .Original comment by abpil...@gmail.com on 2 Jul 2008 at 3:13
GoogleCodeExporter
commented
8 years ago Looks like you are getting this error because the compiler is not able to find
"Python.h" and other include files. Did you check this out that the python-devel
packages are present ?
--AnandOriginal comment by abpil...@gmail.com on 2 Jul 2008 at 3:14
GoogleCodeExporter
commented
8 years ago "gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall
-Wstrict-prototypes -fPIC -I/usr/include/python2.5 -c sgmlop.c -o
build/temp.linux-x86_64-2.5/sgmlop.o
sgmlop.c:79:20: error: Python.h: No such file or directory"
Clearly, Python.h is missing so your python-dev packages are not present.
That is the problem.Original comment by abpil...@gmail.com on 3 Jul 2008 at 4:08
GoogleCodeExporter
commented
8 years ago That means the python-dev is required on linux to install harvestman2
Even after installing python-dev I get this error:
Running sgmlop-1.1.1-20040207/setup.py -q bdist_egg --dist-dir
/tmp/easy_install-zBxzHj/sgmlop-1.1.1-20040207/egg-dist-tmp-UaWyK3
sgmlop.c: In function ‘attrparse’:
sgmlop.c:1234: warning: pointer targets in passing argument 1 of
‘PyString_FromStringAndSize’ differ in signedness
sgmlop.c:1285: warning: pointer targets in passing argument 1 of
‘PyString_FromStringAndSize’ differ in signedness
sgmlop.c: In function ‘attrexpand’:
sgmlop.c:1356: warning: pointer targets in passing argument 1 of
‘PyString_FromStringAndSize’ differ in signedness
sgmlop.c:1398: warning: pointer targets in passing argument 1 of
‘PyString_FromStringAndSize’ differ in signedness
sgmlop.c: At top level:
sgmlop.c:76: warning: ‘copyright’ defined but not used
sgmlop.c:200: warning: ‘wf_checker’ defined but not used
zip_safe flag not set; analyzing archive contents...
Adding sgmlop 1.1.1-20040207 to easy-install.pth file
2nd Item:
Writing
/home/lucas/tmp/baseline4/lib/python2.5/site-packages/HarvestMan-2.0alpha.egg-in
fo
Creating basic configuration in /etc/harvestman...
Traceback (most recent call last):
File "setup.py", line 463, in <module>
main()
File "setup.py", line 417, in main
open(etcfile, 'w').write(conf_data)
IOError: [Errno 13] Permission denied: '/etc/harvestman/config.xml'
We need to find a different place to put this config.xml.Is this a sample
config.xml
or harvestman needs it to run properly?
For system install that will do, but for virtualenv or install without root
permission it won't.
Where should we put it?
~/.harvestman/
LucasOriginal comment by szybal...@gmail.com on 11 Jul 2008 at 11:00
GoogleCodeExporter
commented
8 years ago I dont know much about virtualenv, but is there something like a global /etc/
repository for virtualenv ? If not ,we can skip the step of creating a global
configuration for virtualenv setups and generate only the user-level
configuration at
~/.harvestman .
--AnandOriginal comment by abpil...@gmail.com on 13 Jul 2008 at 8:34
GoogleCodeExporter
commented
8 years ago I have switch the structure to a paster/setuptool programs. I am able to create
eggs etc.
I was successfully able to install harvestman in bin of virtualenv. Now the
steps to
fix are:
import functions:
from my initial test it seems you have to import stuff like this inide of the
harvestman files:
from harvestman import libs
from harvestman import xyz
Which file is the test suit that you use? I wanted to run it on the new install.
LucasOriginal comment by szybal...@gmail.com on 13 Aug 2008 at 9:24
GoogleCodeExporter
commented
8 years ago Please check your email. I have merged the changes to trunk and it works for me.
Please verify the same for you. Then remove the paste branch and close this
bug. Thanks!Original comment by abpil...@gmail.com on 6 Oct 2008 at 11:18
GoogleCodeExporter
commented
8 years ago I'll leave this tickets until the setup.py works perfectly.
Right now I get an error if I try to build just an egg.
python setup.py bdist_egg
I have to look into how we can bypass putting the config.xml until somebody
actually
installs.
As soon as we can build the egg, upload the package to pypi I think this will
use all
the features of setuptools, which is the second part of the setup issue.
LucasOriginal comment by szybal...@gmail.com on 7 Oct 2008 at 3:14
GoogleCodeExporter
commented
8 years ago Maybe, you should close this bug now that virtualenv is working fine and open a
new
bug for setup.py. I think this has been running for too long - doesn't give me a
happy feeling of closing completed work :)
Let me know if this is ok. Original comment by abpil...@gmail.com on 12 Oct 2008 at 10:35
GoogleCodeExporter
commented
8 years ago Virtualenv Setup is fixed. It works on i386 and amd64. Original comment by szybal...@gmail.com on 12 Oct 2008 at 3:36
Original issue reported on code.google.com by
szybal...@gmail.comon 16 Jun 2008 at 11:36