Closed ajthomas1949 closed 13 years ago
Out of curiosity, what happens if you downgrade your couchdb to version 0.7? (You can check to make sure your downgrade worked as expected by examining the output of
>>> import couchdb
>>> print couchdb.__version__
I think some other readers have had problems with couchdb version 0.8 as well. I may go ahead and try to update some of the code to account for whatever the difference seems to be. It would be helpful to know if this resolves your problem, though I'll admit that the error is somewhat nebulous of helpful meaning.
hi, many thanks for your prompt response. Being a newbie with couchDB, I'm not quite sure how to "downgrade". I installed it (version 0.8, on Ubuntu 11.04) with apt-get. Is there some option to apt-get to specify an earlier version? I see from the couchDB web site that it is already up to version 1.1.0 while in the Ubuntu package repository, the latest version is labelled 1.0.1-0ubuntu15. Should I try to install the more recent version from the Apache couchDB version 1.1.0 tarball?
Regards, Arthur
On 02/09/2011 17:48, ptwobrussell wrote:
Out of curiosity, what happens if you downgrade your couchdb to version 0.7? (You can check to make sure your downgrade worked as expected by examining the output of
>>> import couchdb >>> print couchdb.__version__
I think some other readers have had problems with couchdb version 0.8 as well. I may go ahead and try to update some of the code to account for whatever the difference seems to be. It would be helpful to know if this resolves your problem, though I'll admit that the error is somewhat nebulous of helpful meaning.
Actually - I was referring to downgrading the Python couchdb package you installed. Most people use easy_install or pip to do the installation. I'd recommend learning about those utilities if you haven't yet, because they'll be really useful to you. This specific section in easy_install's documentation shows you how to downgrade your couchdb package, for example: http://packages.python.org/distribute/easy_install.html#changing-the-active-version
You shouldn't need to change the version of CouchDB itself that you're using. I think the problem may just be with the Python package. At least, that's what I'd like for you to try and rule out at this point.
Regards - Matthew
http://www.linkedin.com/in/ptwobrussell
On Sep 5, 2011, at 10:48 AM, ajthomas1949 wrote:
hi, many thanks for your prompt response. Being a newbie with couchDB, I'm not quite sure how to "downgrade". I installed it (version 0.8, on Ubuntu 11.04) with apt-get. Is there some option to apt-get to specify an earlier version? I see from the couchDB web site that it is already up to version 1.1.0 while in the Ubuntu package repository, the latest version is labelled 1.0.1-0ubuntu15. Should I try to install the more recent version from the Apache couchDB version 1.1.0 tarball?
Regards, Arthur
On 02/09/2011 17:48, ptwobrussell wrote:
Out of curiosity, what happens if you downgrade your couchdb to version 0.7? (You can check to make sure your downgrade worked as expected by examining the output of
import couchdb print couchdb.version
I think some other readers have had problems with couchdb version 0.8 as well. I may go ahead and try to update some of the code to account for whatever the difference seems to be. It would be helpful to know if this resolves your problem, though I'll admit that the error is somewhat nebulous of helpful meaning.
Reply to this email directly or view it on GitHub: https://github.com/ptwobrussell/Mining-the-Social-Web/issues/9#issuecomment-2002702
ah, sorry I misunderstood. I've now reverted to python couchdb package v 0.7. Here's the result:
Finding docs dated from 2002-1-1 to 2002-2-1
Traceback (most recent call last):
File "mailboxes_map_by_date.py", line 46, in
At least it's different! I hope it gives you some clue as to what is going on... And thanks again for your prompt help.
arthur
On 05/09/2011 17:49, ptwobrussell wrote:
Actually - I was referring to downgrading the Python couchdb package you installed. Most people use easy_install or pip to do the installation. I'd recommend learning about those utilities if you haven't yet, because they'll be really useful to you. This specific section in easy_install's documentation shows you how to downgrade your couchdb package, for example: http://packages.python.org/distribute/easy_install.html#changing-the-active-version
You shouldn't need to change the version of CouchDB itself that you're using. I think the problem may just be with the Python package. At least, that's what I'd like for you to try and rule out at this point.
Regards - Matthew
http://www.linkedin.com/in/ptwobrussell
On Sep 5, 2011, at 10:48 AM, ajthomas1949 wrote:
hi, many thanks for your prompt response. Being a newbie with couchDB, I'm not quite sure how to "downgrade". I installed it (version 0.8, on Ubuntu 11.04) with apt-get. Is there some option to apt-get to specify an earlier version? I see from the couchDB web site that it is already up to version 1.1.0 while in the Ubuntu package repository, the latest version is labelled 1.0.1-0ubuntu15. Should I try to install the more recent version from the Apache couchDB version 1.1.0 tarball?
Regards, Arthur
On 02/09/2011 17:48, ptwobrussell wrote:
Out of curiosity, what happens if you downgrade your couchdb to version 0.7? (You can check to make sure your downgrade worked as expected by examining the output of
import couchdb print couchdb.version I think some other readers have had problems with couchdb version 0.8 as well. I may go ahead and try to update some of the code to account for whatever the difference seems to be. It would be helpful to know if this resolves your problem, though I'll admit that the error is somewhat nebulous of helpful meaning.
Reply to this email directly or view it on GitHub: https://github.com/ptwobrussell/Mining-the-Social-Web/issues/9#issuecomment-2002702
Unfortunately, the example is running as-is without any issues when I re-test it. Try these things and let me know what you see:
Let me know what you find.
Regards - Matthew
http://www.linkedin.com/in/ptwobrussell
On Sep 5, 2011, at 12:03 PM, ajthomas1949 wrote:
ah, sorry I misunderstood. I've now reverted to python couchdb package v 0.7. Here's the result:
Finding docs dated from 2002-1-1 to 2002-2-1 Traceback (most recent call last): File "mailboxes_map_by_date.py", line 46, in
for row in db.view('index/by_date_time', startkey=start, endkey=end): File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/client.py", line 964, in iter for row in self.rows: File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/client.py", line 986, in rows self._fetch() File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/client.py", line 974, in _fetch data = self.view._exec(self.options) File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/client.py", line 859, in exec , _, data = self.resource.get_json(_self._encode_options(options)) File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/http.py", line 372, in get_json status, headers, data = self.get(_a, _k) File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/http.py", line 353, in get return self._request('GET', path, headers=headers, *_params) File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/http.py", line 398, in _request credentials=self.credentials) File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/http.py", line 219, in request resp = _try_request() File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/http.py", line 203, in _try_request return conn.getresponse() File "/usr/lib/python2.7/httplib.py", line 1027, in getresponse response.begin() File "/usr/lib/python2.7/httplib.py", line 407, in begin version, status, reason = self._read_status() File "/usr/lib/python2.7/httplib.py", line 371, in _read_status raise BadStatusLine(line) httplib.BadStatusLine: '' At least it's different! I hope it gives you some clue as to what is going on... And thanks again for your prompt help.
arthur
On 05/09/2011 17:49, ptwobrussell wrote:
Actually - I was referring to downgrading the Python couchdb package you installed. Most people use easy_install or pip to do the installation. I'd recommend learning about those utilities if you haven't yet, because they'll be really useful to you. This specific section in easy_install's documentation shows you how to downgrade your couchdb package, for example: http://packages.python.org/distribute/easy_install.html#changing-the-active-version
You shouldn't need to change the version of CouchDB itself that you're using. I think the problem may just be with the Python package. At least, that's what I'd like for you to try and rule out at this point.
Regards - Matthew
http://www.linkedin.com/in/ptwobrussell
On Sep 5, 2011, at 10:48 AM, ajthomas1949 wrote:
hi, many thanks for your prompt response. Being a newbie with couchDB, I'm not quite sure how to "downgrade". I installed it (version 0.8, on Ubuntu 11.04) with apt-get. Is there some option to apt-get to specify an earlier version? I see from the couchDB web site that it is already up to version 1.1.0 while in the Ubuntu package repository, the latest version is labelled 1.0.1-0ubuntu15. Should I try to install the more recent version from the Apache couchDB version 1.1.0 tarball?
Regards, Arthur
On 02/09/2011 17:48, ptwobrussell wrote:
Out of curiosity, what happens if you downgrade your couchdb to version 0.7? (You can check to make sure your downgrade worked as expected by examining the output of
import couchdb print couchdb.version I think some other readers have had problems with couchdb version 0.8 as well. I may go ahead and try to update some of the code to account for whatever the difference seems to be. It would be helpful to know if this resolves your problem, though I'll admit that the error is somewhat nebulous of helpful meaning.
Reply to this email directly or view it on GitHub: https://github.com/ptwobrussell/Mining-the-Social-Web/issues/9#issuecomment-2002702
Reply to this email directly or view it on GitHub: https://github.com/ptwobrussell/Mining-the-Social-Web/issues/9#issuecomment-2004389
hi, thanks for your continuing patience! As you surmised, I loaded the enron database from the .json file (downloaded from your web site), using the code in Example 3-5. I restarted couchdb, deleted the old enron database and reloaded it, without any obvious error messages. Using the Futon browser, I see that the enron database contains 41,299 rows. This seems a bit odd, since grep/wc says there are 41,670 "Message-ID" lines in the .json file. Random sample records look OK to me. But there is no by_date_time index view visible in Futon (would you expect there to be one before I run Example 3-6, which builds the index?) When I rerun my Example 3-6 code (attached), I get an error message, which is shown in the attached error.log. I notice that no significant time elapsed between the message "Creating index" and "Finished creating index"...
Best arthur
p.s. just before sending this mail, I checked again in Futon, and an index view has now magically appeared (I guess couchdb was busy in the background), but when I click on it I get an error message much like the one above...
On 05/09/2011 20:24, ptwobrussell wrote:
Unfortunately, the example is running as-is without any issues when I re-test it. Try these things and let me know what you see:
- Stop/restart your couchdb and try again
- Go to http://127.0.0.1:5984/_utils/database.html?enron and verify that there are indeed documents loaded in CouchDB. I'm assuming that you loaded them with the previous example, Example 3-5. How many rows does CouchDB report from the bottom of the table?
- Go to http://127.0.0.1:5984/_utils/database.html?enron and under the "View:" combo box, check to see if there is a by_date_time index that appears. If so, select it, and see if it populates a view full of documents.
Let me know what you find.
Regards - Matthew
http://www.linkedin.com/in/ptwobrussell
On Sep 5, 2011, at 12:03 PM, ajthomas1949 wrote:
ah, sorry I misunderstood. I've now reverted to python couchdb package v 0.7. Here's the result:
Finding docs dated from 2002-1-1 to 2002-2-1 Traceback (most recent call last): File "mailboxes_map_by_date.py", line 46, in
for row in db.view('index/by_date_time', startkey=start, endkey=end): File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/client.py", line 964, in iter for row in self.rows: File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/client.py", line 986, in rows self._fetch() File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/client.py", line 974, in _fetch data = self.view._exec(self.options) File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/client.py", line 859, in exec , _, data = self.resource.get_json(_self._encode_options(options)) File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/http.py", line 372, in get_json status, headers, data = self.get(_a, _k) File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/http.py", line 353, in get return self._request('GET', path, headers=headers, *_params) File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/http.py", line 398, in _request credentials=self.credentials) File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/http.py", line 219, in request resp = _try_request() File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/http.py", line 203, in _try_request return conn.getresponse() File "/usr/lib/python2.7/httplib.py", line 1027, in getresponse response.begin() File "/usr/lib/python2.7/httplib.py", line 407, in begin version, status, reason = self._read_status() File "/usr/lib/python2.7/httplib.py", line 371, in _read_status raise BadStatusLine(line) httplib.BadStatusLine: '' At least it's different! I hope it gives you some clue as to what is going on... And thanks again for your prompt help.
arthur
On 05/09/2011 17:49, ptwobrussell wrote:
Actually - I was referring to downgrading the Python couchdb package you installed. Most people use easy_install or pip to do the installation. I'd recommend learning about those utilities if you haven't yet, because they'll be really useful to you. This specific section in easy_install's documentation shows you how to downgrade your couchdb package, for example: http://packages.python.org/distribute/easy_install.html#changing-the-active-version
You shouldn't need to change the version of CouchDB itself that you're using. I think the problem may just be with the Python package. At least, that's what I'd like for you to try and rule out at this point.
Regards - Matthew
http://www.linkedin.com/in/ptwobrussell
On Sep 5, 2011, at 10:48 AM, ajthomas1949 wrote:
hi, many thanks for your prompt response. Being a newbie with couchDB, I'm not quite sure how to "downgrade". I installed it (version 0.8, on Ubuntu 11.04) with apt-get. Is there some option to apt-get to specify an earlier version? I see from the couchDB web site that it is already up to version 1.1.0 while in the Ubuntu package repository, the latest version is labelled 1.0.1-0ubuntu15. Should I try to install the more recent version from the Apache couchDB version 1.1.0 tarball?
Regards, Arthur
On 02/09/2011 17:48, ptwobrussell wrote:
Out of curiosity, what happens if you downgrade your couchdb to version 0.7? (You can check to make sure your downgrade worked as expected by examining the output of
import couchdb print couchdb.version I think some other readers have had problems with couchdb version 0.8 as well. I may go ahead and try to update some of the code to account for whatever the difference seems to be. It would be helpful to know if this resolves your problem, though I'll admit that the error is somewhat nebulous of helpful meaning.
Reply to this email directly or view it on GitHub: https://github.com/ptwobrussell/Mining-the-Social-Web/issues/9#issuecomment-2002702
Reply to this email directly or view it on GitHub: https://github.com/ptwobrussell/Mining-the-Social-Web/issues/9#issuecomment-2004389
-- coding: utf-8 --
import sys, time import couchdb from couchdb.design import ViewDefinition try: import jsonlib2 as json except ImportError: import json
DB = sys.argv[1] START_DATE = sys.argv[2] #YYYY-MM-DD END_DATE = sys.argv[3] #YYYY-MM-DD
server = couchdb.Server('http://localhost:5984') db = server[DB]
def dateTimeToDocMapper(doc):
# Note that you need to include imports used by your mapper
# inside the function definition
from dateutil.parser import parse
from datetime import datetime as dt
if doc.get('Date'):
# [year, month, day, hour, min, sec]
_date = list(dt.timetuple(parse(doc['Date']))[:-3])
yield (_date, doc)
print "Creating index" view = ViewDefinition('index', 'by_date_time', dateTimeToDocMapper, language='python') view.sync(db) print "Finished creating index"
start = [int(i) for i in START_DATE.split("-")] end = [int(i) for i in END_DATE.split("-")] print 'Finding docs dated from %s-%s-%s to %s-%s-%s' % tuple(start + end)
docs = [] for row in db.view('index/by_date_time', startkey=start, endkey=end): docs.append(db.get(row.id)) print json.dumps(docs, indent=4)
ajt@GTW1:~/tools/CouchDB$ python mailboxes_map_by_date.py enron 2002-01-01 2002-02-01
[info] [<0.2259.0>] 127.0.0.1 - - 'HEAD' /enron 200
Creating index
[info] [<0.2259.0>] 127.0.0.1 - - 'GET' /enron/_design%2Findex 301
[info] [<0.2259.0>] 127.0.0.1 - - 'GET' /enron/_design/index 404
[info] [<0.2259.0>] 127.0.0.1 - - 'POST' /enron/_bulk_docs 201
Finished creating index
Finding docs dated from 2002-1-1 to 2002-2-1
No handlers could be found for logger "couchdb.view"
Traceback (most recent call last):
File "mailboxes_map_by_date.py", line 46, in
#Fun<couch_db_updater.8.86519079>,
#Fun<couch_btree.5.124754102>,
#Fun<couch_db_updater.9.24674233>},
{btree,<0.1803.0>,
{170202770,41300},
#Fun<couch_db_updater.10.90337910>,
#Fun<couch_db_updater.11.13595824>,
#Fun<couch_btree.5.124754102>,
#Fun<couch_db_updater.12.34906778>},
{btree,<0.1803.0>,nil,
#Fun<couch_btree.0.83553141>,
#Fun<couch_btree.1.30790806>,
#Fun<couch_btree.2.124754102>,nil},
41300,<<"enron">>,
"/var/lib/couchdb/1.0.1/enron.couch",[],[],nil,
{user_ctx,null,[],undefined},
#Ref<0.0.0.101249>,1000,
[before_header,after_header,on_file_open],
false},
<0.2551.0>,<<"_design/index">>,<<"python">>,[],
[{view,0,
[<<"by_date_time">>],
<<"def dateTimeToDocMapper(doc):\n\n # Note that you need to include imports used by your mapper \n # inside the function definition\n\n from dateutil.parser import parse\n from datetime import datetime as dt\n if doc.get('Date'):\n # [year, month, day, hour, min, sec]\n _date = list(dt.timetuple(parse(doc['Date']))[:-3]) \n yield (_date, doc)">>,
{btree,<0.2551.0>,nil,
#Fun<couch_btree.3.83553141>,
#Fun<couch_btree.4.30790806>,
#Fun<couch_view.less_json_ids.2>,
#Fun<couch_view_group.10.120246376>},
[],[]}],
{btree,<0.2551.0>,nil,#Fun<couch_btree.0.83553141>,
#Fun<couch_btree.1.30790806>,
#Fun<couch_btree.2.124754102>,nil},
0,0,nil,nil},
<0.2553.0>,nil,false,
[{{<0.2259.0>,#Ref<0.0.0.101292>},41300}],
<0.2554.0>}
* Reason for termination ==
* {function_clause,[{lists,map,
[#Fun
for row in self.rows:
File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/client.py", line 986, in rows self._fetch() File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/client.py", line 974, in _fetch data = self.view._exec(self.options) File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/client.py", line 859, in exec , _, data = self.resource.get_json(_self._encode_options(options)) File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/http.py", line 372, in get_json status, headers, data = self.get(_a, _k) File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/http.py", line 353, in get return self._request('GET', path, headers=headers, *_params) File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/http.py", line 398, in _request credentials=self.credentials) File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/http.py", line 219, in request resp = _try_request() File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/http.py", line 203, in _try_request return conn.getresponse() File "/usr/lib/python2.7/httplib.py", line 1027, in getresponse
=ERROR REPORT==== 6-Sep-2011::14:24:02 ===
* Generic server <0.2549.0> terminating
* Last message in was {'EXIT',<0.2553.0>,
{function_clause,
[{lists,map,
[#Fun
#Fun<couch_db_updater.8.86519079>,
#Fun<couch_btree.5.124754102>,
#Fun<couch_db_updater.9.24674233>},
{btree,<0.1803.0>,
{170202770,41300},
#Fun<couch_db_updater.10.90337910>,
#Fun<couch_db_updater.11.13595824>,
#Fun<couch_btree.5.124754102>,
#Fun<couch_db_updater.12.34906778>},
{btree,<0.1803.0>,nil,
#Fun<couch_btree.0.83553141>,
#Fun<couch_btree.1.30790806>,
#Fun<couch_btree.2.124754102>,nil},
41300,<<"enron">>,
"/var/lib/couchdb/1.0.1/enron.couch",[],[],nil,
{user_ctx,null,[],undefined},
#Ref<0.0.0.101249>,1000,
[before_header,after_header,on_file_open],
false},
<0.2551.0>,<<"_design/index">>,<<"python">>,[],
[{view,0,
[<<"by_date_time">>],
<<"def dateTimeToDocMapper(doc):\n\n # Note that you need to include imports used by your mapper \n # inside the function definition\n\n from dateutil.parser import parse\n from datetime import datetime as dt\n if doc.get('Date'):\n # [year, month, day, hour, min, sec]\n _date = list(dt.timetuple(parse(doc['Date']))[:-3]) \n yield (_date, doc)">>,
{btree,<0.2551.0>,nil,
#Fun<couch_btree.3.83553141>,
#Fun<couch_btree.4.30790806>,
#Fun<couch_view.less_json_ids.2>,
#Fun<couch_view_group.10.120246376>},
[],[]}],
{btree,<0.2551.0>,nil,#Fun<couch_btree.0.83553141>,
#Fun<couch_btree.1.30790806>,
#Fun<couch_btree.2.124754102>,nil},
0,0,nil,nil},
<0.2553.0>,nil,false,
[{{<0.2259.0>,#Ref<0.0.0.101292>},41300}],
<0.2554.0>}
* Reason for termination ==
* {function_clause,[{lists,map,
[#Fun
=ERROR REPORT==== 6-Sep-2011::14:24:02 ===
* Generic server <0.2551.0> terminating
* Last message in was {'EXIT',<0.2549.0>,
{function_clause,
[{lists,map,
[#Fun
=CRASH REPORT==== 6-Sep-2011::14:24:02 ===
crasher:
initial call: couch_file:init/1
pid: <0.2551.0>
registered_name: []
exception exit: {function_clause,
[{lists,map,
[#Fun
=CRASH REPORT==== 6-Sep-2011::14:24:02 ===
crasher:
initial call: mochiweb_socket_server:acceptor_loop/1
pid: <0.2259.0>
registered_name: []
exception error: bad argument
in function list_to_binary/1
called as list_to_binary([{lists,map,
[#Fun
=ERROR REPORT==== 6-Sep-2011::14:24:02 ===
{mochiweb_socket_server,235,{child_error,badarg}}
[error] [emulator] Error in process <0.2557.0> with exit value: {function_clause,[{lists,map,[#Fun
=ERROR REPORT==== 6-Sep-2011::14:24:02 ===
Error in process <0.2557.0> with exit value: {function_clause,[{lists,map,[#Fun
[info] [<0.2259.0>] 127.0.0.1 - - 'HEAD' /enron 200
Creating index
[info] [<0.2259.0>] 127.0.0.1 - - 'GET' /enron/_design%2Findex 301
[info] [<0.2259.0>] 127.0.0.1 - - 'GET' /enron/_design/index 404
[info] [<0.2259.0>] 127.0.0.1 - - 'POST' /enron/_bulk_docs 201
Finished creating index
Finding docs dated from 2002-1-1 to 2002-2-1
No handlers could be found for logger "couchdb.view"
Traceback (most recent call last):
File "mailboxes_map_by_date.py", line 46, in
#Fun<couch_db_updater.8.86519079>,
#Fun<couch_btree.5.124754102>,
#Fun<couch_db_updater.9.24674233>},
{btree,<0.1803.0>,
{170202770,41300},
#Fun<couch_db_updater.10.90337910>,
#Fun<couch_db_updater.11.13595824>,
#Fun<couch_btree.5.124754102>,
#Fun<couch_db_updater.12.34906778>},
{btree,<0.1803.0>,nil,
#Fun<couch_btree.0.83553141>,
#Fun<couch_btree.1.30790806>,
#Fun<couch_btree.2.124754102>,nil},
41300,<<"enron">>,
"/var/lib/couchdb/1.0.1/enron.couch",[],[],nil,
{user_ctx,null,[],undefined},
#Ref<0.0.0.101249>,1000,
[before_header,after_header,on_file_open],
false},
<0.2551.0>,<<"_design/index">>,<<"python">>,[],
[{view,0,
[<<"by_date_time">>],
<<"def dateTimeToDocMapper(doc):\n\n # Note that you need to include imports used by your mapper \n # inside the function definition\n\n from dateutil.parser import parse\n from datetime import datetime as dt\n if doc.get('Date'):\n # [year, month, day, hour, min, sec]\n _date = list(dt.timetuple(parse(doc['Date']))[:-3]) \n yield (_date, doc)">>,
{btree,<0.2551.0>,nil,
#Fun<couch_btree.3.83553141>,
#Fun<couch_btree.4.30790806>,
#Fun<couch_view.less_json_ids.2>,
#Fun<couch_view_group.10.120246376>},
[],[]}],
{btree,<0.2551.0>,nil,#Fun<couch_btree.0.83553141>,
#Fun<couch_btree.1.30790806>,
#Fun<couch_btree.2.124754102>,nil},
0,0,nil,nil},
<0.2553.0>,nil,false,
[{{<0.2259.0>,#Ref<0.0.0.101292>},41300}],
<0.2554.0>}
* Reason for termination ==
* {function_clause,[{lists,map,
[#Fun
for row in self.rows:
File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/client.py", line 986, in rows self._fetch() File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/client.py", line 974, in _fetch data = self.view._exec(self.options) File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/client.py", line 859, in exec , _, data = self.resource.get_json(_self._encode_options(options)) File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/http.py", line 372, in get_json status, headers, data = self.get(_a, _k) File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/http.py", line 353, in get return self._request('GET', path, headers=headers, *_params) File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/http.py", line 398, in _request credentials=self.credentials) File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/http.py", line 219, in request resp = _try_request() File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.7-py2.7.egg/couchdb/http.py", line 203, in _try_request return conn.getresponse() File "/usr/lib/python2.7/httplib.py", line 1027, in getresponse
=ERROR REPORT==== 6-Sep-2011::14:24:02 ===
* Generic server <0.2549.0> terminating
* Last message in was {'EXIT',<0.2553.0>,
{function_clause,
[{lists,map,
[#Fun
#Fun<couch_db_updater.8.86519079>,
#Fun<couch_btree.5.124754102>,
#Fun<couch_db_updater.9.24674233>},
{btree,<0.1803.0>,
{170202770,41300},
#Fun<couch_db_updater.10.90337910>,
#Fun<couch_db_updater.11.13595824>,
#Fun<couch_btree.5.124754102>,
#Fun<couch_db_updater.12.34906778>},
{btree,<0.1803.0>,nil,
#Fun<couch_btree.0.83553141>,
#Fun<couch_btree.1.30790806>,
#Fun<couch_btree.2.124754102>,nil},
41300,<<"enron">>,
"/var/lib/couchdb/1.0.1/enron.couch",[],[],nil,
{user_ctx,null,[],undefined},
#Ref<0.0.0.101249>,1000,
[before_header,after_header,on_file_open],
false},
<0.2551.0>,<<"_design/index">>,<<"python">>,[],
[{view,0,
[<<"by_date_time">>],
<<"def dateTimeToDocMapper(doc):\n\n # Note that you need to include imports used by your mapper \n # inside the function definition\n\n from dateutil.parser import parse\n from datetime import datetime as dt\n if doc.get('Date'):\n # [year, month, day, hour, min, sec]\n _date = list(dt.timetuple(parse(doc['Date']))[:-3]) \n yield (_date, doc)">>,
{btree,<0.2551.0>,nil,
#Fun<couch_btree.3.83553141>,
#Fun<couch_btree.4.30790806>,
#Fun<couch_view.less_json_ids.2>,
#Fun<couch_view_group.10.120246376>},
[],[]}],
{btree,<0.2551.0>,nil,#Fun<couch_btree.0.83553141>,
#Fun<couch_btree.1.30790806>,
#Fun<couch_btree.2.124754102>,nil},
0,0,nil,nil},
<0.2553.0>,nil,false,
[{{<0.2259.0>,#Ref<0.0.0.101292>},41300}],
<0.2554.0>}
* Reason for termination == *
Could you try downgrading your dateutil package to version 1.5 (the same way you did this with easy_install for your couchdb package), delete the by_date_time index, restart CouchDB and try again? The errors "File \"/usr/local/lib/python2.7/dist-packages/python_dateutil-2.0-py2.7.egg/dateutil/parser.py\", line 44, in init\n instream = StringIO(instream)\nTypeError: initial_value must be unicode or None, not str\n" make me suspicious that something has changed with dateutil (I developed with version 1.5), and this could be an underlying source of problem.
I appreciate you sticking with it, and I really want to help you get sorted out. Once we are all squared away, I'll try to figure out the simplest way to update readers of the book. With so much OSS involved, not to mention evolving social web APIs, there are a lot of opportunities for these types of things to happen.
hi, Matthew, yes, dateutil seems to have been the culprit! I down-rev'd to dateutil 1.5, reloaded the enron database and am now able to produce sensible-looking results from Example 3-6. I really appreciate your help with this. I hope you're able to track down what changed in dateutil, and find a work-around, so that other people don't have the same problem. I also tried Example 3-7, which worked flawlessly.
I am a bit surprised (coming from experience with MySQL and the like) at the leisurely pace at which CouchDB builds its indices -- very much in the tradition of information retrieval systems! I guess it really is aimed at write-once, read-many kinds of apps...
Cheers arthur
On 06/09/2011 18:31, ptwobrussell wrote:
Could you try downgrading your dateutil package to version 1.5 (the same way you did this with easy_install for your couchdb package), delete the by_date_time index, restart CouchDB and try again? The errors "File \"/usr/local/lib/python2.7/dist-packages/python_dateutil-2.0-py2.7.egg/dateutil/parser.py\", line 44, in init\n instream = StringIO(instream)\nTypeError: initial_value must be unicode or None, not str\n" make me suspicious that something has changed with dateutil (I developed with version 1.5), and this could be an underlying source of problem.
I appreciate you sticking with it, and I really want to help you get sorted out. Once we are all squared away, I'll try to figure out the simplest way to update readers of the book. With so much OSS involved, not to mention evolving social web APIs, there are a lot of opportunities for these types of things to happen.
I'm glad we were able to track it down. As time allows, I'll take a closer look to see what's going on with dateutil v2 vs v1.5 to see what is going on. In the meanwhile, I may just add a warning to the script that prints out a note to the user. Thanks for helping to track this one down.
Out of curiosity, how did you go about finding a dateutil-2.0 package? I just checked PyPi http://pypi.python.org/pypi and the latest version that they host is 1.5, so easy_install shouldn't install anything else I don't think. However, when I checked Labix's page http://labix.org/python-dateutil, I see that under "Downloads", they have both a v1.5 and a v2.0. The v2.0, however is marked as Python 3.x, and you were running Python 2.7. Perhaps you just accidentally downloaded the wrong version? (The reason I'm asking is because I'm trying to determine if you did something common enough that lots of other readers may accidentally do the same thing.) Thanks again.
When I run Example 3-6 (code as downloaded from github) on a CouchDB datebase created from the Enron corpus, on an Ubuntu 10.04 machine, I get:
Finding docs dated from 2002-1-1 to 2002-2-1 Traceback (most recent call last): File "mailboxes_map_by_date.py", line 46, in
for row in db.view('index/by_date_time', startkey=start, endkey=end):
File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.8-py2.7.egg/couchdb/client.py", line 984, in iter
return iter(self.rows)
File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.8-py2.7.egg/couchdb/client.py", line 1003, in rows
self._fetch()
File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.8-py2.7.egg/couchdb/client.py", line 990, in _fetch
data = self.view._exec(self.options)
File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.8-py2.7.egg/couchdb/client.py", line 880, in exec
, _, data = self.resource.get_json(_self._encode_options(options))
File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.8-py2.7.egg/couchdb/http.py", line 393, in get_json
status, headers, data = self.get(_a, _k)
File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.8-py2.7.egg/couchdb/http.py", line 374, in get
return self._request('GET', path, headers=headers, *_params)
File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.8-py2.7.egg/couchdb/http.py", line 419, in _request
credentials=self.credentials)
File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.8-py2.7.egg/couchdb/http.py", line 239, in request
resp = _try_request_with_retries(iter(self.retry_delays))
File "/usr/local/lib/python2.7/dist-packages/CouchDB-0.8-py2.7.egg/couchdb/http.py", line 205, in _try_request_with_retries
raise e
socket.error: 104
This is, I think, a "Connection reset by peer" error. I've looked through the various python and CouchDB forums and not found anything that appears relevant. Any suggestions gratefully received
Arthur