crawl.py

[oynsal]

When I run crawl.py I receive an exception as soon as it starts recording posts to the database.

<nntplib.NNTP instance at 0x887650c>

• inner slice: 0 - 9, 9 (9 - 18) Traceback (most recent call last): File "./crawl.py", line 23, in group.parse() File "/opt/Usenetr/raw/models.py", line 98, in parse last = parser._parse(iterator) File "/opt/Usenetr/raw/models.py", line 314, in _parse post.save() File "/usr/lib/pymodules/python2.6/django/db/models/base.py", line 410, in save self.save_base(force_insert=force_insert, force_update=force_update) File "/usr/lib/pymodules/python2.6/django/db/models/base.py", line 495, in save_base result = manager._insert(values, return_id=update_pk) File "/usr/lib/pymodules/python2.6/django/db/models/manager.py", line 177, in _insert return insert_query(self.model, values, **kwargs) File "/usr/lib/pymodules/python2.6/django/db/models/query.py", line 1087, in insert_query return query.execute_sql(return_id) File "/usr/lib/pymodules/python2.6/django/db/models/sql/subqueries.py", line 320, in execute_sql cursor = super(InsertQuery, self).execute_sql(None) File "/usr/lib/pymodules/python2.6/django/db/models/sql/query.py", line 2369, in execute_sql cursor.execute(sql, params) File "/usr/lib/pymodules/python2.6/django/db/backends/util.py", line 19, in execute return self.cursor.execute(sql, params) File "/usr/lib/pymodules/python2.6/django/db/backends/mysql/base.py", line 84, in execute return self.cursor.execute(query, args) File "/usr/lib/pymodules/python2.6/MySQLdb/cursors.py", line 168, in execute if not self._defer_warnings: self._warning_check() File "/usr/lib/pymodules/python2.6/MySQLdb/cursors.py", line 82, in _warning_check warn(w[-1], self.Warning, 3) _mysql_exceptions.Warning: Data truncated for column 'id' at row 1

If I alter the tables to change the character length to 64 rather then 32 it gets a bit further. Is that a correct solution?

[taktix]

yes it looks like things were being truncated, but not reported since i was testing using sqlite.