search

mysociety / sayit.mysociety.org

A hosting service for data using the SayIt component
http://sayit.mysociety.org/
Other
5 stars 4 forks source link

"speeches_speaker_instance_id_6538f9cb_uniq" Problem #97

Closed johnfelipe closed 8 years ago

johnfelipe commented 8 years ago

I have a new installation

and change wtih git checkout update-for-newer-sayit

I have correct xml folder with 27 xml to upload, but i have a problem

Traceback (most recent call last):
  File "/home/felipe/sayit/virtualenv-sayit/src/django-sayit/speeches/management/import_commands.py", line 124, in import_document
    importer.import_document(path)
  File "/home/felipe/sayit/virtualenv-sayit/src/django-sayit/speeches/importers/import_akomantoso.py", line 25, in import_document
    return self.parse_document()
  File "/home/felipe/sayit/virtualenv-sayit/src/django-sayit/speeches/importers/import_akomantoso.py", line 54, in parse_document
    speaker.save()
  File "/home/felipe/sayit/virtualenv-sayit/src/django-sayit/speeches/models.py", line 163, in save
    return super(Speaker, self).save(*args, **kwargs)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/models/base.py", line 734, in save
    force_update=force_update, update_fields=update_fields)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/models/base.py", line 762, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/models/base.py", line 846, in _save_table
    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/models/base.py", line 885, in _do_insert
    using=using, raw=raw)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/models/manager.py", line 127, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/models/query.py", line 920, in _insert
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 974, in execute_sql
    cursor.execute(sql, params)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/utils.py", line 98, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
IntegrityError: duplicate key value violates unique constraint "speeches_speaker_instance_id_6538f9cb_uniq"
DETAIL:  Key (instance_id, slug)=(1, dr-diego-molano-vega) already exists.

How fix it, pls

dracos commented 8 years ago

That error means you have two speakers in your XML TLCPerson entries with the same name. If they are the same person, they should be combined; if not, currently you will have to give them different showAs attributes for them to be treated differently. So the issue is with your XML.

johnfelipe commented 8 years ago

why show this error in this new installation and not with sayit app?

johnfelipe commented 8 years ago

This is 3 example xml, for me have a good format:

https://drive.google.com/open?id=0B5b2y2kCFpKoLUpRSThqc3VaLXM

dracos commented 8 years ago

The 3 XML files in that directory import fine into SayIt here:

$ python manage.py load_akomantoso --dir acta20160426/ --instance=acta20160426 --commit --verbosity=2
Starting import: acta20160426/acta-09-12-septiembre-11-de-2012.xml
acta20160426/acta-09-12-septiembre-11-de-2012.xml: Imported Speech:153 Speaker:84 Section:5
Starting import: acta20160426/acta-13-12-octubre-2-de-2012.xml
acta20160426/acta-13-12-octubre-2-de-2012.xml: Imported Speech:103 Speaker:0 Section:5
Starting import: acta20160426/acta-27-13-febrero-19-de-2013.xml
acta20160426/acta-27-13-febrero-19-de-2013.xml: Imported Speech:84 Speaker:0 Section:5

So those are clearly not the same as whatever XML files you are importing that give an error.

johnfelipe commented 8 years ago

@dracos you are using new PR?

dracos commented 8 years ago

Yes. Sigh, is that the whole error you quoted at the start of this issue? Because I have managed to replicate it, but the error is caused by a "BulkIndexError" first and then that leads to the "IntegrityError" you quote, because the first error has caused it to not save the href of the person successfully. If you had quoted the whole error at the start, this would have made it a lot simpler. Please do not try and provide only what you think is the required information, because you have cut out information that was integral to diagnosing the problem.

You are running a version of the elasticsearch PyPI module that is incompatible with the version of elasticsearch you are running. So it is unable to index the speaker, which causes the BulkIndexError, which causes the IntegrityError. It is not up to us to make sure you are running the relevant versions of software to run our software. If you are running elasticsearch 0.90, as I suspect you are, then you need to install elasticsearch 0.4.5, not any later version. Inside a virtualenv, you can run "pip freeze" to see what version of elasticsearch you have installed, and "pip install elasticsearch==0.4.5" to install that version, if what I have said is correct.

johnfelipe commented 8 years ago

My new instance is in ubuntu 32 bits and echo "deb http://packages.elastic.co/elasticsearch/1.7/debian stable main" | sudo tee -a /etc/apt/sources.list.d/elasticsearch-1.7.list, because 2.x in 32 bits is not possible.

with pip freeze show:

audioread==2.1.2
beautifulsoup4==4.4.1
bleach==1.4.2
cffi==1.6.0
cryptography==1.3.1
cssselect==0.9.1
Django==1.8.12
django-allauth==0.25.2
django-apptemplates==1.0
django-bleach==0.3.0
django-haystack==2.4.1
django-model-utils==2.5
django-pipeline==1.6.8
-e git+https://github.com/mysociety/sayit@84f4c451ba428e8aaefc136c094cf57be074942f#egg=django_sayit
Django-Select2==4.3.2
django-subdomain-instances==1.0
django-tastypie==0.13.3
easy-thumbnails==2.3
elasticsearch==2.3.0
enum34==1.1.3
gunicorn==17.5
html5lib==0.9999999
idna==2.1
ipaddress==1.0.16
lxml==3.6.0
mysociety-django-pipeline-compass-rubygem==0.1.8.1
mysociety-django-popolo==0.0.5
mysociety-django-sluggable==0.2.7
ndg-httpsclient==0.3.2
oauthlib==1.0.3
pdfminer==20140328
Pillow==3.2.0
psycopg2==2.5.5
pyasn1==0.1.7
pycparser==2.14
pyOpenSSL==0.14
python-dateutil==2.5.3
python-memcached==1.57
python-mimeparse==1.5.1
python-openid==2.2.5
pytz==2016.4
PyYAML==3.11
requests==2.9.1
requests-cache==0.4.12
requests-oauthlib==0.6.1
six==1.10.0
unicode-slugify==0.1.1
urllib3==1.15.1

I install pip install elasticsearch==0.4.5, next

./manage.py load_akomantoso --dir=/home/felipe/comisionsextasenado/xml --instance=comisionsextasenado --commit --merge-existing

I upload all xml files to same folder: https://drive.google.com/open?id=0B5b2y2kCFpKoLUpRSThqc3VaLXM, 27 in total

An all log error is:

http://pastebin.com/Nhyn0KQV

In admin UI create this only: speakercreated

dracos commented 8 years ago

As I said above, pypi elasticsearch 0.4.5 is if you elasticsearch 0.9. You have now said you have 1.7 so you must install a 1.x version of pypi elasticsearch as it says on https://pypi.python.org/pypi/elasticsearch/2.3.0 - you have 2.3.0 installed which is not compatible with elasticsearch 1.7.

dracos commented 8 years ago

PS Once you get the integrity error, you should empty your database or delete everything to do with that instance and start again to make sure it's cleared, otherwise you may still get errors due to bad data in the database.

johnfelipe commented 8 years ago

As i said i install 0.4.5 before load xml, now i flush database, sync, and this is my new pip freeze, with 1.7:

(virtualenv-sayit) felipe@felipeurrego:~/sayit$ pip freeze
audioread==2.1.2
beautifulsoup4==4.4.1
bleach==1.4.2
cffi==1.6.0
cryptography==1.3.1
cssselect==0.9.1
Django==1.8.12
django-allauth==0.25.2
django-apptemplates==1.0
django-bleach==0.3.0
django-haystack==2.4.1
django-model-utils==2.5
django-pipeline==1.6.8
-e git+https://github.com/mysociety/sayit@84f4c451ba428e8aaefc136c094cf57be074942f#egg=django_sayit
Django-Select2==4.3.2
django-subdomain-instances==1.0
django-tastypie==0.13.3
easy-thumbnails==2.3
elasticsearch==1.7.0
enum34==1.1.3
gunicorn==17.5
html5lib==0.9999999
idna==2.1
ipaddress==1.0.16
lxml==3.6.0
mysociety-django-pipeline-compass-rubygem==0.1.8.1
mysociety-django-popolo==0.0.5
mysociety-django-sluggable==0.2.7
ndg-httpsclient==0.3.2
oauthlib==1.0.3
pdfminer==20140328
Pillow==3.2.0
psycopg2==2.5.5
pyasn1==0.1.7
pycparser==2.14
pyOpenSSL==0.14
python-dateutil==2.5.3
python-memcached==1.57
python-mimeparse==1.5.1
python-openid==2.2.5
pytz==2016.4
PyYAML==3.11
requests==2.9.1
requests-cache==0.4.12
requests-oauthlib==0.6.1
six==1.10.0
unicode-slugify==0.1.1
urllib3==1.15.1

I create new instance

Same error, error persist

dracos commented 8 years ago

Thanks for the pip freeze, that should at least rule that out as a source of error. Could you run your load_akomantoso line with --verbosity=2 and a new instance and pastebin all the output including the command line.

johnfelipe commented 8 years ago

For load folder:

http://pastebin.com/1qgu1pR1

For load one file:

(virtualenv-sayit) felipe@felipeurrego:~/sayit/sayit.mysociety.org$ ./manage.py load_akomantoso --file=/home/felipe/comisionsextasenado/xml/acta-09-12-septiembre-11-de-2012.xml --instance=comisionsextasenado --commit --verbosity=2
Starting import: /home/felipe/comisionsextasenado/xml/acta-09-12-septiembre-11-de-2012.xml

An exception of type IntegrityError occurred, arguments:
duplicate key value violates unique constraint "speeches_speaker_instance_id_6538f9cb_uniq"
DETAIL:  Key (instance_id, slug)=(1, dr-diego-molano-vega) already exists.

Traceback (most recent call last):
  File "/home/felipe/sayit/virtualenv-sayit/src/django-sayit/speeches/management/import_commands.py", line 124, in import_document
    importer.import_document(path)
  File "/home/felipe/sayit/virtualenv-sayit/src/django-sayit/speeches/importers/import_akomantoso.py", line 25, in import_document
    return self.parse_document()
  File "/home/felipe/sayit/virtualenv-sayit/src/django-sayit/speeches/importers/import_akomantoso.py", line 54, in parse_document
    speaker.save()
  File "/home/felipe/sayit/virtualenv-sayit/src/django-sayit/speeches/models.py", line 163, in save
    return super(Speaker, self).save(*args, **kwargs)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/models/base.py", line 734, in save
    force_update=force_update, update_fields=update_fields)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/models/base.py", line 762, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/models/base.py", line 846, in _save_table
    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/models/base.py", line 885, in _do_insert
    using=using, raw=raw)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/models/manager.py", line 127, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/models/query.py", line 920, in _insert
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 974, in execute_sql
    cursor.execute(sql, params)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/utils.py", line 98, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
IntegrityError: duplicate key value violates unique constraint "speeches_speaker_instance_id_6538f9cb_uniq"
DETAIL:  Key (instance_id, slug)=(1, dr-diego-molano-vega) already exists.
dracos commented 8 years ago

Your load folder pastebin does not start at the beginning of the error output or include the command line, so I do not know what the first error was, nor if you ran it on a unique instance. I have installed the same version of everything as you have listed in pip freeze, and the import works fine for me on an empty instance. Are you sure you are emptying the database or using a new instance label? How are you emptying/deleting your database? Please try using a new unique instance name, in case you are somehow not managing to delete everything from the database. Here is a full working transcript locally:

$ pip freeze
audioread==2.1.2
beautifulsoup4==4.4.1
bleach==1.4.2
cffi==1.6.0
cryptography==1.3.1
cssselect==0.9.1
Django==1.8.12
django-allauth==0.25.2
django-apptemplates==1.0
django-bleach==0.3.0
django-haystack==2.4.1
django-model-utils==2.5
django-pipeline==1.6.8
-e git+https://github.com/mysociety/sayit@84f4c451ba428e8aaefc136c094cf57be074942f#egg=django_sayit-master
Django-Select2==4.3.2
django-subdomain-instances==1.0
django-tastypie==0.13.3
easy-thumbnails==2.3
elasticsearch==1.7.0
enum34==1.1.3
gunicorn==17.5
html5lib==0.9999999
idna==2.1
ipaddress==1.0.16
lxml==3.6.0
mysociety-django-pipeline-compass-rubygem==0.1.8.1
mysociety-django-popolo==0.0.5
mysociety-django-sluggable==0.2.7
ndg-httpsclient==0.3.2
oauthlib==1.0.3
pdfminer==20140328
Pillow==3.2.0
psycopg2==2.5.5
pyasn1==0.1.7
pycparser==2.14
pyOpenSSL==0.14
python-dateutil==2.5.3
python-memcached==1.57
python-mimeparse==1.5.1
python-openid==2.2.5
pytz==2016.4
PyYAML==3.11
requests==2.9.1
requests-cache==0.4.12
requests-oauthlib==0.6.1
six==1.10.0
unicode-slugify==0.1.1
urllib3==1.15.1
wheel==0.24.0
$ git status
On branch update-for-newer-sayit
Your branch is up-to-date with 'origin/update-for-newer-sayit'.
Untracked files:
  (use "git add <file>..." to include in what will be committed)
    comisionsextasenado-2016-04-26.zip
    comisionsextasenado/
$ curl http://localhost:9200/
{
  "status" : 200,
  "name" : "Ikonn",
  "cluster_name" : "elasticsearch",
  "version" : {
    "number" : "1.7.5",
    "build_hash" : "00f95f4ffca6de89d68b7ccaf80d148f1f70e4d4",
    "build_timestamp" : "2016-02-02T09:55:30Z",
    "build_snapshot" : false,
    "lucene_version" : "4.10.4"
  },
  "tagline" : "You Know, for Search"
$ python manage.py load_akomantoso --file=comisionsextasenado/acta-09-12-septiembre-11-de-2012.xml --instance=testingA --verbosity=2 --commit
Starting import: comisionsextasenado/acta-09-12-septiembre-11-de-2012.xml
Imported Section:5 Speech:153 Speaker:84
$ python manage.py load_akomantoso --dir comisionsextasenado --instance=testingB --verbosity=2 --commit
Starting import: comisionsextasenado/acta-01-12-julio-25-de-2012.xml
comisionsextasenado/acta-01-12-julio-25-de-2012.xml: Imported Section:4 Speech:57 Speaker:105
Starting import: comisionsextasenado/acta-02-12-julio-31-de-2012-f.xml
comisionsextasenado/acta-02-12-julio-31-de-2012-f.xml: Imported Section:4 Speech:79 Speaker:0
Starting import: comisionsextasenado/acta-03-12-agosto-14-de-2012.xml
comisionsextasenado/acta-03-12-agosto-14-de-2012.xml: Imported Section:4 Speech:48 Speaker:0
Starting import: comisionsextasenado/acta-04-12-agosto-15-de-2012.xml
comisionsextasenado/acta-04-12-agosto-15-de-2012.xml: Imported Section:4 Speech:33 Speaker:0
Starting import: comisionsextasenado/acta-05-12-agosto-21-de-2012.xml
comisionsextasenado/acta-05-12-agosto-21-de-2012.xml: Imported Section:4 Speech:62 Speaker:0
Starting import: comisionsextasenado/acta-06-12-agosto-22-de-2012.xml
comisionsextasenado/acta-06-12-agosto-22-de-2012.xml: Imported Section:5 Speech:85 Speaker:0
Starting import: comisionsextasenado/acta-07-12-agosto-28-de-2012.xml
comisionsextasenado/acta-07-12-agosto-28-de-2012.xml: Imported Section:4 Speech:63 Speaker:0
Starting import: comisionsextasenado/acta-08-12-septiembre-4-de-2012.xml
comisionsextasenado/acta-08-12-septiembre-4-de-2012.xml: Imported Section:4 Speech:10 Speaker:0
Starting import: comisionsextasenado/acta-09-12-septiembre-11-de-2012.xml
comisionsextasenado/acta-09-12-septiembre-11-de-2012.xml: Imported Section:5 Speech:153 Speaker:0
Starting import: comisionsextasenado/acta-10-12-septiembre-18-de-2012.xml
comisionsextasenado/acta-10-12-septiembre-18-de-2012.xml: Imported Section:5 Speech:35 Speaker:0
Starting import: comisionsextasenado/acta-11-12-septiembre-25-de-2012.xml
comisionsextasenado/acta-11-12-septiembre-25-de-2012.xml: Imported Section:5 Speech:82 Speaker:0
Starting import: comisionsextasenado/acta-12-12-septiembre-26-de-2012.xml
comisionsextasenado/acta-12-12-septiembre-26-de-2012.xml: Imported Section:5 Speech:77 Speaker:0
Starting import: comisionsextasenado/acta-13-12-octubre-2-de-2012.xml
comisionsextasenado/acta-13-12-octubre-2-de-2012.xml: Imported Section:5 Speech:103 Speaker:0
Starting import: comisionsextasenado/acta-14-12-octubre-9-de-2012.xml
comisionsextasenado/acta-14-12-octubre-9-de-2012.xml: Imported Section:5 Speech:32 Speaker:0
Starting import: comisionsextasenado/acta-15-12-octubre-16-de-2012.xml
comisionsextasenado/acta-15-12-octubre-16-de-2012.xml: Imported Section:5 Speech:93 Speaker:0
Starting import: comisionsextasenado/acta-16-12-octubre-24-de-2012.xml
comisionsextasenado/acta-16-12-octubre-24-de-2012.xml: Imported Section:5 Speech:79 Speaker:0
Starting import: comisionsextasenado/acta-17-12-octubre-30-de-2012.xml
comisionsextasenado/acta-17-12-octubre-30-de-2012.xml: Imported Section:5 Speech:78 Speaker:0
Starting import: comisionsextasenado/acta-18-12-noviembre-7-de-2012.xml
comisionsextasenado/acta-18-12-noviembre-7-de-2012.xml: Imported Section:5 Speech:66 Speaker:0
Starting import: comisionsextasenado/acta-19-12-noviembre-14-de-2012.xml
comisionsextasenado/acta-19-12-noviembre-14-de-2012.xml: Imported Section:5 Speech:48 Speaker:0
Starting import: comisionsextasenado/acta-20-12-noviembre-20-de-2012.xml
comisionsextasenado/acta-20-12-noviembre-20-de-2012.xml: Imported Section:5 Speech:126 Speaker:0
Starting import: comisionsextasenado/acta-21-de-noviembre-de-2012.xml
comisionsextasenado/acta-21-de-noviembre-de-2012.xml: Imported Section:5 Speech:89 Speaker:0
Starting import: comisionsextasenado/acta-22-12-noviembre-27-de-2012.xml
comisionsextasenado/acta-22-12-noviembre-27-de-2012.xml: Imported Section:5 Speech:83 Speaker:0
Starting import: comisionsextasenado/acta-23-12-noviembre-28-de-2012.xml
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
comisionsextasenado/acta-23-12-noviembre-28-de-2012.xml: Imported Section:5 Speech:61 Speaker:0
Starting import: comisionsextasenado/acta-24-12-diciembre-4-de-2012-1.xml
comisionsextasenado/acta-24-12-diciembre-4-de-2012-1.xml: Imported Section:4 Speech:99 Speaker:0
Starting import: comisionsextasenado/acta-26-12-diciembre-12-de-20121.xml
comisionsextasenado/acta-26-12-diciembre-12-de-20121.xml: Imported Section:4 Speech:45 Speaker:0
Starting import: comisionsextasenado/acta-27-13-febrero-19-de-2013.xml
comisionsextasenado/acta-27-13-febrero-19-de-2013.xml: Imported Section:5 Speech:84 Speaker:0
Starting import: comisionsextasenado/acta-28-13-febrero-20-de-2013.xml
comisionsextasenado/acta-28-13-febrero-20-de-2013.xml: Imported Section:5 Speech:77 Speaker:0
$

As you can see, it works fine. My assumption is that you are not deleting everything from the database correctly, so bad data is still there for when you next try to run. If there is nothing else in the database you need, why not just delete it and create a new one? As I already said, you must empty all the bad data out or you'll just get the error, even if the error would not happen on an empty database.

johnfelipe commented 8 years ago

With my sayit.mysociety.org,in ubuntu 32 bits this is:

(virtualenv-sayit) felipe@felipeurrego:~/sayit/sayit.mysociety.org$ python manage.py flush
You have requested a flush of the database.
This will IRREVERSIBLY DESTROY all data currently in the 'sayit' database,
and return each table to an empty state.
Are you sure you want to do this?

    Type 'yes' to continue, or 'no' to cancel: yes
(virtualenv-sayit) felipe@felipeurrego:~/sayit/sayit.mysociety.org$ pip install -r requirements.txt
Requirement already satisfied (use --upgrade to upgrade): Django<1.9,>=1.8 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from -r requirements.txt (line 1))
Requirement already satisfied (use --upgrade to upgrade): django-allauth==0.25.2 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from -r requirements.txt (line 2))
Requirement already satisfied (use --upgrade to upgrade): PyYAML in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from -r requirements.txt (line 4))
Requirement already satisfied (use --upgrade to upgrade): django-pipeline<1.7,>=1.6 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from -r requirements.txt (line 5))
Requirement already satisfied (use --upgrade to upgrade): mysociety-django-pipeline-compass-rubygem in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from -r requirements.txt (line 7))
Requirement already satisfied (use --upgrade to upgrade): python-memcached in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from -r requirements.txt (line 8))
Requirement already satisfied (use --upgrade to upgrade): django-tastypie in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from -r requirements.txt (line 9))
Requirement already satisfied (use --upgrade to upgrade): beautifulsoup4 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from -r requirements.txt (line 12))
Requirement already satisfied (use --upgrade to upgrade): requests-cache in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from -r requirements.txt (line 13))
Requirement already satisfied (use --upgrade to upgrade): django-apptemplates==1.0 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from -r requirements.txt (line 14))
Obtaining django-sayit from git+https://github.com/mysociety/sayit#egg=django-sayit (from -r requirements.txt (line 16))
  Updating /home/felipe/sayit/virtualenv-sayit/src/django-sayit clone
Requirement already satisfied (use --upgrade to upgrade): requests>=1.0.3 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from django-allauth==0.25.2->-r requirements.txt (line 2))
Requirement already satisfied (use --upgrade to upgrade): python-openid>=2.2.5 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from django-allauth==0.25.2->-r requirements.txt (line 2))
Requirement already satisfied (use --upgrade to upgrade): requests-oauthlib>=0.3.0 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from django-allauth==0.25.2->-r requirements.txt (line 2))
Requirement already satisfied (use --upgrade to upgrade): six in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from python-memcached->-r requirements.txt (line 8))
Requirement already satisfied (use --upgrade to upgrade): python-dateutil!=2.0,>=1.5 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from django-tastypie->-r requirements.txt (line 9))
Requirement already satisfied (use --upgrade to upgrade): python-mimeparse!=1.5,>=0.1.4 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from django-tastypie->-r requirements.txt (line 9))
Requirement already satisfied (use --upgrade to upgrade): psycopg2<2.6,>=2.5.1 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from django-sayit->-r requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): pytz>=2013d in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from django-sayit->-r requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): Django-Select2==4.3.2 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from django-sayit->-r requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): audioread>=1.0.1 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from django-sayit->-r requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): elasticsearch>=0.4 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from django-sayit->-r requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): django-haystack<2.5,>=2.4 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from django-sayit->-r requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): django-bleach>=0.2.1 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from django-sayit->-r requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): mysociety-django-popolo>=0.0.5 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from django-sayit->-r requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): mysociety-django-sluggable>=0.2.7 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from django-sayit->-r requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): django-subdomain-instances>=1.0 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from django-sayit->-r requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): easy-thumbnails>=2.1 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from django-sayit->-r requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): unicode-slugify==0.1.1 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from django-sayit->-r requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): pyOpenSSL==0.14 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from django-sayit->-r requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): ndg-httpsclient==0.3.2 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from django-sayit->-r requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): pyasn1==0.1.7 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from django-sayit->-r requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): oauthlib>=0.6.2 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from requests-oauthlib>=0.3.0->django-allauth==0.25.2->-r requirements.txt (line 2))
Requirement already satisfied (use --upgrade to upgrade): urllib3<2.0,>=1.8 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from elasticsearch>=0.4->django-sayit->-r requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): bleach in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from django-bleach>=0.2.1->django-sayit->-r requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): django-model-utils in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from mysociety-django-popolo>=0.0.5->django-sayit->-r requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): pillow in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from easy-thumbnails>=2.1->django-sayit->-r requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): cryptography>=0.2.1 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from pyOpenSSL==0.14->django-sayit->-r requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): html5lib>=0.999 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from bleach->django-bleach>=0.2.1->django-sayit->-r requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): setuptools>=11.3 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from cryptography>=0.2.1->pyOpenSSL==0.14->django-sayit->-r requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): enum34 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from cryptography>=0.2.1->pyOpenSSL==0.14->django-sayit->-r requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): ipaddress in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from cryptography>=0.2.1->pyOpenSSL==0.14->django-sayit->-r requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): idna>=2.0 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from cryptography>=0.2.1->pyOpenSSL==0.14->django-sayit->-r requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): cffi>=1.4.1 in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from cryptography>=0.2.1->pyOpenSSL==0.14->django-sayit->-r requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): pycparser in /home/felipe/sayit/virtualenv-sayit/lib/python2.7/site-packages (from cffi>=1.4.1->cryptography>=0.2.1->pyOpenSSL==0.14->django-sayit->-r requirements.txt (line 16))
Installing collected packages: django-sayit
  Running setup.py develop for django-sayit
Successfully installed django-sayit-1.4
(virtualenv-sayit) felipe@felipeurrego:~/sayit/sayit.mysociety.org$ ./manage.py syncdb
Operations to perform:
  Synchronize unmigrated apps: pipeline, staticfiles, admindocs, twitter, messages, about, allauth, humanize, django_select2, django_bleach, haystack
  Apply all migrations: account, sessions, admin, speeches, login_token, sites, auth, tastypie, instances, contenttypes, popolo, easy_thumbnails, socialaccount
Synchronizing apps without migrations:
  Creating tables...
    Running deferred SQL...
  Installing custom SQL...
Running migrations:
  No migrations to apply.

You have installed Django's auth system, and don't have any superusers defined.
Would you like to create one now? (yes/no): yes
Username (leave blank to use 'felipe'):
Email address: ingenierofelipeurrego@gmail.com
Password:
Password (again):
Superuser created successfully.

(virtualenv-sayit) felipe@felipeurrego:~/sayit/sayit.mysociety.org$ pip freeze
audioread==2.1.2
beautifulsoup4==4.4.1
bleach==1.4.2
cffi==1.6.0
cryptography==1.3.1
cssselect==0.9.1
Django==1.8.12
django-allauth==0.25.2
django-apptemplates==1.0
django-bleach==0.3.0
django-haystack==2.4.1
django-model-utils==2.5
django-pipeline==1.6.8
-e git+https://github.com/mysociety/sayit@84f4c451ba428e8aaefc136c094cf57be074942f#egg=django_sayit
Django-Select2==4.3.2
django-subdomain-instances==1.0
django-tastypie==0.13.3
easy-thumbnails==2.3
elasticsearch==1.7.0
enum34==1.1.3
gunicorn==17.5
html5lib==0.9999999
idna==2.1
ipaddress==1.0.16
lxml==3.6.0
mysociety-django-pipeline-compass-rubygem==0.1.8.1
mysociety-django-popolo==0.0.5
mysociety-django-sluggable==0.2.7
ndg-httpsclient==0.3.2
oauthlib==1.0.3
pdfminer==20140328
Pillow==3.2.0
psycopg2==2.5.5
pyasn1==0.1.7
pycparser==2.14
pyOpenSSL==0.14
python-dateutil==2.5.3
python-memcached==1.57
python-mimeparse==1.5.1
python-openid==2.2.5
pytz==2016.4
PyYAML==3.11
requests==2.9.1
requests-cache==0.4.12
requests-oauthlib==0.6.1
six==1.10.0
unicode-slugify==0.1.1
urllib3==1.15.1

http://comisionsexta.jhonfelipeurregomejia.com/, error persist

With sayt app (no multi instance) ubuntu 64 bits is good:

(virtualenv-sayit) felipe@ACTASINTELIGENTES:~/sayit/sayit$ ./manage.py load_akomantoso --dir=/home/felipe/comisionsextasenado/xml --instance=default --commit --merge-existing --verbosity=2
Starting import: /home/felipe/comisionsextasenado/xml/acta-01-12-julio-25-de-2012.xml

Importing Comisión Sexta Senado
Importing 2012
Importing Julio
Importing ACTA No. 01 / 25-07-2012
/home/felipe/comisionsextasenado/xml/acta-01-12-julio-25-de-2012.xml: Imported Speaker:105 Speech:57 Section:4

Starting import: /home/felipe/comisionsextasenado/xml/acta-02-12-julio-31-de-2012-f.xml

Merging Comisión Sexta Senado
Merging 2012
Merging Julio
Importing ACTA No. 02 / 31-07-2012
/home/felipe/comisionsextasenado/xml/acta-02-12-julio-31-de-2012-f.xml: Imported Speaker:0 Speech:79 Section:1

Starting import: /home/felipe/comisionsextasenado/xml/acta-03-12-agosto-14-de-2012.xml

Merging Comisión Sexta Senado
Merging 2012
Importing Agosto
Importing ACTA No. 03 / 14-08-2012
/home/felipe/comisionsextasenado/xml/acta-03-12-agosto-14-de-2012.xml: Imported Speaker:0 Speech:48 Section:2

Starting import: /home/felipe/comisionsextasenado/xml/acta-04-12-agosto-15-de-2012.xml

Merging Comisión Sexta Senado
Merging 2012
Merging Agosto
Importing ACTA No. 04 / 15-08-2012
/home/felipe/comisionsextasenado/xml/acta-04-12-agosto-15-de-2012.xml: Imported Speaker:0 Speech:33 Section:1

Starting import: /home/felipe/comisionsextasenado/xml/acta-05-12-agosto-21-de-2012.xml

Merging Comisión Sexta Senado
Merging 2012
Merging Agosto
Importing ACTA No. 05 / 21-08-2012
/home/felipe/comisionsextasenado/xml/acta-05-12-agosto-21-de-2012.xml: Imported Speaker:0 Speech:62 Section:1

Starting import: /home/felipe/comisionsextasenado/xml/acta-06-12-agosto-22-de-2012.xml

Merging Comisión Sexta Senado
Merging 2012
Merging Agosto
Importing ACTA No. 06 / 22-08-2012
Importing CUESTIONARIO
/home/felipe/comisionsextasenado/xml/acta-06-12-agosto-22-de-2012.xml: Imported Speaker:0 Speech:85 Section:2

Starting import: /home/felipe/comisionsextasenado/xml/acta-07-12-agosto-28-de-2012.xml

Merging Comisión Sexta Senado
Merging 2012
Merging Agosto
Importing ACTA No. 07 / 28-08-2012
/home/felipe/comisionsextasenado/xml/acta-07-12-agosto-28-de-2012.xml: Imported Speaker:0 Speech:63 Section:1

Starting import: /home/felipe/comisionsextasenado/xml/acta-08-12-septiembre-4-de-2012.xml

Merging Comisión Sexta Senado
Merging 2012
Importing Septiembre
Importing ACTA No. 08 / 04-09-2012
/home/felipe/comisionsextasenado/xml/acta-08-12-septiembre-4-de-2012.xml: Imported Speaker:0 Speech:10 Section:2

Starting import: /home/felipe/comisionsextasenado/xml/acta-09-12-septiembre-11-de-2012.xml

Merging Comisión Sexta Senado
Merging 2012
Merging Septiembre
Importing ACTA No. 09 / 11-09-2012
Importing CUESTIONARIO
/home/felipe/comisionsextasenado/xml/acta-09-12-septiembre-11-de-2012.xml: Imported Speaker:0 Speech:153 Section:2

Starting import: /home/felipe/comisionsextasenado/xml/acta-10-12-septiembre-18-de-2012.xml

Merging Comisión Sexta Senado
Merging 2012
Merging Septiembre
Importing ACTA No. 10 / 18-09-2012
Importing CUESTIONARIO
/home/felipe/comisionsextasenado/xml/acta-10-12-septiembre-18-de-2012.xml: Imported Speaker:0 Speech:35 Section:2

Starting import: /home/felipe/comisionsextasenado/xml/acta-11-12-septiembre-25-de-2012.xml

Merging Comisión Sexta Senado
Merging 2012
Merging Septiembre
Importing ACTA No. 11 / 25-09-2012
Importing CUESTIONARIO
/home/felipe/comisionsextasenado/xml/acta-11-12-septiembre-25-de-2012.xml: Imported Speaker:0 Speech:82 Section:2

Starting import: /home/felipe/comisionsextasenado/xml/acta-12-12-septiembre-26-de-2012.xml

Merging Comisión Sexta Senado
Merging 2012
Merging Septiembre
Importing ACTA No. 12 / 26-09-2012
Importing CUESTIONARIO
/home/felipe/comisionsextasenado/xml/acta-12-12-septiembre-26-de-2012.xml: Imported Speaker:0 Speech:77 Section:2

Starting import: /home/felipe/comisionsextasenado/xml/acta-13-12-octubre-2-de-2012.xml

Merging Comisión Sexta Senado
Merging 2012
Importing Octubre
Importing ACTA No. 13 / 02-10-2012
Importing CUESTIONARIO
/home/felipe/comisionsextasenado/xml/acta-13-12-octubre-2-de-2012.xml: Imported Speaker:0 Speech:103 Section:3

Starting import: /home/felipe/comisionsextasenado/xml/acta-14-12-octubre-9-de-2012.xml

Merging Comisión Sexta Senado
Merging 2012
Merging Octubre
Importing ACTA No. 14 / 09-10-2012
Importing CUESTIONARIO
/home/felipe/comisionsextasenado/xml/acta-14-12-octubre-9-de-2012.xml: Imported Speaker:0 Speech:32 Section:2

Starting import: /home/felipe/comisionsextasenado/xml/acta-15-12-octubre-16-de-2012.xml

Merging Comisión Sexta Senado
Merging 2012
Merging Octubre
Importing ACTA No. 15 / 16-10-2012
Importing CUESTIONARIO
/home/felipe/comisionsextasenado/xml/acta-15-12-octubre-16-de-2012.xml: Imported Speaker:0 Speech:93 Section:2

Starting import: /home/felipe/comisionsextasenado/xml/acta-16-12-octubre-24-de-2012.xml

Merging Comisión Sexta Senado
Merging 2012
Merging Octubre
Importing ACTA No.  / 24-10-2012
Importing CUESTIONARIO
/home/felipe/comisionsextasenado/xml/acta-16-12-octubre-24-de-2012.xml: Imported Speaker:0 Speech:79 Section:2

Starting import: /home/felipe/comisionsextasenado/xml/acta-17-12-octubre-30-de-2012.xml

Merging Comisión Sexta Senado
Merging 2012
Merging Octubre
Importing ACTA No. 17 / 30-10-2012
Importing CUESTIONARIO
/home/felipe/comisionsextasenado/xml/acta-17-12-octubre-30-de-2012.xml: Imported Speaker:0 Speech:78 Section:2

Starting import: /home/felipe/comisionsextasenado/xml/acta-18-12-noviembre-7-de-2012.xml

Merging Comisión Sexta Senado
Merging 2012
Importing Noviembre
Importing ACTA No. 18 / 07-11-2012
Importing CUESTIONARIO
/home/felipe/comisionsextasenado/xml/acta-18-12-noviembre-7-de-2012.xml: Imported Speaker:0 Speech:66 Section:3

Starting import: /home/felipe/comisionsextasenado/xml/acta-19-12-noviembre-14-de-2012.xml

Merging Comisión Sexta Senado
Merging 2012
Merging Noviembre
Importing ACTA No. 19 / 14-11-2012
Importing CUESTIONARIO
/home/felipe/comisionsextasenado/xml/acta-19-12-noviembre-14-de-2012.xml: Imported Speaker:0 Speech:48 Section:2

Starting import: /home/felipe/comisionsextasenado/xml/acta-20-12-noviembre-20-de-2012.xml

Merging Comisión Sexta Senado
Merging 2012
Merging Noviembre
Importing ACTA No. 20 / 20-11-2012
Importing CUESTIONARIO
/home/felipe/comisionsextasenado/xml/acta-20-12-noviembre-20-de-2012.xml: Imported Speaker:0 Speech:126 Section:2

Starting import: /home/felipe/comisionsextasenado/xml/acta-21-de-noviembre-de-2012.xml

Merging Comisión Sexta Senado
Merging 2012
Merging Noviembre
Importing ACTA No. 21 / 21-11-2012
Importing CUESTIONARIO
/home/felipe/comisionsextasenado/xml/acta-21-de-noviembre-de-2012.xml: Imported Speaker:0 Speech:89 Section:2

Starting import: /home/felipe/comisionsextasenado/xml/acta-22-12-noviembre-27-de-2012.xml

Merging Comisión Sexta Senado
Merging 2012
Merging Noviembre
Importing ACTA No. 22 / 27-11-2012
Importing CUESTIONARIO
/home/felipe/comisionsextasenado/xml/acta-22-12-noviembre-27-de-2012.xml: Imported Speaker:0 Speech:83 Section:2

Starting import: /home/felipe/comisionsextasenado/xml/acta-23-12-noviembre-28-de-2012.xml

Merging Comisión Sexta Senado
Merging 2012
Merging Noviembre
Importing ACTA No. 23 / 28-11-2012
Importing CUESTIONARIO
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
ID #h-s-carlos-ferro-solanilla in speech has no corresponding TLCPerson entry
/home/felipe/comisionsextasenado/xml/acta-23-12-noviembre-28-de-2012.xml: Imported Speaker:0 Speech:61 Section:2

Starting import: /home/felipe/comisionsextasenado/xml/acta-24-12-diciembre-4-de-2012-1.xml

Merging Comisión Sexta Senado
Merging 2012
Importing Diciembre
Importing ACTA No. 24 / 04-12-2012
/home/felipe/comisionsextasenado/xml/acta-24-12-diciembre-4-de-2012-1.xml: Imported Speaker:0 Speech:99 Section:2

Starting import: /home/felipe/comisionsextasenado/xml/acta-26-12-diciembre-12-de-20121.xml

Merging Comisión Sexta Senado
Merging 2012
Merging Diciembre
Importing ACTA No. 26 / 12-12-2012
/home/felipe/comisionsextasenado/xml/acta-26-12-diciembre-12-de-20121.xml: Imported Speaker:0 Speech:45 Section:1

Starting import: /home/felipe/comisionsextasenado/xml/acta-27-13-febrero-19-de-2013.xml

Merging Comisión Sexta Senado
Importing 2013
Importing Febrero
Importing ACTA No. 27 / 19-02-2013
Importing CUESTIONARIO
/home/felipe/comisionsextasenado/xml/acta-27-13-febrero-19-de-2013.xml: Imported Speaker:0 Speech:84 Section:4

Starting import: /home/felipe/comisionsextasenado/xml/acta-28-13-febrero-20-de-2013.xml

Merging Comisión Sexta Senado
Merging 2013
Merging Febrero
Importing ACTA No. 28 / 20-02-2013
Importing CUESTIONARIO
/home/felipe/comisionsextasenado/xml/acta-28-13-febrero-20-de-2013.xml: Imported Speaker:0 Speech:77 Section:2

sayitnormal

It works with 64 bits but only instance

I need to have in ubuntu 32 bits sayit.mysociety.org, multi instance, pls help me

dracos commented 8 years ago

The fact your instance you link to has "Dr. Diego Molano Vega" in it when the first person that should be imported (and erroring) from your directory import is "H. S. Carlos Ferro Solanilla" implies that either your database is not empty when you run the importer, or you are running something else first. As you have still not provided a complete log of your actions (you stop after "pip freeze" when it is vitally important to know precisely what you did next!), it's impossible to say. Provide a full log of everything you enter and the output from the flush to the end of the errors. Do not miss anything out or cut anything. As I have already shown, it works fine locally on an empty database.

johnfelipe commented 8 years ago

Where can i find full log?, i send u only log show in command prompt of putty

dracos commented 8 years ago

I googled putty logging and look what I found: http://www.viktorious.nl/2013/01/14/putty-log-all-session-output/ – that should let you log all output to a file which you can then put on pastebin or similar.

johnfelipe commented 8 years ago

Hi, this is putty log

https://drive.google.com/open?id=0B5b2y2kCFpKoSnhVUnYxLS05UDA

I hope can u help me, for have sayit.mysociety.org correctly functional in ubuntu 32 bits

dracos commented 8 years ago

Finally you have quoted the entire error log, and as I already said, the first error is different from the others (which is all you have quoted before now), and that is the actual issue, not any error you get after it. If you had quoted everything at the start, we could have saved a lot of time! The first error in your log is:

An exception of type ConnectionError occurred, arguments:
ConnectionError(<urllib3.connection.HTTPConnection object at 0xb526d98c>: Failed to establish a new connection: [Errno 111] Connection refused) caused by:

Which means that your elasticsearch program is not running! Did you start the elasticsearch program, or just install it? https://www.elastic.co/guide/en/elasticsearch/reference/1.7/setup-repositories.html says it does not start automatically unless you run some extra configuration...

johnfelipe commented 8 years ago 
felipe@felipeurrego:~$ sudo /etc/init.d/elasticsearch start
[sudo] password for felipe:
 * Starting Elasticsearch Server                                                 * Already running.                                                      [ OK ]
felipe@felipeurrego:~$ cd sayit/
felipe@felipeurrego:~/sayit$ source virtualenv-sayit/bin/activate
(virtualenv-sayit) felipe@felipeurrego:~/sayit$ cd sayit.mysociety.org/
(virtualenv-sayit) felipe@felipeurrego:~/sayit/sayit.mysociety.org$ ./manage.py load_akomantoso --dir=/home/felipe/comisionsexta/xml --instance=comisionsextasenado --commit  --verbosity=2 --merge-existing
Starting import: /home/felipe/comisionsexta/xml/acta-01-12-julio-25-de-2012.xml

An exception of type IntegrityError occurred, arguments:
duplicate key value violates unique constraint "speeches_speaker_instance_id_6538f9cb_uniq"
DETAIL:  Key (instance_id, slug)=(3, h-s-carlos-ferro-solanilla) already exists.

Traceback (most recent call last):
  File "/home/felipe/sayit/virtualenv-sayit/src/django-sayit/speeches/management/import_commands.py", line 124, in import_document
    importer.import_document(path)
  File "/home/felipe/sayit/virtualenv-sayit/src/django-sayit/speeches/importers/import_akomantoso.py", line 25, in import_document
    return self.parse_document()
  File "/home/felipe/sayit/virtualenv-sayit/src/django-sayit/speeches/importers/import_akomantoso.py", line 54, in parse_document
    speaker.save()
  File "/home/felipe/sayit/virtualenv-sayit/src/django-sayit/speeches/models.py", line 163, in save
    return super(Speaker, self).save(*args, **kwargs)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/models/base.py", line 734, in save
    force_update=force_update, update_fields=update_fields)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/models/base.py", line 762, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/models/base.py", line 846, in _save_table
    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/models/base.py", line 885, in _do_insert
    using=using, raw=raw)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/models/manager.py", line 127, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/models/query.py", line 920, in _insert
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 974, in execute_sql
    cursor.execute(sql, params)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/utils.py", line 98, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/home/felipe/sayit/virtualenv-sayit/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
IntegrityError: duplicate key value violates unique constraint "speeches_speaker_instance_id_6538f9cb_uniq"
DETAIL:  Key (instance_id, slug)=(3, h-s-carlos-ferro-solanilla) already exists.
dracos commented 8 years ago

You didn't empty the database. Regardless, if it says "already running" but you got a connection refused error, then something is up with your elasticsearch installation, I cannot help you with that.

johnfelipe commented 8 years ago

It works change elasticsearch to elasticsearch-2.3.2.deb, dont work with 1.7 in ubuntu 32 bits, pls ensure put in readme TNKS