django.db.utils.DataError on load of Cvr2SoCd

[hcharley]

During a run of downloadcalaccessrawdata in the load step, I encountered this database error. Did I muck something up on my database setup or is it on the calaccess side?

Loading data files
 Loading CvrSoCd
 Loading Cvr2SoCd
Traceback (most recent call last):
  File "manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
    utility.execute()
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/__init__.py", line 377, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/base.py", line 288, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/base.py", line 338, in execute
    output = self.handle(*args, **options)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/calaccess_raw/management/commands/downloadcalaccessrawdata.py", line 121, in handle
    self.load()
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/calaccess_raw/management/commands/downloadcalaccessrawdata.py", line 251, in load
    call_command("loadcalaccessrawfile", model.__name__)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/__init__.py", line 115, in call_command
    return klass.execute(*args, **defaults)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/base.py", line 338, in execute
    output = self.handle(*args, **options)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/base.py", line 503, in handle
    label_output = self.handle_label(label, **options)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/calaccess_raw/management/commands/loadcalaccessrawfile.py", line 22, in handle_label
    self.load(label)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/calaccess_raw/management/commands/loadcalaccessrawfile.py", line 41, in load
    self.load_postgresql(model, csv_path)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/calaccess_raw/management/commands/loadcalaccessrawfile.py", line 160, in load_postgresql
    self.cursor.execute(temp_insert)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/db/backends/utils.py", line 81, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/db/backends/utils.py", line 63, in execute
    return self.cursor.execute(sql)
django.db.utils.DataError: value too long for type character varying(19)
CONTEXT:  COPY temporary_table, line 37468, column TRAN_ID: "9A0E4A6D-5C20-49AE-B"

I reproduced by calling loadcalaccessrawfile directly:

10:11:21 $ python manage.py loadcalaccessrawfile Cvr2SoCd
 Loading Cvr2SoCd
Traceback (most recent call last):
  File "manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
    utility.execute()
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/__init__.py", line 377, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/base.py", line 288, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/base.py", line 338, in execute
    output = self.handle(*args, **options)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/base.py", line 503, in handle
    label_output = self.handle_label(label, **options)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/calaccess_raw/management/commands/loadcalaccessrawfile.py", line 22, in handle_label
    self.load(label)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/calaccess_raw/management/commands/loadcalaccessrawfile.py", line 41, in load
    self.load_postgresql(model, csv_path)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/calaccess_raw/management/commands/loadcalaccessrawfile.py", line 160, in load_postgresql
    self.cursor.execute(temp_insert)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/db/backends/utils.py", line 81, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/db/backends/utils.py", line 63, in execute
    return self.cursor.execute(sql)
django.db.utils.DataError: value too long for type character varying(19)
CONTEXT:  COPY temporary_table, line 37468, column TRAN_ID: "9A0E4A6D-5C20-49AE-B"

[palewire]

Are you using the latest pypi release? If so what version?

On Sat, Dec 17, 2016, 10:22 AM Charley Bodkin notifications@github.com wrote:

During a run of downloadcalaccessrawdata in the load step, I encountered this database error. Did I muck something up on my database setup or is it on the calaccess side?

Loading data files Loading CvrSoCd Loading Cvr2SoCd Traceback (most recent call last): File "manage.py", line 22, in execute_from_command_line(sys.argv) File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/init.py", line 385, in execute_from_command_line utility.execute() File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/init.py", line 377, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/base.py", line 288, in run_from_argv self.execute(*args, options.dict) File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/base.py", line 338, in execute output = self.handle(*args, *options) File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/calaccess_raw/management/commands/downloadcalaccessrawdata.py", line 121, in handle self.load() File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/calaccess_raw/management/commands/downloadcalaccessrawdata.py", line 251, in load call_command("loadcalaccessrawfile", model.name) File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/init.py", line 115, in call_command return klass.execute(args, defaults) File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/base.py", line 338, in execute output = self.handle(*args, options) File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/base.py", line 503, in handle label_output = self.handle_label(label, options) File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/calaccess_raw/management/commands/loadcalaccessrawfile.py", line 22, in handle_label self.load(label) File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/calaccess_raw/management/commands/loadcalaccessrawfile.py", line 41, in load self.load_postgresql(model, csv_path) File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/calaccess_raw/management/commands/loadcalaccessrawfile.py", line 160, in load_postgresql self.cursor.execute(temp_insert) File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/db/backends/utils.py", line 81, in execute return super(CursorDebugWrapper, self).execute(sql, params) File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute return self.cursor.execute(sql, params) File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/db/utils.py", line 94, in exit six.reraise(dj_exc_type, dj_exc_value, traceback) File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/db/backends/utils.py", line 63, in execute return self.cursor.execute(sql) django.db.utils.DataError: value too long for type character varying(19) CONTEXT: COPY temporary_table, line 37468, column TRAN_ID: "9A0E4A6D-5C20-49AE-B"

I reproduced by calling loadcalaccessrawfile directly:

10:11:21 $ python manage.py loadcalaccessrawfile Cvr2SoCd Loading Cvr2SoCd Traceback (most recent call last): File "manage.py", line 22, in execute_from_command_line(sys.argv) File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/init.py", line 385, in execute_from_command_line utility.execute() File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/init.py", line 377, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/base.py", line 288, in run_from_argv self.execute(*args, *options.dict) File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/base.py", line 338, in execute output = self.handle(args, options) File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/base.py", line 503, in handle label_output = self.handle_label(label, options) File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/calaccess_raw/management/commands/loadcalaccessrawfile.py", line 22, in handle_label self.load(label) File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/calaccess_raw/management/commands/loadcalaccessrawfile.py", line 41, in load self.load_postgresql(model, csv_path) File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/calaccess_raw/management/commands/loadcalaccessrawfile.py", line 160, in load_postgresql self.cursor.execute(temp_insert) File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/db/backends/utils.py", line 81, in execute return super(CursorDebugWrapper, self).execute(sql, params) File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute return self.cursor.execute(sql, params) File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/db/utils.py", line 94, in exit six.reraise(dj_exc_type, dj_exc_value, traceback) File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/db/backends/utils.py", line 63, in execute return self.cursor.execute(sql) django.db.utils.DataError: value too long for type character varying(19) CONTEXT: COPY temporary_table, line 37468, column TRAN_ID: "9A0E4A6D-5C20-49AE-B"

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/california-civic-data-coalition/django-calaccess-raw-data/issues/1496, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAnCXm6hGjviXVm9-ZMHsulQ7fqN3buks5rJChjgaJpZM4LP8q_ .

[palewire]

I just ran the complete update process on the master branch of this repo and did not get an error. So there must be some difference between our two configurations. Are you using a PyPI release? If so, which version?

[hcharley]

I was using pypi release, but then I had switched to master branch because of a separate problem. I'm going to close until I can reproduce reliably using pypi version.

[palewire]

As a second measure, I just created a blank virtualenv and installed the latest PyPI release according to our official instructions. Everything loaded fine on a first run through updatecalaccessrawdata. So something else must be different between us. I am using PostgreSQL 9.5 on Ubuntu Linux 16.04 with django-calaccess-raw-data 1.4.6.

[hcharley]

Believe I just overcame this issue. Not sure exactly what was going on--probably on an out of sync tree or something screwy with my DB, but thank you @palewire for assisting.

On PostgreSQL 9.4.5 on Mac OS 10.11.6 (15G31)