datamade / nyc-council-councilmatic

NYC Council version of Councilmatic
MIT License
7 stars 3 forks source link

DataError: value too long for type character varying(255) #160

Open sentry-io[bot] opened 4 years ago

sentry-io[bot] commented 4 years ago

Sentry Issue: NYC-COUNCIL-COUNCILMATIC-2M

DataError: (psycopg2.DataError) value too long for type character varying(255)
 [SQL: '\n            INSERT INTO raw_bill (\n                ocd_id,\n                ocd_created_at,\n                ocd_updated_at,\n                description,\n                identifier,\n                classification,\n                source_url,\n                source_note,\n                from_organization_id,\n                full_text,\n                ocr_full_text,\n                html_text,\n                abstract,\n                legislative_session_id,\n                bill_type,\n                slug\n            ) VALUES (\n                %(ocd_id)s,\n                %(ocd_created_at)s,\n                %(ocd_updated_at)s,\n                %(description)s,\n                %(identifier)s,\n                %(classification)s,\n                %(source_url)s,\n                %(source_note)s,\n                %(from_organization_id)s,\n                %(full_text)s,\n                %(ocr_full_text)s,\n                %(html_text)s,\n                %(abstract)s,\n                %(legislative_session_id)s,\n                %(bill_type)s,\n                %(slug)s\n            )\n            '] [parameters: ({'full_text': 'L.U. 527\r\nBy Council Member Eisland:\r\nApplication no. 20005037 HAK, an Urban Development Action Area Project, located at 406 Utica Avenue, Counci ... (136 characters truncated) ... ral Municipal Law, at the request of the New York City Department of Housing Preservation and Development.\r\nPlanning, Dispositions  and Concessions', 'ocr_full_text': 'L.U. 527\r\nBy Council Member Eisland:\r\nApplication no. 20005037 HAK, an Urban Development Action Area Project, located at 406 Utica Avenue, Counci ... (136 characters truncated) ... ral Municipal Law, at the request of the New York City Department of Housing Preservation and Development.\r\nPlanning, Dispositions  and Concessions', 'ocd_updated_at': '2019-07-10T09:19:20.566894+00:00', 'ocd_created_at': '2017-12-02T12:51:24.572085+00:00', 'bill_type': 'Land Use Application', 'from_organization_id': 'ocd-organization/0f63aae8-16fd-4d3c-b525-00747a482cf9', 'source_note': 'api', 'ocd_id': 'ocd-bill/92adf63e-7172-47cc-ada6-d73baca4d6e2', 'identifier': 'LU 0527-1999', 'html_text': None, 'classification': 'bill', 'legislative_session_id': '1998', 'slug': 'lu-0527-1999', 'source_url': 'https://legistar.council.nyc.gov/LegislationDetail.aspx?ID=433264&GUID=34A9C23D-AAE1-498A-A884-87A1BC14AF67', 'abstract': None, 'description': 'UDAAP, 406 Utica Avenue, Brooklyn (20005037HAK)'}, {'full_text': None, 'ocr_full_text': None, 'ocd_updated_at': '2019-08-28T11:50:53.185190+00:00', 'ocd_created_at': '2017-12-02T16:20:09.291492+00:00', 'bill_type': 'Communication', 'from_organization_id': 'ocd-organization/0f63aae8-16fd-4d3c-b525-00747a482cf9', 'source_note': 'api', 'ocd_id': 'ocd-bill/7cbb50cc-7c3c-4aa9-9fa1-61fe8c24c88d', 'identifier': 'M 0194-2002', 'html_text': None, 'classification': 'bill', 'legislative_session_id': '2002', 'slug': 'm-0194-2002', 'source_url': 'https://legistar.council.nyc.gov/LegislationDetail.aspx?ID=438131&GUID=34118AD2-C259-4ED0-96DA-9D90C5C85173', 'abstract': None, 'description': 'Base Station License, Caprice Car Service Corp.'}, {'full_text': None, 'ocr_full_text': None, 'ocd_updated_at': '2019-07-02T15:21:20.074571+00:00', 'ocd_created_at': '2017-12-02T12:51:23.381094+00:00', 'bill_type': "Mayor's Message", 'from_organization_id': 'ocd-organization/0f63aae8-16fd-4d3c-b525-00747a482cf9', 'source_note': 'api', 'ocd_id': 'ocd-bill/15fbe6ca-92ff-4811-bac7-ed37395cab8f', 'identifier': 'M 0008-1998', 'html_text': None, 'classification': 'bill', 'legislative_session_id': '1998', 'slug': 'm-0008-1998', 'source_url': 'https://legistar.council.nyc.gov/LegislationDetail.aspx?ID=430188&GUID=329E06CE-F3D3-423B-AD95-149D904E5C25', 'abstract': None, 'description': 'Committee Membership and Chairs'}, {'full_text': '\r\n\r\n\r\n\r\n\r\n\t\t              \t\tState Legislation Resolution No. 17\r\n\r\n\r\n\r\n..Title\r\nState Legislation Resolution requesting the N ... (1586 characters truncated) ... to enact into law the aforesaid pending bill.\r\n\r\n\tReferred to the Committee on State and Federal Legislation.\r\n\r\n \r\n\r\n\t\r\n\r\n\r\n\r\n', 'ocr_full_text': '\r\n\r\n\r\n\r\n\r\n\t\t              \t\tState Legislation Resolution No. 17\r\n\r\n\r\n\r\n..Title\r\nState Legislation Resolution requesting the N ... (1586 characters truncated) ... to enact into law the aforesaid pending bill.\r\n\r\n\tReferred to the Committee on State and Federal Legislation.\r\n\r\n \r\n\r\n\t\r\n\r\n\r\n\r\n', 'ocd_updated_at': '2019-09-04T11:16:50.053048+00:00', 'ocd_created_at': '2017-12-02T16:18:40.642766+00:00', 'bill_type': 'SLR', 'from_organization_id': 'ocd-organization/0f63aae8-16fd-4d3c-b525-00747a482cf9', 'source_note': 'api', 'ocd_id': 'ocd-bill/b5c51543-72b1-414a-94ea-9dab3f693587', 'identifier': 'SLR 0017-2002', 'html_text': None, 'classification': 'bill', 'legislative_session_id': '2002', 'slug': 'slr-0017-2002', 'source_url': 'https://legistar.council.nyc.gov/LegislationDetail.aspx?ID=437201&GUID=F9699273-6797-4C32-B1A2-876D3F4FDD7A', 'abstract': None, 'description': 'In rem, Mount Calvary United American Free Will Baptist Church, Brooklyn (A.3850)'}, {'full_text': None, 'ocr_full_text': None, 'ocd_updated_at': '2019-08-22T14:30:33.204502+00:00', 'ocd_created_at': '2017-12-02T16:20:20.872330+00:00', 'bill_type': 'Land Use Application', 'from_organization_id': 'ocd-organization/0f63aae8-16fd-4d3c-b525-00747a482cf9', 'source_note': 'api', 'ocd_id': 'ocd-bill/920a94d7-0a50-4264-b6be-29fa0e8d266a', 'identifier': 'LU 0257-2002', 'html_text': None, 'classification': 'bill', 'legislative_session_id': '2002', 'slug': 'lu-0257-2002', 'source_url': 'https://legistar.council.nyc.gov/LegislationDetail.aspx?ID=438504&GUID=169530AB-9AC4-4CE8-A4D0-4846B8134F66', 'abstract': None, 'description': 'UDAAP, In Rem Action #44, Brooklyn (20035007HAK)'}, {'full_text': None, 'ocr_full_text': None, 'ocd_updated_at': '2019-09-04T11:16:50.787055+00:00', 'ocd_created_at': '2017-12-02T16:19:25.557340+00:00', 'bill_type': 'Communication', 'from_organization_id': 'ocd-organization/0f63aae8-16fd-4d3c-b525-00747a482cf9', 'source_note': 'api', 'ocd_id': 'ocd-bill/6037909f-0739-41ee-862f-99bd96306b00', 'identifier': 'M 0763-2003', 'html_text': None, 'classification': 'bill', 'legislative_session_id': '2002', 'slug': 'm-0763-2003', 'source_url': 'https://legistar.council.nyc.gov/LegislationDetail.aspx?ID=440642&GUID=B5E2FFA5-0EB3-4232-B4A3-A52881C7E992', 'abstract': None, 'description': 'Base Station License, New Jerusalem Car & Limo Service II'}, {'full_text': "{\\rtf1\\fbidis\\ansi\\ansicpg1252\\deff0\\deflang1033\\deflangfe1033{\\fonttbl{\\f0\\froman\\fprq2\\fcharset0 Times New Roman;}}\r\n\\viewkind4\\uc1 ... (853 characters truncated) ... not be changed by local law.  \\par\r\n\\pard\\ltrpar\\sl480\\slmult1\\qj\\par\r\n\\pard\\ltrpar\\qj\\ulnone LS # 4659\\par\r\n12/14/01\\par\r\n}\r\n", 'ocr_full_text': "Int. No. 1020\r\n\r\nBy Council Member Pinkett; also Council Member O'Donovan\r\n\r\n..Title\r\nA Local Law to amend the administrative code of the c ... (413 characters truncated) ... visions of section 25-303 of the administrative code, shall not be changed by local law.  \r\n\r\nLS # 4659\r\n12/14/01\r\n1\r\n\r\n\r\n1\r\n\r\n\r\n", 'ocd_updated_at': '2019-06-27T12:45:42.418918+00:00', 'ocd_created_at': '2017-12-02T20:19:35.675481+00:00', 'bill_type': 'Introduction', 'from_organization_id': 'ocd-organization/0f63aae8-16fd-4d3c-b525-00747a482cf9', 'source_note': 'api', 'ocd_id': 'ocd-bill/88ad8c86-875c-4d97-a47c-b87d8572c7aa', 'identifier': 'Int 1020-2001', 'html_text': None, 'classification': 'bill', 'legislative_session_id': '1998', 'slug': 'int-1020-2001', 'source_url': 'https://legistar.council.nyc.gov/LegislationDetail.aspx?ID=436893&GUID=E3C76959-64DC-424E-B401-DADCB4C646B8', 'abstract': None, 'description': 'Changing the names of designated landmarks.'}, {'full_text': '\r\n\r\n\tTHE COUNCIL OF THE CITY OF NEW YORK\r\n\tRESOLUTION NO. 507\r\n\r\n..Title\r\nResolution approving the designation by th...
(8 additional frame(s) were not displayed)
...
  File "councilmatic_core/management/commands/import_data.py", line 156, in handle
    delete=options['delete'])
  File "councilmatic_core/management/commands/import_data.py", line 274, in bills_etl
    self.insert_raw_bills(delete=delete)
  File "councilmatic_core/management/commands/import_data.py", line 1019, in insert_raw_bills
    self.executeTransaction(sa.text(insert_query), *inserts)
  File "councilmatic_core/management/commands/import_data.py", line 3318, in executeTransaction
    self.connection.execute(query, *args)

DataError: value too long for type character varying(255)

  File "sqlalchemy/engine/base.py", line 1159, in _execute_context
    context)
  File "sqlalchemy/engine/default.py", line 467, in do_executemany
    cursor.executemany(statement, parameters)

DataError: (psycopg2.DataError) value too long for type character varying(255)
 [SQL: '\n            INSERT INTO raw_bill (\n                ocd_id,\n                ocd_created_at,\n                ocd_updated_at,\n                description,\n                identifier,\n                classification,\n                source_url,\n                source_note,\n                from_organization_id,\n                full_text,\n                ocr_full_text,\n                html_text,\n                abstract,\n                legislative_session_id,\n                bill_type,\n                slug\n            ) VALUES (\n                %(ocd_id)s,\n                %(ocd_created_at)s,\n                %(ocd_updated_at)s,\n                %(description)s,\n                %(identifier)s,\n                %(classification)s,\n                %(source_url)s,\n                %(source_note)s,\n                %(from_organization_id)s,\n                %(full_text)s,\n                %(ocr_full_text)s,\n                %(html_text)s,\n                %(abstract)s,\n                %(legislative_session_id)s,\n                %(bill_type)s,\n                %(slug)s\n            )\n            '] [parameters: ({'full_text': 'L.U. 527\r\nBy Council Member Eisland:\r\nApplication no. 20005037 HAK, an Urban Development Action Area Project, located at 406 Utica Avenue, Counci ... (136 characters truncated) ... ral Municipal Law, at the request of the New York City Department of Housing Preservation and Development.\r\nPlanning, Dispositions  and Concessions', 'ocr_full_text': 'L.U. 527\r\nBy Council Member Eisland:\r\nApplication no. 20005037 HAK, an Urban Development Action Area Project, located at 406 Utica Avenue, Counci ... (136 characters truncated) ... ral Municipal Law, at the request of the New York City Department of Housing Preservation and Development.\r\nPlanning, Dispositions  and Concessions', 'ocd_updated_at': '2019-07-10T09:19:20.566894+00:00', 'ocd_created_at': '2017-12-02T12:51:24.572085+00:00', 'bill_type': 'Land Use Application', 'from_organization_id': 'ocd-organization/0f63aae8-16fd-4d3c-b525-00747a482cf9', 'source_note': 'api', 'ocd_id': 'ocd-bill/92adf63e-7172-47cc-ada6-d73baca4d6e2', 'identifier': 'LU 0527-1999', 'html_text': None, 'classification': 'bill', 'legislative_session_id': '1998', 'slug': 'lu-0527-1999', 'source_url': 'https://legistar.council.nyc.gov/LegislationDetail.aspx?ID=433264&GUID=34A9C23D-AAE1-498A-A884-87A1BC14AF67', 'abstract': None, 'description': 'UDAAP, 406 Utica Avenue, Brooklyn (20005037HAK)'}, {'full_text': None, 'ocr_full_text': None, 'ocd_updated_at': '2019-08-28T11:50:53.185190+00:00', 'ocd_created_at': '2017-12-02T16:20:09.291492+00:00', 'bill_type': 'Communication', 'from_organization_id': 'ocd-organization/0f63aae8-16fd-4d3c-b525-00747a482cf9', 'source_note': 'api', 'ocd_id': 'ocd-bill/7cbb50cc-7c3c-4aa9-9fa1-61fe8c24c88d', 'identifier': 'M 0194-2002', 'html_text': None, 'classification': 'bill', 'legislative_session_id': '2002', 'slug': 'm-0194-2002', 'source_url': 'https://legistar.council.nyc.gov/LegislationDetail.aspx?ID=438131&GUID=34118AD2-C259-4ED0-96DA-9D90C5C85173', 'abstract': None, 'description': 'Base Station License, Caprice Car Service Corp.'}, {'full_text': None, 'ocr_full_text': None, 'ocd_updated_at': '2019-07-02T15:21:20.074571+00:00', 'ocd_created_at': '2017-12-02T12:51:23.381094+00:00', 'bill_type': "Mayor's Message", 'from_organization_id': 'ocd-organization/0f63aae8-16fd-4d3c-b525-00747a482cf9', 'source_note': 'api', 'ocd_id': 'ocd-bill/15fbe6ca-92ff-4811-bac7-ed37395cab8f', 'identifier': 'M 0008-1998', 'html_text': None, 'classification': 'bill', 'legislative_session_id': '1998', 'slug': 'm-0008-1998', 'source_url': 'https://legistar.council.nyc.gov/LegislationDetail.aspx?ID=430188&GUID=329E06CE-F3D3-423B-AD95-149D904E5C25', 'abstract': None, 'description': 'Committee Membership and Chairs'}, {'full_text': '\r\n\r\n\r\n\r\n\r\n\t\t              \t\tState Legislation Resolution No. 17\r\n\r\n\r\n\r\n..Title\r\nState Legislation Resolution requesting the N ... (1586 characters truncated) ... to enact into law the aforesaid pending bill.\r\n\r\n\tReferred to the Committee on State and Federal Legislation.\r\n\r\n \r\n\r\n\t\r\n\r\n\r\n\r\n', 'ocr_full_text': '\r\n\r\n\r\n\r\n\r\n\t\t              \t\tState Legislation Resolution No. 17\r\n\r\n\r\n\r\n..Title\r\nState Legislation Resolution requesting the N ... (1586 characters truncated) ... to enact into law the aforesaid pending bill.\r\n\r\n\tReferred to the Committee on State and Federal Legislation.\r\n\r\n \r\n\r\n\t\r\n\r\n\r\n\r\n', 'ocd_updated_at': '2019-09-04T11:16:50.053048+00:00', 'ocd_created_at': '2017-12-02T16:18:40.642766+00:00', 'bill_type': 'SLR', 'from_organization_id': 'ocd-organization/0f63aae8-16fd-4d3c-b525-00747a482cf9', 'source_note': 'api', 'ocd_id': 'ocd-bill/b5c51543-72b1-414a-94ea-9dab3f693587', 'identifier': 'SLR 0017-2002', 'html_text': None, 'classification': 'bill', 'legislative_session_id': '2002', 'slug': 'slr-0017-2002', 'source_url': 'https://legistar.council.nyc.gov/LegislationDetail.aspx?ID=437201&GUID=F9699273-6797-4C32-B1A2-876D3F4FDD7A', 'abstract': None, 'description': 'In rem, Mount Calvary United American Free Will Baptist Church, Brooklyn (A.3850)'}, {'full_text': None, 'ocr_full_text': None, 'ocd_updated_at': '2019-08-22T14:30:33.204502+00:00', 'ocd_created_at': '2017-12-02T16:20:20.872330+00:00', 'bill_type': 'Land Use Application', 'from_organization_id': 'ocd-organization/0f63aae8-16fd-4d3c-b525-00747a482cf9', 'source_note': 'api', 'ocd_id': 'ocd-bill/920a94d7-0a50-4264-b6be-29fa0e8d266a', 'identifier': 'LU 0257-2002', 'html_text': None, 'classification': 'bill', 'legislative_session_id': '2002', 'slug': 'lu-0257-2002', 'source_url': 'https://legistar.council.nyc.gov/LegislationDetail.aspx?ID=438504&GUID=169530AB-9AC4-4CE8-A4D0-4846B8134F66', 'abstract': None, 'description': 'UDAAP, In Rem Action #44, Brooklyn (20035007HAK)'}, {'full_text': None, 'ocr_full_text': None, 'ocd_updated_at': '2019-09-04T11:16:50.787055+00:00', 'ocd_created_at': '2017-12-02T16:19:25.557340+00:00', 'bill_type': 'Communication', 'from_organization_id': 'ocd-organization/0f63aae8-16fd-4d3c-b525-00747a482cf9', 'source_note': 'api', 'ocd_id': 'ocd-bill/6037909f-0739-41ee-862f-99bd96306b00', 'identifier': 'M 0763-2003', 'html_text': None, 'classification': 'bill', 'legislative_session_id': '2002', 'slug': 'm-0763-2003', 'source_url': 'https://legistar.council.nyc.gov/LegislationDetail.aspx?ID=440642&GUID=B5E2FFA5-0EB3-4232-B4A3-A52881C7E992', 'abstract': None, 'description': 'Base Station License, New Jerusalem Car & Limo Service II'}, {'full_text': "{\\rtf1\\fbidis\\ansi\\ansicpg1252\\deff0\\deflang1033\\deflangfe1033{\\fonttbl{\\f0\\froman\\fprq2\\fcharset0 Times New Roman;}}\r\n\\viewkind4\\uc1 ... (853 characters truncated) ... not be changed by local law.  \\par\r\n\\pard\\ltrpar\\sl480\\slmult1\\qj\\par\r\n\\pard\\ltrpar\\qj\\ulnone LS # 4659\\par\r\n12/14/01\\par\r\n}\r\n", 'ocr_full_text': "Int. No. 1020\r\n\r\nBy Council Member Pinkett; also Council Member O'Donovan\r\n\r\n..Title\r\nA Local Law to amend the administrative code of the c ... (413 characters truncated) ... visions of section 25-303 of the administrative code, shall not be changed by local law.  \r\n\r\nLS # 4659\r\n12/14/01\r\n1\r\n\r\n\r\n1\r\n\r\n\r\n", 'ocd_updated_at': '2019-06-27T12:45:42.418918+00:00', 'ocd_created_at': '2017-12-02T20:19:35.675481+00:00', 'bill_type': 'Introduction', 'from_organization_id': 'ocd-organization/0f63aae8-16fd-4d3c-b525-00747a482cf9', 'source_note': 'api', 'ocd_id': 'ocd-bill/88ad8c86-875c-4d97-a47c-b87d8572c7aa', 'identifier': 'Int 1020-2001', 'html_text': None, 'classification': 'bill', 'legislative_session_id': '1998', 'slug': 'int-1020-2001', 'source_url': 'https://legistar.council.nyc.gov/LegislationDetail.aspx?ID=436893&GUID=E3C76959-64DC-424E-B401-DADCB4C646B8', 'abstract': None, 'description': 'Changing the names of designated landmarks.'}, {'full_text': '\r\n\r\n\tTHE COUNCIL OF THE CITY OF NEW YORK\r\n\tRESOLUTION NO. 507\r\n\r\n..Title\r\nResolution approving the design...