Google Civic Information API choices not appearing in Scheduled Jobs

[joemcl]

I'm running Civi 5.18.3. I've installed the extension, added API keys for Google. I'm not seeing Google Civic Information API in Scheduled Jobs.

In #9, @josephlacey wrote this (and I've created a readme PR - #24 - with the full reply text) -

The second major step is to enable the Scheduled Jobs that will do the regular lookups. Again from the administration bar, Administer > System Settings > Scheduled Jobs. All the relevant jobs should be prefaced, 'Google Civic Information API'. Enable the ones relevant to your electoral level. Note that Google's Civic Information API has a free limit of 2500/day, about 100/hour. To get faster results, you'll need to pay for extra daily lookups.

I've also added the APIv4 extension. Could that be the cause - this extension needs an update for API v4?

[joemcl]

I've uninstalled, reinstalled and re-enabled from the command line, and getting the same result.

[joemcl]

This is the error message I got when running cv en -

Enabling extension "com.jlacey.electoral"
Error: API Call Failed: Array
(
    [entity] => Extension
    [action] => install
    [params] => Array
        (
            [keys] => Array
                (
                    [0] => com.jlacey.electoral
                )

            [debug] => 1
            [version] => 3
        )

    [result] => Array
        (
            [trace] => #0 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/CRM/Core/ManagedEntities.php(236): CRM_Core_ManagedEntities->onApiError('CustomGroup', 'create', Array, Array)
#1 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/CRM/Core/ManagedEntities.php(179): CRM_Core_ManagedEntities->insertNewEntity(Array)
#2 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/CRM/Core/ManagedEntities.php(141): CRM_Core_ManagedEntities->reconcileEnabledModule(Object(CRM_Core_Module), Array)
#3 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/CRM/Core/ManagedEntities.php(122): CRM_Core_ManagedEntities->reconcileEnabledModules()
#4 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/CRM/Core/Invoke.php(376): CRM_Core_ManagedEntities->reconcile()
#5 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/CRM/Extension/Manager.php(269): CRM_Core_Invoke::rebuildMenuAndCaches(true)
#6 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/api/v3/Extension.php(58): CRM_Extension_Manager->install(Array)
#7 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/Civi/API/Provider/MagicFunctionProvider.php(101): civicrm_api3_extension_install(Array)
#8 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/Civi/API/Kernel.php(168): Civi\API\Provider\MagicFunctionProvider->invoke(Array)
#9 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/Civi/API/Kernel.php(99): Civi\API\Kernel->runRequest(Array)
#10 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/api/api.php(23): Civi\API\Kernel->runSafe('Extension', 'install', Array, NULL)
#11 phar:///usr/local/bin/cv/src/Command/BaseCommand.php(49): civicrm_api('Extension', 'install', Array)
#12 phar:///usr/local/bin/cv/src/Command/ExtensionEnableCommand.php(91): Civi\Cv\Command\BaseCommand->callApiSuccess(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput), 'Extension', 'install', Array)
#13 phar:///usr/local/bin/cv/vendor/symfony/console/Command/Command.php(257): Civi\Cv\Command\ExtensionEnableCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#14 phar:///usr/local/bin/cv/vendor/symfony/console/Application.php(850): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#15 phar:///usr/local/bin/cv/vendor/symfony/console/Application.php(193): Symfony\Component\Console\Application->doRunCommand(Object(Civi\Cv\Command\ExtensionEnableCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#16 phar:///usr/local/bin/cv/src/Application.php(46): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#17 phar:///usr/local/bin/cv/vendor/symfony/console/Application.php(124): Civi\Cv\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#18 phar:///usr/local/bin/cv/src/Application.php(15): Symfony\Component\Console\Application->run()
#19 phar:///usr/local/bin/cv/bin/cv(26): Civi\Cv\Application::main('phar:///usr/loc...')
#20 /usr/local/bin/cv(10): require('phar:///usr/loc...')
#21 {main}
            [is_error] => 1
            [error_message] => API error: DB Error: already exists
        )

)

[joemcl]

also @MegaphoneJon any thoughts re the above?

[MegaphoneJon]

At this point you've got an issue due to uninstalling/reinstalling. It seems like something didn't uninstall completely. Delete any custom group fields associated with this extension then try again.

[joemcl]

@MegaphoneJon I think the above error is not caused by the Electoral API but CiviCase, discovered this since posting the error message above - https://lab.civicrm.org/dev/core/issues/329 . But yeah I'll uninstall any custom groups fields I might have added.

[MegaphoneJon]

Not custom group fields YOU added, custom group fields added by the extension(s).

[joemcl]

Yeah I didn't see a group of custom fields for Electoral API, looked. I disabled CiviCase, did cv ext:uninstall com.jlacey.electoral, rm -r com.jlacey.electoral. If you mean uninstall all custom group fields for all other extensions added - that's gonna be a big bummer.

[josephlacey]

@joemcl @MegaphoneJon I haven't configured the extension to delete the any custom fields on uninstall in case folks want to retain that data. Maybe that wasn't the best design choice, but the amount of data that some folks may have in these fields could be hours of (automated) work, so I left them.

The custom fields for this extension are installed via XML, which means that if they exist already, the install process should just skip over them and not be bothered by it. As far as I've tested it should be possible to uninstall the extension and reinstall it without getting this complaint.

But as always there could be bugs. If you can paste some more output @joemcl, probably from the CiviCRM log file, it might help clarify which custom group is being complained about here.

[joemcl]

Ah got it. Yeah so I should turn on debugging, open up the console, etc. I have some stuff in Drupal logs too I'll post.

[joemcl]

So first and as an FYI re potential conflicts - here's the Custom Field sets I do have installed right now...and I did have Jamie's Voter Fields extension installed, had added some more fields to that, I removed that set...

[joemcl]

and here's all the Extensions I have installed...

All extensions are up-to-date:
US County Loader (com.aghstrategies.uscounties) version 2.0
CiviMobileAPI (com.agiliway.civimobileapi) version 4.2
OSDI Contact Sync (com.example.osdi) version 1.0
Geodata Filler (com.ginkgostreet.geofill) version 1.0.0-alpha1
DonorSearch Integration (com.greenleafadvancement.donorsearch) version 1.1.5
Report Plus (com.ixiam.modules.reportplus) version 2.3.7
Eventbrite Integration (com.joineryhq.eventbrite) version 1.2
Joinery's More Summary Fields (com.joineryhq.jsumfields) version 1.0
Fast Action Links (com.megaphonetech.fastactionlinks) version 2.0
Megaphone Monitoring (com.megaphonetech.monitoring) version 1.0
Campaign Manager (de.systopia.campaign) version 1.3-dev
CallHub Phone Banking (io.callhub.phonebanking) version 1.1
Constituent Fields (net.ourpowerbase.constituentfields) version 1.0
Foundation Fields (net.ourpowerbase.foundationfields) version 1.0
Media Fields (net.ourpowerbase.mediafields) version 1.0
Participant Fields (net.ourpowerbase.participantfields) version 1.0
QR Code Checkin (net.ourpowerbase.qrcodecheckin) version 1.0.3
Event Count (net.ourpowerbase.search.eventcount) version 0.1
Spouse Settings (net.ourpowerbase.spouse) version 1.0
Summary Fields (net.ourpowerbase.sumfields) version 4.0.2
Turnout Extension (net.ourpowerbase.turnout) version 1.0
ExtendedReport (nz.co.fuzion.extendedreport) version 5.0
CiviRules (org.civicoop.civirules) version 2.10
Angular Profiles (org.civicrm.angularprofiles) version 4.7.31-1.1.2
API v4 (org.civicrm.api4) version 4.5.2
Contact Layout Editor (org.civicrm.contactlayout) version 1.4.3
CiviCRM Extensions Manager (org.civicrm.extensionsui) version 0.1
CiviCRM Bootstrap theme (org.civicrm.shoreditch) version 0.1-alpha35
twilio (org.civicrm.sms.twilio) version 1.2.1
CiviTutorial (org.civicrm.tutorial) version 1.1
CiviVolunteer (org.civicrm.volunteer) version 4.7.31-2.3.1
Contact Editor (org.wikimedia.contacteditor) version 2.1
Gender Self-Identify (org.woolman.genderselfidentify) version 1.2
Pivot Report (uk.co.compucorp.civicrm.pivotreport) version 2.0.1
Mailchimp (uk.co.vedaconsulting.mailchimp) version 2.0.1

[joemcl]

So I did this: ran cv ext:uninstall com.jlacey.electoral ran rm -r com.jlacey.electoral b/c com.jlacey.electoral was still in /extensions ran git clone https://github.com/josephlacey/com.jlacey.electoral.git ran cv en com.jlacey.electoral got this same error after ^^^ -

Error: API Call Failed: Array
(
    [entity] => Extension
    [action] => install
    [params] => Array
        (
            [keys] => Array
                (
                    [0] => com.jlacey.electoral
                )

            [debug] => 1
            [version] => 3
        )

    [result] => Array
        (
            [error_code] => already exists
            [trace] => #0 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/vendor/pear/pear-core-minimal/src/PEAR.php(922): CRM_Core_Error::exceptionHandler(Object(DB_Error))
#1 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/packages/DB.php(987): PEAR_Error->__construct('DB Error: alrea...', -5, 16, Array, 'INSERT INTO civ...')
#2 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/vendor/pear/pear-core-minimal/src/PEAR.php(575): DB_Error->__construct(-5, 16, Array, 'INSERT INTO civ...')
#3 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/vendor/pear/pear-core-minimal/src/PEAR.php(223): PEAR->_raiseError(Object(DB_mysqli), NULL, -5, 16, Array, 'INSERT INTO civ...', 'DB_Error', true)
#4 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/packages/DB/common.php(1920): PEAR->__call('raiseError', Array)
#5 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/packages/DB/mysqli.php(933): DB_common->raiseError(-5, NULL, NULL, 'INSERT INTO civ...', '1062 ** Duplica...')
#6 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/packages/DB/mysqli.php(403): DB_mysqli->mysqliRaiseError()
#7 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/packages/DB/common.php(1229): DB_mysqli->simpleQuery('INSERT INTO civ...')
#8 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/packages/DB/DataObject.php(2416): DB_common->query('INSERT INTO civ...')
#9 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/packages/DB/DataObject.php(1040): DB_DataObject->_query('INSERT INTO civ...')
#10 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/CRM/Core/DAO.php(569): DB_DataObject->insert()
#11 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/CRM/Core/BAO/CustomField.php(2038): CRM_Core_DAO->save()
#12 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/CRM/Core/BAO/CustomField.php(182): CRM_Core_BAO_CustomField::createCustomFieldRecord(Array)
#13 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/CRM/Utils/Migrate/Import.php(366): CRM_Core_BAO_CustomField::bulkSave(Array, Array)
#14 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/CRM/Utils/Migrate/Import.php(82): CRM_Utils_Migrate_Import->customFields(Object(SimpleXMLElement), Array)
#15 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/CRM/Utils/Migrate/Import.php(59): CRM_Utils_Migrate_Import->runXmlElement(Object(SimpleXMLElement))
#16 /var/www/joe-demo/htdocs/sites/all/civicrm-custom/extensions/com.jlacey.electoral/CRM/Electoral/Upgrader/Base.php(96): CRM_Utils_Migrate_Import->run('/var/www/joe-de...')
#17 /var/www/joe-demo/htdocs/sites/all/civicrm-custom/extensions/com.jlacey.electoral/CRM/Electoral/Upgrader/Base.php(253): CRM_Electoral_Upgrader_Base::executeCustomDataFileByAbsPath('/var/www/joe-de...')
#18 /var/www/joe-demo/htdocs/sites/all/civicrm-custom/extensions/com.jlacey.electoral/electoral.civix.php(50): CRM_Electoral_Upgrader_Base->onInstall()
#19 /var/www/joe-demo/htdocs/sites/all/civicrm-custom/extensions/com.jlacey.electoral/electoral.php(65): _electoral_civix_civicrm_install()
#20 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/CRM/Extension/Manager/Module.php(76): electoral_civicrm_install()
#21 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/CRM/Extension/Manager/Module.php(48): CRM_Extension_Manager_Module->callHook(Object(CRM_Extension_Info), 'install')
#22 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/CRM/Extension/Manager.php(251): CRM_Extension_Manager_Module->onPreInstall(Object(CRM_Extension_Info))
#23 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/api/v3/Extension.php(58): CRM_Extension_Manager->install(Array)
#24 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/Civi/API/Provider/MagicFunctionProvider.php(101): civicrm_api3_extension_install(Array)
#25 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/Civi/API/Kernel.php(168): Civi\API\Provider\MagicFunctionProvider->invoke(Array)
#26 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/Civi/API/Kernel.php(99): Civi\API\Kernel->runRequest(Array)
#27 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/api/api.php(23): Civi\API\Kernel->runSafe('Extension', 'install', Array, NULL)
#28 phar:///usr/local/bin/cv/src/Command/BaseCommand.php(49): civicrm_api('Extension', 'install', Array)
#29 phar:///usr/local/bin/cv/src/Command/ExtensionEnableCommand.php(91): Civi\Cv\Command\BaseCommand->callApiSuccess(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput), 'Extension', 'install', Array)
#30 phar:///usr/local/bin/cv/vendor/symfony/console/Command/Command.php(257): Civi\Cv\Command\ExtensionEnableCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#31 phar:///usr/local/bin/cv/vendor/symfony/console/Application.php(850): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#32 phar:///usr/local/bin/cv/vendor/symfony/console/Application.php(193): Symfony\Component\Console\Application->doRunCommand(Object(Civi\Cv\Command\ExtensionEnableCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#33 phar:///usr/local/bin/cv/src/Application.php(46): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#34 phar:///usr/local/bin/cv/vendor/symfony/console/Application.php(124): Civi\Cv\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#35 phar:///usr/local/bin/cv/src/Application.php(15): Symfony\Component\Console\Application->run()
#36 phar:///usr/local/bin/cv/bin/cv(26): Civi\Cv\Application::main('phar:///usr/loc...')
#37 /usr/local/bin/cv(10): require('phar:///usr/loc...')
#38 {main}
            [is_error] => 1
            [error_message] => DB Error: already exists
        )

)

Refreshed in the UI at ~/civicrm/a/#/extensions Jumped over to ~/civicrm/admin/extensions?reset=1 (legacy Extensions location), see Electoral 2.0 in Extensions, installing from there

[joemcl]

So when I click Install, I get this in the UI, with debugging enabled -

CiviCRM_API3_Exception: "DB Error: already exists"

#0 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/CRM/Admin/Form/Extensions.php(189): civicrm_api3("Extension", "install", (Array:2))
#1 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/CRM/Core/Form.php(495): CRM_Admin_Form_Extensions->postProcess()
#2 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/CRM/Core/StateMachine.php(160): CRM_Core_Form->mainProcess()
#3 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/CRM/Core/QuickForm/Action/Next.php(61): CRM_Core_StateMachine->perform(Object(CRM_Admin_Form_Extensions), "next", "Next")
#4 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/packages/HTML/QuickForm/Controller.php(203): CRM_Core_QuickForm_Action_Next->perform(Object(CRM_Admin_Form_Extensions), "next")
#5 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/packages/HTML/QuickForm/Page.php(103): HTML_QuickForm_Controller->handle(Object(CRM_Admin_Form_Extensions), "next")
#6 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/CRM/Core/Controller.php(351): HTML_QuickForm_Page->handle("next")
#7 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/CRM/Core/Page/Basic.php(411): CRM_Core_Controller->run()
#8 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/CRM/Core/Page/Basic.php(156): CRM_Core_Page_Basic->edit(1, NULL)
#9 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/CRM/Admin/Page/Extensions.php(121): CRM_Core_Page_Basic->run()
#10 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/CRM/Core/Invoke.php(290): CRM_Admin_Page_Extensions->run((Array:3), NULL)
#11 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/CRM/Core/Invoke.php(84): CRM_Core_Invoke::runItem((Array:16))
#12 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/CRM/Core/Invoke.php(52): CRM_Core_Invoke::_invoke((Array:3))
#13 /var/www/joe-demo/htdocs/sites/all/modules/civicrm/drupal/civicrm.module(444): CRM_Core_Invoke::invoke((Array:3))
#14 /var/www/joe-demo/htdocs/includes/menu.inc(527): civicrm_invoke("admin", "extensions")
#15 /var/www/joe-demo/htdocs/index.php(21): menu_execute_active_handler()
#16 {main}

[joemcl]

and I see this in the Console -

POST https://[site]/civicrm/admin/extensions 500 (Internal Server Error)

[joemcl]

also see this in the Console -

GET https://[site]favicon.ico 404 (Not Found)

[joemcl]

So I get the same error messages (circled on the right) as @jimcrist, this is a screenshot from #7 -

[joemcl]

OK @josephlacey @MegaphoneJon here's a CiviCRM log file excerpt, which appears repeatedly (I tried to Install several times) and I think it means I need to get into the back-end tables and get rid of/drop the current custom group Level-31 - or maybe I can just do that with the API Explorer?

Nov 20 13:17:11  [error] $Fatal Error Details = Array
(
    [callback] => Array
        (
            [0] => CRM_Core_Error
            [1] => exceptionHandler
        )

    [code] => -5
    [message] => DB Error: already exists
    [mode] => 16
    [debug_info] => INSERT INTO civicrm_custom_field (custom_group_id , name , label , data_type , html_type , is_required , is_searchable , is_search_range , weight , is_active , is_view , text_length , column_name , in_selector ) VALUES ( 31 , 'electoral_level' , 'Level' , 'String' , 'Select' ,  0 ,  1 ,  0 ,  1 ,  1 ,  0 ,  128 , 'electoral_districts_level' ,  1 )  [nativecode=1062 ** Duplicate entry 'Level-31' for key 'UI_label_custom_group_id']
    [type] => DB_Error
    [user_info] => INSERT INTO civicrm_custom_field (custom_group_id , name , label , data_type , html_type , is_required , is_searchable , is_search_range , weight , is_active , is_view , text_length , column_name , in_selector ) VALUES ( 31 , 'electoral_level' , 'Level' , 'String' , 'Select' ,  0 ,  1 ,  0 ,  1 ,  1 ,  0 ,  128 , 'electoral_districts_level' ,  1 )  [nativecode=1062 ** Duplicate entry 'Level-31' for key 'UI_label_custom_group_id']
    [to_string] => [db_error: message="DB Error: already exists" code=-5 mode=callback callback=CRM_Core_Error::exceptionHandler prefix="" info="INSERT INTO civicrm_custom_field (custom_group_id , name , label , data_type , html_type , is_required , is_searchable , is_search_range , weight , is_active , is_view , text_length , column_name , in_selector ) VALUES ( 31 , 'electoral_level' , 'Level' , 'String' , 'Select' ,  0 ,  1 ,  0 ,  1 ,  1 ,  0 ,  128 , 'electoral_districts_level' ,  1 )  [nativecode=1062 ** Duplicate entry 'Level-31' for key 'UI_label_custom_group_id']"]
)

[joemcl]

Here's the values for CustomGroup 31 using get with the API Explorer -

{
    "id": "31",
    "name": "electoral_districts",
    "title": "Electoral Districts",
    "extends": "Contact",
    "extends_entity_column_id": null,
    "extends_entity_column_value": null,
    "style": "Tab with table",
    "collapse_display": "0",
    "help_pre": null,
    "help_post": null,
    "weight": "2",
    "is_active": "1",
    "table_name": "civicrm_value_electoral_districts_31",
    "is_multiple": "1",
    "min_multiple": null,
    "max_multiple": null,
    "collapse_adv_display": "0",
    "created_id": null,
    "created_date": null,
    "is_reserved": "1",
    "is_public": "1"
  },

[joemcl]

So thinking I'm just going to delete CustomGroup 31 using the API Explorer...

[joemcl]

Did that, so will now try to install Electoral again....

[joemcl]

OK so still getting DB Error: already exists on cv en and on Install in the UI. I'm seeing two Option Group values that I'm going to also delete using the API Explorer -

  {
    "id": "137",
    "name": "electoral_districts_level_options",
    "title": "Level",
    "description": null,
    "data_type": null,
    "is_reserved": "1",
    "is_active": "1",
    "is_locked": "0"
  },
  {
    "id": "138",
    "name": "electoral_districts_chamber_options",
    "title": "Chamber",
    "description": null,
    "data_type": null,
    "is_reserved": "1",
    "is_active": "1",
    "is_locked": "0"
  }
]

[joemcl]

OK so gonna delete these Custom Fields installed by Electoral -

[
  {
    "id": "98",
    "custom_group_id": "32",
    "name": "electoral_status_error_code",
    "label": "Error Code",
    "data_type": "Int",
    "html_type": "Text",
    "default_value": null,
    "is_required": "0",
    "is_searchable": "1",
    "is_search_range": "0",
    "weight": "20",
    "help_pre": null,
    "help_post": null,
    "mask": null,
    "attributes": null,
    "javascript": null,
    "is_active": "1",
    "is_view": "0",
    "options_per_line": null,
    "text_length": "255",
    "start_date_years": null,
    "end_date_years": null,
    "date_format": null,
    "time_format": null,
    "note_columns": null,
    "note_rows": null,
    "column_name": "electoral_status_error_code",
    "option_group_id": null,
    "filter": null,
    "in_selector": "0"
  },
  {
    "id": "99",
    "custom_group_id": "32",
    "name": "electoral_status_error_reason",
    "label": "Error Reason",
    "data_type": "String",
    "html_type": "Text",
    "default_value": null,
    "is_required": "0",
    "is_searchable": "1",
    "is_search_range": "0",
    "weight": "27",
    "help_pre": null,
    "help_post": null,
    "mask": null,
    "attributes": null,
    "javascript": null,
    "is_active": "1",
    "is_view": "0",
    "options_per_line": null,
    "text_length": "255",
    "start_date_years": null,
    "end_date_years": null,
    "date_format": null,
    "time_format": null,
    "note_columns": null,
    "note_rows": null,
    "column_name": "electoral_status_error_reason",
    "option_group_id": null,
    "filter": null,
    "in_selector": "0"
  },
  {
    "id": "100",
    "custom_group_id": "32",
    "name": "electoral_status_error_message",
    "label": "Error Message",
    "data_type": "String",
    "html_type": "Text",
    "default_value": null,
    "is_required": "0",
    "is_searchable": "1",
    "is_search_range": "0",
    "weight": "33",
    "help_pre": null,
    "help_post": null,
    "mask": null,
    "attributes": null,
    "javascript": null,
    "is_active": "1",
    "is_view": "0",
    "options_per_line": null,
    "text_length": "255",
    "start_date_years": null,
    "end_date_years": null,
    "date_format": null,
    "time_format": null,
    "note_columns": null,
    "note_rows": null,
    "column_name": "electoral_status_error_message",
    "option_group_id": null,
    "filter": null,
    "in_selector": "0"
  },
  {
    "id": "156",
    "custom_group_id": "53",
    "name": "electoral_level",
    "label": "Level",
    "data_type": "String",
    "html_type": "Select",
    "default_value": null,
    "is_required": "0",
    "is_searchable": "1",
    "is_search_range": "0",
    "weight": "1",
    "help_pre": null,
    "help_post": null,
    "mask": null,
    "attributes": null,
    "javascript": null,
    "is_active": "1",
    "is_view": "0",
    "options_per_line": null,
    "text_length": "128",
    "start_date_years": null,
    "end_date_years": null,
    "date_format": null,
    "time_format": null,
    "note_columns": null,
    "note_rows": null,
    "column_name": "electoral_districts_level",
    "option_group_id": null,
    "filter": null,
    "in_selector": "1"
  },
  {
    "id": "157",
    "custom_group_id": "53",
    "name": "electoral_states_provinces",
    "label": "States/Provinces",
    "data_type": "StateProvince",
    "html_type": "Select State/Province",
    "default_value": null,
    "is_required": "0",
    "is_searchable": "1",
    "is_search_range": "0",
    "weight": "2",
    "help_pre": null,
    "help_post": null,
    "mask": null,
    "attributes": null,
    "javascript": null,
    "is_active": "1",
    "is_view": "0",
    "options_per_line": null,
    "text_length": null,
    "start_date_years": null,
    "end_date_years": null,
    "date_format": null,
    "time_format": null,
    "note_columns": null,
    "note_rows": null,
    "column_name": "electoral_districts_states_provinces",
    "option_group_id": null,
    "filter": null,
    "in_selector": "1"
  },
  {
    "id": "158",
    "custom_group_id": "53",
    "name": "electoral_counties",
    "label": "County",
    "data_type": "String",
    "html_type": "Text",
    "default_value": null,
    "is_required": "0",
    "is_searchable": "1",
    "is_search_range": "0",
    "weight": "3",
    "help_pre": null,
    "help_post": null,
    "mask": null,
    "attributes": null,
    "javascript": null,
    "is_active": "1",
    "is_view": "0",
    "options_per_line": null,
    "text_length": "128",
    "start_date_years": null,
    "end_date_years": null,
    "date_format": null,
    "time_format": null,
    "note_columns": null,
    "note_rows": null,
    "column_name": "electoral_districts_county",
    "option_group_id": null,
    "filter": null,
    "in_selector": "1"
  },
  {
    "id": "159",
    "custom_group_id": "53",
    "name": "electoral_cities",
    "label": "City",
    "data_type": "String",
    "html_type": "Text",
    "default_value": null,
    "is_required": "0",
    "is_searchable": "1",
    "is_search_range": "0",
    "weight": "4",
    "help_pre": null,
    "help_post": null,
    "mask": null,
    "attributes": null,
    "javascript": null,
    "is_active": "1",
    "is_view": "0",
    "options_per_line": null,
    "text_length": "128",
    "start_date_years": null,
    "end_date_years": null,
    "date_format": null,
    "time_format": null,
    "note_columns": null,
    "note_rows": null,
    "column_name": "electoral_districts_city",
    "option_group_id": null,
    "filter": null,
    "in_selector": "1"
  },
  {
    "id": "160",
    "custom_group_id": "53",
    "name": "electoral_chamber",
    "label": "Chamber",
    "data_type": "String",
    "html_type": "Select",
    "default_value": null,
    "is_required": "0",
    "is_searchable": "1",
    "is_search_range": "0",
    "weight": "5",
    "help_pre": null,
    "help_post": null,
    "mask": null,
    "attributes": null,
    "javascript": null,
    "is_active": "1",
    "is_view": "0",
    "options_per_line": null,
    "text_length": "128",
    "start_date_years": null,
    "end_date_years": null,
    "date_format": null,
    "time_format": null,
    "note_columns": null,
    "note_rows": null,
    "column_name": "electoral_districts_chamber",
    "option_group_id": null,
    "filter": null,
    "in_selector": "1"
  },
  {
    "id": "161",
    "custom_group_id": "53",
    "name": "electoral_district",
    "label": "District",
    "data_type": "String",
    "html_type": "Text",
    "default_value": null,
    "is_required": "0",
    "is_searchable": "1",
    "is_search_range": "0",
    "weight": "6",
    "help_pre": null,
    "help_post": null,
    "mask": null,
    "attributes": null,
    "javascript": null,
    "is_active": "1",
    "is_view": "0",
    "options_per_line": null,
    "text_length": "128",
    "start_date_years": null,
    "end_date_years": null,
    "date_format": null,
    "time_format": null,
    "note_columns": null,
    "note_rows": null,
    "column_name": "electoral_districts_district",
    "option_group_id": null,
    "filter": null,
    "in_selector": "1"
  },
  {
    "id": "162",
    "custom_group_id": "53",
    "name": "electoral_in_office",
    "label": "In Office?",
    "data_type": "Boolean",
    "html_type": "Radio",
    "default_value": null,
    "is_required": "0",
    "is_searchable": "1",
    "is_search_range": "0",
    "weight": "7",
    "help_pre": null,
    "help_post": null,
    "mask": null,
    "attributes": null,
    "javascript": null,
    "is_active": "1",
    "is_view": "0",
    "options_per_line": null,
    "text_length": null,
    "start_date_years": null,
    "end_date_years": null,
    "date_format": null,
    "time_format": null,
    "note_columns": null,
    "note_rows": null,
    "column_name": "electoral_districts_in_office",
    "option_group_id": null,
    "filter": null,
    "in_selector": "1"
  }
]

[joemcl]

and I'm gonna delete those Custom Fields now.

[joemcl]

So apparently there are some Option Value fields that don't have electoral in the field name, might want to add that to those, use Upper etc just for the label -

[
  {
    "id": "1084",
    "option_group_id": "141",
    "label": "Upper",
    "value": "upper",
    "name": "Upper",
    "grouping": null,
    "filter": null,
    "is_default": "0",
    "weight": "1",
    "description": null,
    "is_optgroup": "1",
    "is_reserved": "1",
    "is_active": "1",
    "component_id": null,
    "domain_id": null,
    "visibility_id": null,
    "icon": null,
    "color": null
  },
  {
    "id": "1085",
    "option_group_id": "141",
    "label": "Lower",
    "value": "lower",
    "name": "Lower",
    "grouping": null,
    "filter": null,
    "is_default": "0",
    "weight": "2",
    "description": null,
    "is_optgroup": "1",
    "is_reserved": "1",
    "is_active": "1",
    "component_id": null,
    "domain_id": null,
    "visibility_id": null,
    "icon": null,
    "color": null
  }
]

[joemcl]

I'm gonna delete these fields and see if those are the cause of the DB Error: already exists errors...

[joemcl]

OK so.. after doing all of the above... I could finally do a new Install from the Extensions UI, and didn't hit the DB Error: already exists error. I see Electoral API again under Administer -> System Settings and, for the first time, I see all the Google Civic APIs in Scheduled Jobs, finally :) .

[joemcl]

so @jimcrist @josephlacey @MegaphoneJon not sure what needs to get changed in the code, but the above seem to be the steps to clear out a failed Install and start over from scratch....

[joemcl]

Also it's a problem that Level, County, City, Chamber names aren't being populated... just district numbers. I'm in CA AD 68, CA SD 37, CA CD 45, and Orange County Board of Supervisors District 3, and the City of Irvine, which does not have City Council districts - all are at-large - so district number is blank, appropriately.

Screenshot:

[joemcl]

Right, so since I removed the labels fields, I have to add them back in.

[jimcrist]

@joemcl congrats on getting it working. So, deleting the right fields and tables in the database was the key to your success? Here’s my summary for upgrading:

Uninstall version 1, delete the custom fields (thus losing all the data in those custom fields, so beware), install version 2 (which will recreate the empty fields) and then redistrict all contacts. Google has a free limit of 2500 look ups/day, so divide your contact total by that and you'll have how long it'll take to redistrict all your contacts.

“Delete the custom fields” means delete the electoral references in the database: in civicrm_custom_field and civicrm_custom_group and drop the tables like civicrm_value_electoral…

You need to have a Google API key. If you're using Google as your geocoding or mapping provider, you can reuse that same key. Enable the API key by visiting: https://console.developers.google.com/apis/api/civicinfo.googleapis.com/overview?project=xxxxxx

It’s been working for me since July 23rd, but my version seems to be inefficient in locating which records need to be districted. I think @twomice has made improvements in this area. I’m not as techie as ya’ll. Instead of starting over again, can I just install the newer files??? That would be GoogleCivicInformation.php, Electoral.setting.php and Electoral.php.

I'm so pleased that this is being improved. It's a very valuable feature. Thanks so much for everyone's improvements!

[joemcl]

thanks @jimcrist . @josephlacey @MegaphoneJon @twomice I think it would be worth adding a second custom field set that the district etc results get sent and saved to, and that could be saved between installs and updates, so folks could delete the main field set to re-install or update, and not worry about losing existing data.

@jimcrist for finding records to be districted, you should be able to do that with a 'Smart Group' with a search for records that are not districted or for which an address was updated recently. Check out Include/Exclude Search in Custom Searches for including and excluding Groups and Smart Groups, as well as the CiviRules extension.

[joemcl]

So I can see using the API Explorer that the label for each district is in the correct field in my record, it's just not displaying in the UI...

[josephlacey]

Thanks for the details here @joemcl. Hopefully it will be helpful for anyone else upgrading from version 1 to 2. As I mentioned on #7, there unfortunately isn't an upgrade path between the two.

There is no tested upgrade path between version 1 and 2. Uninstalling version 1 and then installing version 2 might be totally fine, but I suspect the custom data fields from version 1 will turn out to be a problem. I don't delete those when you uninstall the extension; installing version 2 might then throw an error when it tries to create them. And the new Scheduled Jobs might not find the old custom fields correctly.

You could uninstall version 1, delete the custom fields (thus losing all the data in those custom fields, so beware), install version 2 (which will recreate the empty fields) and then redistrict all your contacts. I'm not sure how many contacts you have, but Google has a free limit of 2500 look ups/day, so divide your contact total by that and you'll have how long it'll take to redistrict all your contacts. If losing the lookup data you have isn't an option, which is totally reasonable, then you could hire someone to handle the migration in a more elegant way.

I'm going to close this issue. If there's bug or a feature request that's come out of this, you can create a new issue for discussion, etc.

[joemcl]

OK so an addendum - as @MegaphoneJon helped me figure out, the problem is that I was installing/uninstalling as the wrong user :( and that caused the problem.