APIv3: Support multiple groups

[xurizaemon]

I observed that Mailchimp's Webhook data did not include multiple Mailchimp Groups in their submission, and that a new field was available which did so.

I contacted @mailchimp's apihelp email, and they confirmed that while this is not yet documented, merges.INTERESTS is now for legacy and merges.GROUPINGS is the right information to use.

In regards to that, there is some transitioning going on to make sure that all necessary group information is shown.

"data.merges.INTERESTS" is still there for legacy use, but "data.merges.GROUPINGS" breaks it down more thoroughly, and adds interest group ID numbers. It is strange that data.merges.INTERESTS is only providing one grouping, but that is a legacy holdover, and the data is included in data.merges.GROUPINGS now.

Here's a sample response observed today -

array (
  'type' => 'profile',
  'fired_at' => '2016-09-20 04:25:11',
  'data' =>
  array (
    'id' => 'XXXXXX',
    'email' => 'test@example.org',
    'email_type' => 'html',
    'ip_opt' => '6.6.6.6',
    'web_id' => 'XXXXXX',
    'merges' =>
    array (
      'EMAIL' => 'test@example.org',
      'FNAME' => 'A',
      'LNAME' => 'B',
      'INTERESTS' => 'Cats, Dogs, Birds',
      'GROUPINGS' =>
      array (
        0 =>
        array (
          'id' => '123',
          'name' => 'My interests',
          'groups' => 'Cats, Dogs, Birds',
        ),
        1 =>
        array (
          'id' => '124',
          'name' => 'Interest level',
          'groups' => '',
        ),
        2 =>
        array (
          'id' => '125',
          'name' => 'Regions',
          'groups' => 'NZ',
        ),
        3 =>
        array (
          'id' => '126',
          'name' => 'My disinterests',
          'groups' => 'Tulips, Daffodils',
        ),
      ),
    ),
    'list_id' => 'XXXXXX',
  ),
),

Currently the extension does not support associating CiviCRM Groups with Mailchimp Groups which are not in that (first?) MC Group.

[samuelsov]

The update from CiviCRM to Mailchimp seems to work for those extra GROUPINGS so it's only a problem for synchronization from Mailchimp to CiviCRM. A refactoring of the function splitMailchimpWebhookGroupsToCiviGroupIds should resolve the problem.

[samuelsov]

Clarification about the structure sent from Mailchimp : there is also a 'unique_id' which correspond to the list_id in CiviCRM. We should ignore the id.

array (
  'type' => 'profile',
  'fired_at' => '2016-09-20 04:25:11',
  'data' =>
  array (
    'id' => 'XXXXXX',
    'email' => 'test@example.org',
    'email_type' => 'html',
    'ip_opt' => '6.6.6.6',
    'web_id' => 'XXXXXX',
    'merges' =>
    array (
      'EMAIL' => 'test@example.org',
      'FNAME' => 'A',
      'LNAME' => 'B',
      'INTERESTS' => 'Cats, Dogs, Birds',
      'GROUPINGS' =>
      array (
        0 =>
        array (
          'id' => '123',
          'unique_id' => '2d4e3d6b55',
          'name' => 'My interests',
          'groups' => 'Cats, Dogs, Birds',
        ),
        1 =>
        array (
          'id' => '124',
          'unique_id' => '3d4e3d6b55',
          'name' => 'Interest level',
          'groups' => '',
        ),
        2 =>
        array (
          'id' => '125',
          'unique_id' => '4d4e3d6b55',
          'name' => 'Regions',
          'groups' => 'NZ',
        ),
        3 =>
        array (
          'id' => '126',
          'unique_id' => '5d4e3d6b55',
          'name' => 'My disinterests',
          'groups' => 'Tulips, Daffodils',
        ),
      ),
    ),
    'list_id' => 'XXXXXX',
  ),
),

[samuelsov]

Ok so the 'unique_id' correspond to the category and not the list_id... Anyway, i should be able to provide a working patch soon.

[veda-consulting]

@deepak-srivastava can we get this PR in? All tests should still pass.

[jdorner4]

I contacted @mailchimp's api help because the documentation still doesn't mention the GROUPINGS field.

After a couple of messages referring to the documentation they responded (8/16/2019): "It looks like it [INTERESTS] will display for a single public group on the audience, which may be related to how we used to processed groups. It also look like our webhook documentation may also be a bit outdated and the article you share [THIS ARTICLE] does appear to be correct about INTERESTS going legacy . I do apologize for that and I'll pass information along to our team to get our documentation updated. GROUPINGS looks to provide the most information and you would want to use that for reference moving forward."