search

tutorcruncher / hermes

1 stars 0 forks source link

Handle Pipedrive merging of Org, Person or Deal #227

Closed PrenSJ2 closed 4 months ago

PrenSJ2 commented 5 months ago

Fix #188

The Problem

The sales team often merge Pipedrive Objects (Organizations, Persons, Deals)

this causes the custom field hermes_id to be combined 123 --> '123, 456'

which in turn causes a validation error as we are expecting the hermes_id to be type FK_FIELD which is an int

Solution

In PipedriveEvent vaidate_obj, if hermes_id is a str we use the newly created handle_duplicate_hermes_ids to select a main object and update all the associated objects to have the main object id

Testing

  1. run make restore-from-live
  2. run uvicorn - python -m uvicorn app.main:app --reload
  3. setup a postman request to post to localhost:8000/pipedrive/callback/

Test Deal

  1. we need to create a new deal, do this on the hermes admin page
  2. create a new meeting on hermes admin, and set the deal id to the new deal - meeting/create
  3. then post to /pipedrive/callback/ with: 
    {
"v": 1,
"matches_filters": {
    "current": []
},
"meta": {
    "action": "updated",
    "object": "deal"
},
"current": {
    "id": 5712,
    "acv": null,
    "arr": null,
    "mrr": null,
    "label": "58",
    "title": "Tutor Company",
    "value": 1,
    "active": true,
    "org_id": 7663,
    "origin": "API",
    "status": "open",
    "channel": null,
    "deleted": false,
    "user_id": 15761943,
    "add_time": "2024-05-23 07:42:35",
    "cc_email": "tutorcruncher+deal5712@pipedrivemail.com",
    "currency": "GBP",
    "obj_type": "deal",
    "org_name": "Tutor Company",
    "stage_id": 72,
    "won_time": null,
    "lost_time": null,
    "origin_id": null,
    "person_id": 30169,
    "channel_id": null,
    "close_time": null,
    "org_hidden": false,
    "owner_name": "Sam",
    "visible_to": "3",
    "files_count": 0,
    "lost_reason": null,
    "notes_count": 0,
    "person_name": "Tutor Company Owner",
    "pipeline_id": 7,
    "probability": null,
    "rotten_time": null,
    "update_time": "2024-05-23 15:02:03",
    "acv_currency": null,
    "arr_currency": null,
    "mrr_currency": null,
    "person_hidden": false,
    "first_won_time": null,
    "local_won_date": null,
    "products_count": 0,
    "stage_order_nr": 4,
    "weighted_value": 1,
    "creator_user_id": 15708604,
    "followers_count": 1,
    "formatted_value": "£1",
    "local_lost_date": null,
    "activities_count": 0,
    "last_activity_id": null,
    "local_close_date": null,
    "next_activity_id": null,
    "stage_change_time": "2024-05-23 07:42:40",
    "last_activity_date": null,
    "next_activity_date": null,
    "next_activity_note": null,
    "next_activity_time": null,
    "next_activity_type": null,
    "participants_count": 1,
    "expected_close_date": "2024-06-06",
    "email_messages_count": 4,
    "done_activities_count": 0,
    "next_activity_subject": null,
    "next_activity_duration": null,
    "last_incoming_mail_time": "2024-05-23 14:52:21",
    "last_outgoing_mail_time": "2024-05-23 14:19:38",
    "undone_activities_count": 0,
    "weighted_value_currency": "GBP",
    "formatted_weighted_value": "£1",
    "177e444cef4dc6f0101a3ee51276eab1b02d335d": null,
    "306f085fd53c5ae792aeed30e3978484a26689d6": null,
    "5be1188db52a8c7f0ea49331eb391ae54aeabafc": "8303",
    "8d3fb70224a8f613a6cc7416e121996606868929": null,
    "90e1589738e5dc4ecec002c6f8424348f0c17143": null,
    "9d1836a71ef185ab2103eae0c902b4f4af6e1b03": null,
    "a8fb293ba4cfedc80a733b3411620e00921ba7d8": null,
    "abec03eee90b12d8b5658b7f988126e254a4bd79": null,
    "b0d672e094f9057c86f31a3383bedbb163ce718f": null,
    "c87e16b3aa96fb56c6d4bacf824d9a4fae2611d9": null,
    "f0a2dc95b937e2ec8ad860f0dd9cbfe4a338f818": null
},
"previous": {
    "id": 5712,
    "acv": null,
    "arr": null,
    "mrr": null,
    "label": "58",
    "title": "Tutor Company",
    "value": 1,
    "active": true,
    "org_id": 7663,
    "origin": "API",
    "status": "open",
    "channel": null,
    "deleted": false,
    "user_id": 15761943,
    "add_time": "2024-05-23 07:42:35",
    "cc_email": "tutorcruncher+deal5712@pipedrivemail.com",
    "currency": "GBP",
    "obj_type": "deal",
    "org_name": "Tutor Company",
    "stage_id": 72,
    "won_time": null,
    "lost_time": null,
    "origin_id": null,
    "person_id": 30169,
    "channel_id": null,
    "close_time": null,
    "org_hidden": false,
    "owner_name": "Sam",
    "visible_to": "3",
    "files_count": 0,
    "lost_reason": null,
    "notes_count": 0,
    "person_name": "Tutor Company Owner",
    "pipeline_id": 7,
    "probability": null,
    "rotten_time": null,
    "update_time": "2024-05-23 15:02:03",
    "acv_currency": null,
    "arr_currency": null,
    "mrr_currency": null,
    "person_hidden": false,
    "first_won_time": null,
    "local_won_date": null,
    "products_count": 0,
    "stage_order_nr": 4,
    "weighted_value": 1,
    "creator_user_id": 15708604,
    "followers_count": 1,
    "formatted_value": "£1",
    "local_lost_date": null,
    "activities_count": 0,
    "last_activity_id": null,
    "local_close_date": null,
    "next_activity_id": null,
    "stage_change_time": "2024-05-23 07:42:40",
    "last_activity_date": null,
    "next_activity_date": null,
    "next_activity_note": null,
    "next_activity_time": null,
    "next_activity_type": null,
    "participants_count": 1,
    "expected_close_date": "2024-06-06",
    "email_messages_count": 4,
    "done_activities_count": 0,
    "next_activity_subject": null,
    "next_activity_duration": null,
    "last_incoming_mail_time": "2024-05-23 14:52:21",
    "last_outgoing_mail_time": "2024-05-23 14:19:38",
    "undone_activities_count": 0,
    "weighted_value_currency": "GBP",
    "formatted_weighted_value": "£1",
    "177e444cef4dc6f0101a3ee51276eab1b02d335d": null,
    "306f085fd53c5ae792aeed30e3978484a26689d6": null,
    "5be1188db52a8c7f0ea49331eb391ae54aeabafc": "8303, {new_deal.id}", // this is the hermes deal id
    "8d3fb70224a8f613a6cc7416e121996606868929": null,
    "90e1589738e5dc4ecec002c6f8424348f0c17143": null,
    "9d1836a71ef185ab2103eae0c902b4f4af6e1b03": null,
    "a8fb293ba4cfedc80a733b3411620e00921ba7d8": null,
    "abec03eee90b12d8b5658b7f988126e254a4bd79": null,
    "b0d672e094f9057c86f31a3383bedbb163ce718f": null,
    "c87e16b3aa96fb56c6d4bacf824d9a4fae2611d9": null,
    "f0a2dc95b937e2ec8ad860f0dd9cbfe4a338f818": null
},
"event": "updated.deal"
}

check that the deal id for the newly created meeting is now 5712 check that the deal you created has been deleted

Testing Organisation

  1. Create a new Company 'A' in hermes admin
  2. Create a new Contact in hermes admin and set the company.id to A - /contact/create
  3. Create a new Deal in hermes admin and set the company.id to A - /deal/create
  4. post to /pipedrive/callback/ 
    
{
"v": 1,
"matches_filters": {
"current": []
},
"meta": {
"action": "updated",
"object": "organization"
},
"current": {
"id": 6998,
"name": "new Tutor Company",
"label": null,
"address": null,
"add_time": "2024-02-23 14:55:34",
"cc_email": "tutorcruncher@pipedrivemail.com",
"obj_type": "organization",
"owner_id": 15761943,
"edit_name": true,
"label_ids": [],
"company_id": 11324733,
"first_char": "a",
"owner_name": "Sam",
"picture_id": null,
"visible_to": "3",
"active_flag": true,
"category_id": null,
"delete_time": null,
"files_count": 0,
"notes_count": 1,
"update_time": "2024-05-23 15:33:09",
"country_code": null,
"people_count": 1,
"address_route": null,
"address_country": "GB",
"followers_count": 1,
"won_deals_count": 0,
"activities_count": 0,
"address_locality": null,
"last_activity_id": null,
"lost_deals_count": 0,
"next_activity_id": null,
"open_deals_count": 1,
"address_subpremise": null,
"closed_deals_count": 0,
"last_activity_date": null,
"next_activity_date": null,
"next_activity_time": null,
"address_postal_code": null,
"address_sublocality": null,
"email_messages_count": 214,
"address_street_number": null,
"done_activities_count": 0,
"related_won_deals_count": 0,
"undone_activities_count": 0,
"related_lost_deals_count": 0,
"related_open_deals_count": 0,
"address_formatted_address": null,
"address_admin_area_level_1": null,
"address_admin_area_level_2": null,
"related_closed_deals_count": 0,
"067f5298c65be333811f54203503839b9d9341ee": null,
"0dce4a9cac0177a95c0233756b8cd045bf7e28f2": null,
"1e09f790cd937cfc1c34f8a8b664a8b0293253c4": null,
"385e5989540972c85c9659a62a8ab82fd8c6c5fa": null,
"45f62b0fc120d201ea02fdfa5e7282273add2f20": null,
"4be5bf6e60e2a01e2653532e872cd15b5308da23": null,
"57170eb130b8fa45925381623c86011e4e598e21": "pending_email_conf",
"5ce5a41297410f570d97d78341aa2fbcf5801012": 3,
"70527310be44839c869854b055788a69ecbbab66": "0",
"770b2fee9c89906b60a74057719509e087342ae9": "http://bulldog-manatee-nbrd.squarespace.com",
"7c18fff1ec6b2ddf7e22a4571d24a511079d2ec8": null,
"7f8959760703808f36b3795c15310566b74f5134": "48553, {A.id}", // this is the hermes id
"8c5b9aa3af06e36d7136bddf32d55e04bd9508b5": null,
"bdef8d12d2f2af6a61e907b6296e410fdfbef9e3": null,
"d30bf32a173cdfa780901d5eeb92a8f2d1ccd980": null,
"d8b615ce8885544ed228feaae3ce9f28dcf04531": "https://secure.tutorcruncher.com/clients/2755259/",
"dbe81f3fdf69ce3dfbfc7609caee68f1654c901a": "just starting out",
"f6a981f2aa1199ecbb7fda5aa39a2d9ad672234b": null
},
"previous": {
"id": 6998,
"name": "Tutor Company",
"label": null,
"address": null,
"add_time": "2024-02-23 14:55:34",
"cc_email": "tutorcruncher@pipedrivemail.com",
"obj_type": "organization",
"owner_id": 15761943,
"edit_name": true,
"label_ids": [],
"company_id": 11324733,
"first_char": "a",
"owner_name": "Sam",
"picture_id": null,
"visible_to": "3",
"active_flag": true,
"category_id": null,
"delete_time": null,
"files_count": 0,
"notes_count": 1,
"update_time": "2024-05-23 15:33:09",
"country_code": null,
"people_count": 1,
"address_route": null,
"address_country": "GB",
"followers_count": 1,
"won_deals_count": 0,
"activities_count": 0,
"address_locality": null,
"last_activity_id": null,
"lost_deals_count": 0,
"next_activity_id": null,
"open_deals_count": 1,
"address_subpremise": null,
"closed_deals_count": 0,
"last_activity_date": null,
"next_activity_date": null,
"next_activity_time": null,
"address_postal_code": null,
"address_sublocality": null,
"email_messages_count": 214,
"address_street_number": null,
"done_activities_count": 0,
"related_won_deals_count": 0,
"undone_activities_count": 0,
"related_lost_deals_count": 0,
"related_open_deals_count": 0,
"address_formatted_address": null,
"address_admin_area_level_1": null,
"address_admin_area_level_2": null,
"related_closed_deals_count": 0,
"067f5298c65be333811f54203503839b9d9341ee": null,
"0dce4a9cac0177a95c0233756b8cd045bf7e28f2": null,
"1e09f790cd937cfc1c34f8a8b664a8b0293253c4": null,
"385e5989540972c85c9659a62a8ab82fd8c6c5fa": null,
"45f62b0fc120d201ea02fdfa5e7282273add2f20": null,
"4be5bf6e60e2a01e2653532e872cd15b5308da23": null,
"57170eb130b8fa45925381623c86011e4e598e21": "pending_email_conf",
"5ce5a41297410f570d97d78341aa2fbcf5801012": 3,
"70527310be44839c869854b055788a69ecbbab66": "0",
"770b2fee9c89906b60a74057719509e087342ae9": "http://bulldog-manatee-nbrd.squarespace.com",
"7c18fff1ec6b2ddf7e22a4571d24a511079d2ec8": null,
"7f8959760703808f36b3795c15310566b74f5134": "48553, {A.id}", // this is the hermes id
"8c5b9aa3af06e36d7136bddf32d55e04bd9508b5": null,
"bdef8d12d2f2af6a61e907b6296e410fdfbef9e3": null,
"d30bf32a173cdfa780901d5eeb92a8f2d1ccd980": null,
"d8b615ce8885544ed228feaae3ce9f28dcf04531": "https://secure.tutorcruncher.com/clients/2755259/",
"dbe81f3fdf69ce3dfbfc7609caee68f1654c901a": "just starting out",
"f6a981f2aa1199ecbb7fda5aa39a2d9ad672234b": null
},
"event": "updated.organization"
}

test that the contact's company was updated to 6998
test that the deal's company was updated to 6998
test that company A was deleted

#### Test Contact

1. Create a new contact 'A' in hermes admin
2. create a new deal with the contact = A
3. create a new meeting with the contact = A
4. post to `/pipedrive/callback/` with

{ "v": 1, "matches_filters": { "current": [] }, "meta": { "action": "updated", "object": "person" }, "current": { "id": 29664, "im": [ { "value": "", "primary": true } ], "name": "Test Person", "email": [ { "label": "work", "value": "test_person@example.com", "primary": true } ], "label": null, "notes": null, "phone": [ { "label": "", "value": "61 666666666", "primary": true } ], "org_id": 7578, "add_time": "2024-05-13 06:40:18", "birthday": null, "cc_email": "tutorcruncher@pipedrivemail.com", "obj_type": "person", "org_name": "Tutor Company", "owner_id": 15761943, "job_title": null, "label_ids": [], "last_name": "Owner", "company_id": 11324733, "doi_status": 1, "first_char": "p", "first_name": "Tutor", "owner_name": "Sam", "picture_id": null, "visible_to": "3", "active_flag": true, "delete_time": null, "files_count": 0, "notes_count": 1, "update_time": "2024-05-23 15:42:03", "postal_address": null, "followers_count": 1, "picture_128_url": null, "won_deals_count": 0, "activities_count": 0, "last_activity_id": null, "lost_deals_count": 0, "marketing_status": "no_consent", "next_activity_id": null, "open_deals_count": 1, "closed_deals_count": 0, "last_activity_date": null, "next_activity_date": null, "next_activity_time": null, "email_messages_count": 3, "postal_address_route": null, "done_activities_count": 0, "postal_address_country": null, "last_incoming_mail_time": "2024-05-13 10:32:54", "last_outgoing_mail_time": "2024-05-23 15:41:56", "postal_address_locality": null, "related_won_deals_count": 0, "undone_activities_count": 0, "related_lost_deals_count": 0, "related_open_deals_count": 0, "postal_address_subpremise": null, "postal_address_postal_code": null, "postal_address_sublocality": null, "related_closed_deals_count": 0, "participant_open_deals_count": 0, "postal_address_street_number": null, "participant_closed_deals_count": 0, "postal_address_formatted_address": null, "postal_address_admin_area_level_1": null, "postal_address_admin_area_level_2": null, "00230e1468e98ca253e39870d14256cfc8b77514": null, "5b1e493cfb81e5817c481b149b3b32891bdf3312": null, "8c2f326b6be255cd3d5cf4ee7385eaf544a47f1d": "89348", "c1c3c51f1a26713b786a5274e450e997fd0c813b": null, "c2cce53d5a10e5124154a4cabc0aaf4199e4d94d": null, "c52a0f158c49ae8a2c428ac4fdc5575554e3760c": null, "c608baee9f204e6a09e5c741dc8159e9295c1dbb": null, "ccefbc26a1f2c753e467440c7420e7342e761c44": null, "f11fc7f795d827ef2edb02e729599ca291b77311": null } , "previous": { "id": 29664, "im": [ { "value": "", "primary": true } ], "name": "Test Person", "email": [ { "label": "work", "value": "test_person@example.com", "primary": true } ], "label": null, "notes": null, "phone": [ { "label": "", "value": "61 666666666", "primary": true } ], "org_id": 7578, "add_time": "2024-05-13 06:40:18", "birthday": null, "cc_email": "tutorcruncher@pipedrivemail.com", "obj_type": "person", "org_name": "Tutor Company", "owner_id": 15761943, "job_title": null, "label_ids": [], "last_name": "Owner", "company_id": 11324733, "doi_status": 1, "first_char": "p", "first_name": "Tutor", "owner_name": "Sam", "picture_id": null, "visible_to": "3", "active_flag": true, "delete_time": null, "files_count": 0, "notes_count": 1, "update_time": "2024-05-23 15:42:03", "postal_address": null, "followers_count": 1, "picture_128_url": null, "won_deals_count": 0, "activities_count": 0, "last_activity_id": null, "lost_deals_count": 0, "marketing_status": "no_consent", "next_activity_id": null, "open_deals_count": 1, "closed_deals_count": 0, "last_activity_date": null, "next_activity_date": null, "next_activity_time": null, "email_messages_count": 3, "postal_address_route": null, "done_activities_count": 0, "postal_address_country": null, "last_incoming_mail_time": "2024-05-13 10:32:54", "last_outgoing_mail_time": "2024-05-23 15:41:56", "postal_address_locality": null, "related_won_deals_count": 0, "undone_activities_count": 0, "related_lost_deals_count": 0, "related_open_deals_count": 0, "postal_address_subpremise": null, "postal_address_postal_code": null, "postal_address_sublocality": null, "related_closed_deals_count": 0, "participant_open_deals_count": 0, "postal_address_street_number": null, "participant_closed_deals_count": 0, "postal_address_formatted_address": null, "postal_address_admin_area_level_1": null, "postal_address_admin_area_level_2": null, "00230e1468e98ca253e39870d14256cfc8b77514": null, "5b1e493cfb81e5817c481b149b3b32891bdf3312": null, "8c2f326b6be255cd3d5cf4ee7385eaf544a47f1d": "89348, {new_person_id}", // hermes id "c1c3c51f1a26713b786a5274e450e997fd0c813b": null, "c2cce53d5a10e5124154a4cabc0aaf4199e4d94d": null, "c52a0f158c49ae8a2c428ac4fdc5575554e3760c": null, "c608baee9f204e6a09e5c741dc8159e9295c1dbb": null, "ccefbc26a1f2c753e467440c7420e7342e761c44": null, "f11fc7f795d827ef2edb02e729599ca291b77311": null }, "event": "updated.person" }

codecov-commenter commented 5 months ago

Codecov Report

Attention: Patch coverage is 93.90244% with 5 lines in your changes missing coverage. Please review.

Project coverage is 93.86%. Comparing base (ed2cefa) to head (fd16a05). Report is 11 commits behind head on main.

Files Patch % Lines
app/pipedrive/_schema.py 92.98% 1 Missing and 3 partials :warning:
app/pipedrive/views.py 96.00% 0 Missing and 1 partial :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #227 +/- ## ========================================== + Coverage 93.76% 93.86% +0.10% ========================================== Files 29 29 Lines 1652 1712 +60 Branches 211 228 +17 ========================================== + Hits 1549 1607 +58 Misses 66 66 - Partials 37 39 +2 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.