Overlapping CEO terms causes committee pages to error out

[sentry-io[bot]]

Email out to Shelly and Omar for clarification on data.

Sentry Issue: LA-METRO-COUNCILMATIC-GQ0

Membership.MultipleObjectsReturned: get() returned more than one Membership -- it returned 2!
(5 additional frame(s) were not displayed)
...
  File "django/views/generic/list.py", line 142, in get
    self.object_list = self.get_queryset()
  File "lametro/views.py", line 486, in get_queryset
    ceo = LAMetroPerson.ceo()
  File "lametro/models.py", line 353, in ceo
    end_date_dt__gt=Now())\
  File "django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "django/db/models/query.py", line 412, in get
    (self.model._meta.object_name, num)

[hancush]

Per @shrayshray:

It looks like Phil’s CEO record was modified with Stephanie’s name and term dates, instead of creating a new record for her. I readded all Phil’s previous terms, and set the 2020/2021 terms to end on 5/31/2021.

The modified record resulted in two current CEO memberships, Phil's original membership and a new one for Stephanie, leading to this error. I re-ran the scrape to capture Shelly's data updates and everything is now loading without error – yay! Welcome to the new CEO. @shrayshray, should we open an issue to add Stephanie Wiggins?

[shrayshray]

@hancush yes, i'll open an issue for adding Stephanie. One weird thing I noticed once the Internal Server Error was resolved - now Phil Washington is listed in the Board Member section of a few of the committees: Planning and Programming Executive Management Construction

[hancush]

Good catch, @shrayshray, looks like our system was holding on to some outdated information. I went ahead and reconciled our database with what's in the API. Stashing some debugging here, for my future reference:

(lametro) datamade@ip-10-0-0-48:~/lametro$ python manage.py shell
Python 3.6.9 (default, Jan 26 2021, 15:33:00)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from lametro.models import LAMetroPerson
>>> pw = LAMetroPerson.objects.get(family_name='Washington')
>>> pw
<LAMetroPerson: Phil Washington>
>>> err = pw.memberships.filter(end_date='2021-06-30').exclude(organization__name__in=('Board of Directors - Special Board Meeting', 'Crenshaw Project Corporation'))
>>> [(m.start_date, m.end_date) for m in err]
[('2014-07-01', '2021-06-30'), ('2014-07-01', '2021-06-30'), ('2014-07-01', '2021-06-30')]
>>> nonerr = pw.memberships.filter(end_date='2021-05-31').exclude(organization__name__in=('Board of Directors - Special Board Meeting', 'Crenshaw Project Corporation'))
>>> [(m.label, m.role) for m in err]
[('', 'Chief Executive Officer'), ('', 'Chief Executive Officer'), ('', 'Chief Executive Officer')]
>>> [(m.label, m.role) for m in nonerr]
[('', 'Chief Executive Officer'), ('', 'Chief Executive Officer'), ('', 'Chief Executive Officer'), ('', 'Chief Executive Officer'), ('', 'Chief Executive Officer'), ('', 'Chief Executive Officer'), ('', 'Chief Executive Officer'), ('', 'Chief Executive Officer')]
>>> [(m.updated_at) for m in err]
[datetime.datetime(2020, 7, 16, 13, 18, 17, 129685, tzinfo=<UTC>), datetime.datetime(2020, 7, 16, 13, 18, 17, 447301, tzinfo=<UTC>), datetime.datetime(2020, 7, 16, 13, 18, 25, 875353, tzinfo=<UTC>)]
>>> [(m.updated_at) for m in nonerr]
[datetime.datetime(2021, 6, 3, 20, 21, 32, 403501, tzinfo=<UTC>), datetime.datetime(2021, 6, 3, 20, 21, 29, 640761, tzinfo=<UTC>), datetime.datetime(2021, 6, 3, 20, 21, 32, 2512, tzinfo=<UTC>), datetime.datetime(2021, 6, 3, 20, 21, 29, 971292, tzinfo=<UTC>), datetime.datetime(2021, 6, 3, 20, 21, 30, 138147, tzinfo=<UTC>), datetime.datetime(2021, 6, 3, 20, 21, 30, 309574, tzinfo=<UTC>), datetime.datetime(2021, 6, 3, 20, 21, 30, 666206, tzinfo=<UTC>), datetime.datetime(2021, 6, 3, 20, 21, 31, 781309, tzinfo=<UTC>)]
>>> [(m.start_date, m.end_date) for m in err]
[('2014-07-01', '2021-06-30'), ('2014-07-01', '2021-06-30'), ('2014-07-01', '2021-06-30')]
>>> [(m.start_date, m.end_date) for m in nonerr]
[('2020-07-01', '2021-05-31'), ('2020-07-01', '2021-05-31'), ('2020-07-01', '2021-05-31'), ('2020-07-01', '2021-05-31'), ('2020-07-01', '2021-05-31'), ('2020-07-01', '2021-05-31'), ('2020-07-01', '2021-05-31'), ('2020-07-01', '2021-05-31')]
>>> import pprint
>>> pprint.pprint([(m.start_date, m.end_date, m.updated_at) for m in pw.memberships.filter(organization__name='Construction Committee').order_by('start_date')])
[('2014-07-01',
  '2021-06-30',
  datetime.datetime(2020, 7, 16, 13, 18, 17, 129685, tzinfo=<UTC>)),
 ('2015-05-01',
  '2015-06-30',
  datetime.datetime(2021, 6, 3, 20, 21, 30, 709765, tzinfo=<UTC>)),
 ('2015-07-01',
  '2016-06-30',
  datetime.datetime(2021, 6, 3, 20, 21, 29, 669162, tzinfo=<UTC>)),
 ('2016-07-01',
  '2017-06-30',
  datetime.datetime(2021, 6, 3, 20, 21, 30, 842145, tzinfo=<UTC>)),
 ('2017-07-01',
  '2018-06-30',
  datetime.datetime(2021, 6, 3, 20, 21, 32, 374145, tzinfo=<UTC>)),
 ('2018-07-01',
  '2019-06-30',
  datetime.datetime(2021, 6, 3, 20, 21, 31, 855064, tzinfo=<UTC>)),
 ('2019-07-01',
  '2020-06-30',
  datetime.datetime(2021, 6, 3, 20, 21, 30, 130415, tzinfo=<UTC>)),
 ('2020-07-01',
  '2021-05-31',
  datetime.datetime(2021, 6, 3, 20, 21, 29, 971292, tzinfo=<UTC>))]
>>> pprint.pprint([(m.start_date, m.end_date, m.updated_at) for m in pw.memberships.filter(organization__name='Planning and Programming Committee').order_by('start_date')])
[('2014-07-01',
  '2021-06-30',
  datetime.datetime(2020, 7, 16, 13, 18, 17, 447301, tzinfo=<UTC>)),
 ('2015-05-01',
  '2015-06-30',
  datetime.datetime(2021, 6, 3, 20, 21, 32, 248341, tzinfo=<UTC>)),
 ('2015-07-01',
  '2016-06-30',
  datetime.datetime(2021, 6, 3, 20, 21, 31, 427067, tzinfo=<UTC>)),
 ('2016-07-01',
  '2017-06-30',
  datetime.datetime(2021, 6, 3, 20, 21, 32, 389584, tzinfo=<UTC>)),
 ('2017-07-01',
  '2018-06-30',
  datetime.datetime(2021, 6, 3, 20, 21, 31, 184606, tzinfo=<UTC>)),
 ('2018-07-01',
  '2019-06-30',
  datetime.datetime(2021, 6, 3, 20, 21, 29, 423893, tzinfo=<UTC>)),
 ('2019-07-01',
  '2020-06-30',
  datetime.datetime(2021, 6, 3, 20, 21, 32, 124317, tzinfo=<UTC>)),
 ('2020-07-01',
  '2021-05-31',
  datetime.datetime(2021, 6, 3, 20, 21, 30, 309574, tzinfo=<UTC>))]

[shrayshray]

@hancush can i add more hurrahs? i'm trying

[shrayshray]

✨✨✨✨✨✨✨

I did it. ❤️ 😎 👽 😺 ❤️