Committees: Committee pages missing Recent Activity

[shrayshray]

It looks like the "Recent Activity" sections on committee pages are not updating. The last activity is from 11/2017, see Construction Committee for example: https://boardagendas.metro.net/committee/construction-committee-ebbc3e5b8887/ The newest committee, which met for the first time 7/19/18, has no recent activity listed: https://boardagendas.metro.net/committee/operations-safety-and-customer-experience-committee-b28353454c39/

[shrayshray]

Is it possible to please hide/disable this section on the website until the issue is resolved?

[shrayshray]

@reginafcompton Is it possible to please hide/disable the "Recent Activity" section on committee pages until this issue is resolved?

[shrayshray]

It looks like this same issue is occurring on Board Member pages, as well. Examples: https://boardagendas.metro.net/person/ara-najarian-a9c58166e1da/ https://boardagendas.metro.net/person/jacquelyn-dupont-walker-851703b7b19d/

[reginafcompton]

I spent a little time investigating this issue. It looks like the Legistar API contains some unexpected information. I'll summarize.

The following board reports have actions or histories from 2018. The left column shows the date of the action, and the right column shows its url in Legistar.

Relevant queries

 2018-05-24 | http://webapi.legistar.com/v1/metro/matters/4846
 2018-05-24 | http://webapi.legistar.com/v1/metro/matters/4729
 2018-05-24 | http://webapi.legistar.com/v1/metro/matters/4862
 2018-05-24 | http://webapi.legistar.com/v1/metro/matters/4945
 2018-05-24 | http://webapi.legistar.com/v1/metro/matters/4897
 2018-05-24 | http://webapi.legistar.com/v1/metro/matters/4879
 2018-05-24 | http://webapi.legistar.com/v1/metro/matters/4759
 2018-05-24 | http://webapi.legistar.com/v1/metro/matters/4984
 2018-05-24 | http://webapi.legistar.com/v1/metro/matters/4840
 2018-05-24 | http://webapi.legistar.com/v1/metro/matters/4600
 2018-05-24 | http://webapi.legistar.com/v1/metro/matters/4861
 2018-05-24 | http://webapi.legistar.com/v1/metro/matters/4920
 2018-05-24 | http://webapi.legistar.com/v1/metro/matters/4847
 2018-05-24 | http://webapi.legistar.com/v1/metro/matters/4535
 2018-05-24 | http://webapi.legistar.com/v1/metro/matters/4950
 2018-05-24 | http://webapi.legistar.com/v1/metro/matters/4898
 2018-05-24 | http://webapi.legistar.com/v1/metro/matters/4904
 2018-05-24 | http://webapi.legistar.com/v1/metro/matters/4608
 2018-05-24 | http://webapi.legistar.com/v1/metro/matters/4882
 2018-05-24 | http://webapi.legistar.com/v1/metro/matters/5058
 2018-05-24 | http://webapi.legistar.com/v1/metro/matters/5004
 2018-05-24 | http://webapi.legistar.com/v1/metro/matters/4804
 2018-04-26 | http://webapi.legistar.com/v1/metro/matters/4880
 2018-04-26 | http://webapi.legistar.com/v1/metro/matters/4793
 2018-04-26 | http://webapi.legistar.com/v1/metro/matters/4852
 2018-04-26 | http://webapi.legistar.com/v1/metro/matters/4826
 2018-04-26 | http://webapi.legistar.com/v1/metro/matters/4750
 2018-04-26 | http://webapi.legistar.com/v1/metro/matters/4892
 2018-04-26 | http://webapi.legistar.com/v1/metro/matters/4849
 2018-04-26 | http://webapi.legistar.com/v1/metro/matters/4860
 2018-04-26 | http://webapi.legistar.com/v1/metro/matters/4604
 2018-04-26 | http://webapi.legistar.com/v1/metro/matters/4886
 2018-04-26 | http://webapi.legistar.com/v1/metro/matters/4639
 2018-04-26 | http://webapi.legistar.com/v1/metro/matters/4831
 2018-04-26 | http://webapi.legistar.com/v1/metro/matters/4928
 2018-04-26 | http://webapi.legistar.com/v1/metro/matters/4827
 2018-04-26 | http://webapi.legistar.com/v1/metro/matters/4738
 2018-04-26 | http://webapi.legistar.com/v1/metro/matters/4687
 2018-04-26 | http://webapi.legistar.com/v1/metro/matters/4839
 2018-04-26 | http://webapi.legistar.com/v1/metro/matters/4936
 2018-04-26 | http://webapi.legistar.com/v1/metro/matters/4866
 2018-04-26 | http://webapi.legistar.com/v1/metro/matters/4806
 2018-04-26 | http://webapi.legistar.com/v1/metro/matters/4927
 2018-04-26 | http://webapi.legistar.com/v1/metro/matters/4709
 2018-04-26 | http://webapi.legistar.com/v1/metro/matters/4709
 2018-04-26 | http://webapi.legistar.com/v1/metro/matters/4796
 2018-04-26 | http://webapi.legistar.com/v1/metro/matters/4746
 2018-04-26 | http://webapi.legistar.com/v1/metro/matters/4798
 2018-03-22 | http://webapi.legistar.com/v1/metro/matters/4784
 2018-03-22 | http://webapi.legistar.com/v1/metro/matters/4783
 2018-03-22 | http://webapi.legistar.com/v1/metro/matters/4856
 2018-03-22 | http://webapi.legistar.com/v1/metro/matters/4757
 2018-03-22 | http://webapi.legistar.com/v1/metro/matters/4692
 2018-03-22 | http://webapi.legistar.com/v1/metro/matters/4820
 2018-03-22 | http://webapi.legistar.com/v1/metro/matters/4739
 2018-03-22 | http://webapi.legistar.com/v1/metro/matters/4493
 2018-03-22 | http://webapi.legistar.com/v1/metro/matters/4734
 2018-03-22 | http://webapi.legistar.com/v1/metro/matters/4743
 2018-03-22 | http://webapi.legistar.com/v1/metro/matters/3631
 2018-03-22 | http://webapi.legistar.com/v1/metro/matters/4857
 2018-03-22 | http://webapi.legistar.com/v1/metro/matters/4720
 2018-03-22 | http://webapi.legistar.com/v1/metro/matters/4823
 2018-03-22 | http://webapi.legistar.com/v1/metro/matters/4704
 2018-03-22 | http://webapi.legistar.com/v1/metro/matters/4853
 2018-03-22 | http://webapi.legistar.com/v1/metro/matters/4736
 2018-03-22 | http://webapi.legistar.com/v1/metro/matters/4822
 2018-03-22 | http://webapi.legistar.com/v1/metro/matters/4748
 2018-01-25 | http://webapi.legistar.com/v1/metro/matters/4708
 2018-01-25 | http://webapi.legistar.com/v1/metro/matters/4726
 2018-01-25 | http://webapi.legistar.com/v1/metro/matters/4521
 2018-01-25 | http://webapi.legistar.com/v1/metro/matters/4574
 2018-01-25 | http://webapi.legistar.com/v1/metro/matters/4522
 2018-01-25 | http://webapi.legistar.com/v1/metro/matters/4617
 2018-01-25 | http://webapi.legistar.com/v1/metro/matters/4624
 2018-01-25 | http://webapi.legistar.com/v1/metro/matters/4546
 2018-01-25 | http://webapi.legistar.com/v1/metro/matters/4603
 2018-01-25 | http://webapi.legistar.com/v1/metro/matters/4583
 2018-01-25 | http://webapi.legistar.com/v1/metro/matters/4605
 2018-01-25 | http://webapi.legistar.com/v1/metro/matters/4612
 2018-01-25 | http://webapi.legistar.com/v1/metro/matters/4705
 2018-01-25 | http://webapi.legistar.com/v1/metro/matters/4640
 2018-01-25 | http://webapi.legistar.com/v1/metro/matters/4541
 2018-01-25 | http://webapi.legistar.com/v1/metro/matters/4545
 2018-01-25 | http://webapi.legistar.com/v1/metro/matters/4632
 2018-01-25 | http://webapi.legistar.com/v1/metro/matters/4730
 2018-01-25 | http://webapi.legistar.com/v1/metro/matters/4412
 2018-01-25 | http://webapi.legistar.com/v1/metro/matters/4523
 2018-01-25 | http://webapi.legistar.com/v1/metro/matters/4581
 2018-01-25 | http://webapi.legistar.com/v1/metro/matters/4627
 2018-01-25 | http://webapi.legistar.com/v1/metro/matters/4631
 2018-01-25 | http://webapi.legistar.com/v1/metro/matters/4728

All these actions seem to come from the same organization, the Board of Directors, i.e., they all have ocd-organization/42e23f04-de78-436a-bec5-ab240c1b977c as their related organization id.

The MatterBodyName for these entries, however, is not always the Board of Directors. So, what's going on?

The MatterHistoryActionBodyName on their respective histories page points to "Board of Directors - Regular Board Meeting." For example, http://webapi.legistar.com/v1/metro/matters/4546 has "Planning and Programming Committee" as the MatterBodyName, http://webapi.legistar.com/v1/metro/matters/4546/histories has the "Board of Directors" as the MatterHistoryActionBodyName.

Our scrapers determine the organization responsible for an action by looking at the MatterHistoryActionBodyName. This has worked well for previous legislative sessions.

@shrayshray - did something change about how Metro enters data about actions in 2018?

[reginafcompton]

@shrayshray - sure, we can disable. I can do that on Monday.

[shrayshray]

@reginafcompton I'm going to do my very best to confirm your observations and hopefully give a clear explanation! The MatterBodyName is the committee which first receives the report. The process for Board Reports is they first go to a Committee meeting, and after review, the Committee determines whether the Report should go to the full Board of Directors, at a Regular Board Meeting, for a vote. The Regular Board Meeting is where the Final Action takes place (Approved/Failed/No Action, etc.), as recorded in the Minutes History.

When Legistar was first implemented in 2015, the Minutes History was not being utilized. The Board Secretary's office has recently started going back and entering Minutes Histories on reports, starting in 2015 and working forward, chronologically.

So, in this example, you can see there's a full history recorded from "Finance, Budget and Audit Committee", to "Executive Management Committee", to "Board of Directors". The MatterBodyName should be "Finance, Budget and Audit Committee", but I'm not sure what the MatterHistoryActionBodyName holds? Is it just the final action from "Board of Directors", or are there multiple entries, one for each action recorded?

What's going on with the reports on your list, from 1/2018-5/2018? My guess is during that time the Board Secretary's Office was entering the Final Action (whatever happened in the Regular Board Meeting) consistently, but was not entering in History the actions taken in Committees prior to going to the full Board; perhaps focusing instead on getting the backlog of missing Histories entered. Though it looks like they haven't been entering Final Actions consistently since 5/2018 either. Omar is going to speak with them about this next week.

Does the above make sense otherwise?

[reginafcompton]

@shrayshray - I disabled the "Recent Activity" view for affected committees, e.g., https://boardagendas.metro.net/committee/operations-safety-and-customer-experience-committee-b28353454c39/

Your explanation makes a great deal of sense (particularly, "the Board Secretary's Office ... was not entering in History the actions taken in Committees prior to going to the full Board") - this explains why I only see histories related to the Board of Directors. Can you let me know if your suspicion is accurate?

The MatterHistoryActionBodyName holds the name of the organization responsible for the action. So, given your explanation, all board reports should have at least two actions (or "histories"): one by the committee, one by the board.

Your above cited example in the Legistar API - this has ALL the expected actions! Another example in the Legistar API - this has just the action from the board.

[shrayshray]

@hancush @jmithani Could you please also disable "Recent Activity" display on the Planning and Programming Committee, Finance Budget and Audit Committee, Executive Management Committee and Construction Committee? It looks like only the Operations, Safety ... Committee had this section disabled, but all the other committees are also showing the most recent activity as 2017.

[hancush]

@shrayshray sure thing! how would you like us to prioritize this with the existing milestone issues?

[shrayshray]

@hancush do you think you could get to it before 9/12? It would be nice to have it removed before the next round of agendas are posted, which will happen on 9/13.

[hancush]

sure thing, i can actually get on it this afternoon! will update.

[hancush]

changes are live on the staging site: https://lametro.datamade.us/committees/

if the committees all look ok to you, @shrayshray, i can deploy in the am!

[shrayshray]

@hancush looks great! yes, please deploy!

[hancush]

done! i'll leave this issue open, since we didn't address the root data issue, but i'll move it back into the icebox on our project board for now.