Feature Request: Length of Adult Service

[arbitrarypunter]

I started playing around with this but the complexity of the date ordering made my brain implode. Thought I'd run this by you, Feel free to close this straight away if you don't think its suitable for inclusion.

What I'm kind of thinking is a function, say people.adult_service_length() taking a membership ID as input and returning an integer, perhaps days.

Function would need to iterate through all roles for a member, identify which ones count for adult service, and then do something clever with the dates to identify gaps in service etc. Ultimately giving you the total length of service for that member.

This came about as I was trying to ascertain what had happened to a large group of members long service awards, some of which seemed delayed by up to 4 years. Some were because their service had gaps, others were just not awarded. While I was trying to piece together total service for some of the users in Compass, my head started to hurt between the combination of overlaps, gaps, ordering in the interface etc. It made it a difficult task but one that should be able to be automated.

Too niche? Too crazy on the date sorting front? Thanks Adam

[AA-Turner]

Open to the idea (this seems pretty small and low-level data in keeping with the ethos of this library). Immediate thoughts:

API Surface

I am not sure that this should be a method on the ci.People class.

• A lot of the API design at the moment is intended to reduce the number of outbound requests to Compass as much as possible, and I could see users doing a request to People.roles to get role info and then People.adult_service_length which would duplicate calls to the roles tab.
• This would suggest some membership_duration property on the model returned from People.roles. This might also be a sensible place to put the primary role identifier I've been discussing with @rglss.
• Alternative: we could implement a cache on the People class that redirects requests.
  • Would give more flexibility (e.g. I would be happier to implement more high-level methods)
  • Would increases memory usage, and if the People object is thrown away at the end of each request (as in the current implementation of the Compass Interface HTTP API) somewhat pointless.
  • I also am unsure how frequently multiple high-level methods would be called contemperaneously, and if this would be a wasted 'optimisation'.
• To some extent, this is a three way issue - usability & API design for end users vs limiting burden on the remote server (Compass) vs performance (speed/memory).
  • A property would be fast and memory efficient, and one request per member would take place
  • Internal cacheing would provide greater usability (and future extensibility), one request per member, but may use more memory (double?).

Implementation

• We will need to be confident of what roles afford membership duration.
• What excludes a role from membership duration? What if it was Cancelled not Closed? Does satisfactory or unsatisfactory closure have an impact on membership duration? If so, how do we get this information?
• Once we have an algorithm for inclusion/exclusion of roles, implementation will be pretty easy. Challenge here I think is actually getting detail from policy.

A

[AA-Turner]

Resources:

https://thescouts.zendesk.com/hc/en-gb/articles/360017947198-Does-service-need-to-be-continuous-to-count-towards-Length-of-Service-Awards-

All roles in Table 2 Appointments of POR with a minimum membership category of 'Member' or 'Associate Member' are eligible for Length of Service Awards. This does not cover service held in a youth role such as Scout Network, however if an eligible role was held alongside, this service would count.

https://www.scouts.org.uk/volunteers/learning-development-and-awards/awards-and-recognition/length-of-service-awards/

All roles in Table 2 Appointments of POR with a minimum membership category of 'Member' or 'Associate Member' are eligible for Length of Service Awards. All roles that are eligible count towards the length of service. This does not cover service held in a youth role including Scout Network however, if an eligible role was held alongside these roles, this service would count.

(Pretty much identical)

https://www.scouts.org.uk/por/11-awards-and-recognition-of-service/#11.8

Pretty much the same as above, doesn't cover Closed/Cancelled.

[arbitrarypunter]

That generated a lot more questions than i had even considered.

On the policy front, getting the exact specifics feels like a challenge. From here "All roles in Table 2 Appointments of POR with a minimum membership category of 'Member' or 'Associate Member' are eligible for Length of Service Awards. All roles that are eligible count towards the length of service. This does not cover service held in a youth role including Scout Network however, if an eligible role was held alongside these roles, this service would count."

As for Cancelled/Closed - i've only ever saw cancelled with no end date (you will no doubt have worked with larger data samples so this may not be the case). If no end date i imagine it won;t be counted. As for unsatisfactory, we are in to guess work now. But chances are i'm unlikely to be worrying about the length of service in that case! :-)

I don;t feel qualified to offer anything on implementation. That said your point about it being part of roles rather than people makes sense for your stated reasons.

[arbitrarypunter]

You beat me to it with some of that, thanks for taking the time to look at it!

[AA-Turner]

Compass (or the membership system generically) should be driven declarativley from policy. However a lot is clearly undocumented. A lot of work has gone into making Table 2 authoritative, which is really useful from a programmatic perspective.

I think the least harm approach is probably to include all role types except Cancelled, and present the resulting number as an upper bound, documenting the caveats we have dicussed here somewhere.

We should also do the usual OH/PVG/Network/Staff exclusions.

[AA-Turner]

Note also POR: TAP (7) https://www.scouts.org.uk/por/the-appointment-process/#7

[arbitrarypunter]

Your comments here Sound like they suit my use case perfectly. So i'd be more than happy if that was your chosen implementation and it was feasible.

[AA-Turner]

and returning an integer, perhaps days.

Days or years? Days would be easy to return as a datetime.timedelta object, but would years be more useful, given long service awards are in years? I could return a float with fractional years...

[arbitrarypunter]

You are right, years would be more useful.

On Wed, 3 Mar 2021, 20:06 Adam Turner, notifications@github.com wrote:

and returning an integer, perhaps days.

Days or years? Days would be easy to return as a datetime.timedelta object, but would years be more useful, given long service awards are in years? I could return a float with fractional years...

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/the-scouts/compass-interface-core/issues/55#issuecomment-790020244, or unsubscribe https://github.com/notifications/unsubscribe-auth/AROADZYAHCXKFZK4ASYP7OTTB2JD3ANCNFSM4YPOVE4Q .

[AA-Turner]

See #56 for a sample implementation, adding tests at the moment

[AA-Turner]

Try v0.10.0 - people.roles(...).membership_duration

[arbitrarypunter]

Upgraded to 0.10.0 to test this. You've flagged it as a breaking change. I've read through all the diffs, but i can't get to the bottom of what i need to change to get this bit working:

member_ids = hierarchy.get_unique_members(unit_level="Country", unit_id="12345678")

Exception has occurred: AttributeError 'str' object has no attribute 'unit_id' File "C:\Users\adamg\scripts\get-award-dates-both-units.py", line 86, in

member_ids = hierarchy.get_unique_members(unit_level="Country", unit_id="12345678") Any clues as to what i am doing wrong? Thanks On Wed, 3 Mar 2021 at 22:10, Adam Turner wrote: > Closed #55 > . > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > , > or unsubscribe > > . > -- Adam Goodall

[arbitrarypunter]

@AA-Turner bypassed the bit above and passed in some member_id's manually, so i could test the new functionality. Getting errors like this for each member: service = people.roles(member_id).membership_duration Exception has occurred: ValueError time data 'Clackmannanshire' does not match format '%d %b %Y' File "C:\Users\adamg\Dropbox (Personal)\Scouts\SSAS\scripts\testing.py", line 25, in service = people.roles(member_id).membership_duration

Looks like some fields may be out of order somewhere?

[AA-Turner]

Can't repro locally, do you mind emailing over your current script?

Your first issue might be if you are passing a string literal instead of an integer?

[arbitrarypunter]

script is:

  #!/usr/bin/env python
 import compass.core as ci

 if __name__ == "__main__":
    # Login to Compass
    compass_session = ci.login("username", "password")

    # Setup Compass Helpers
    hierarchy = ci.Hierarchy(compass_session)
    people = ci.People(compass_session)

    # Get all unique members from the in your hierarchy
    member_ids = hierarchy.get_unique_members(level="Country", unit_id=11964671)
    #member_ids = {123456, 654321 }

    # Loop for each member record
    for member_id in member_ids:
        # Basic personal details
        personal = people.personal(member_id)
        #service = people.roles(member_id).membership_duration

        print(
            personal.forenames,
            personal.surname,
            personal.birth_date.month,
            #service,
            sep=",",
        )

If i run it as above in 0.9.6 it works fine. If i upgrade to 0.10.0 and don;t change the script then it bombs with this error:

Exception has occurred: ValidationError
1 validation error for UnitData
unit_id
  field required (type=value_error.missing)
  File "C:\Users\adamg\Dropbox (Personal)\Scouts\SSAS\scripts\testing.py", line 13, in <module>
    member_ids = hierarchy.get_unique_members(level="Country", unit_id=11964671)

If i then amend the script to manually pass in some member id's and un-comment the 2 lines referring to service i get this error:

Exception has occurred: ValueError
time data 'Scouts @ 78th Lanarkshire 3rd Blantyre' does not match format '%d %b %Y'
  File "C:\Users\adamg\Dropbox (Personal)\Scouts\SSAS\scripts\testing.py", line 20, in <module>
    service = people.roles(member_id).membership_duration

Hope that's of some help in tracking it down. I really must get on with the python tutorials so i can help myself here! :-/

[AA-Turner]

Stupid question - have you deleted the cache JSON files after the schema change?

[AA-Turner]

I think that could be causing both issues - if so explains why I can't repro locally.

[AA-Turner]

Relates to #8 - still need to think about this. Possible ideas are to add a version tag to the cached data, implementing time-based expiry. But e.g. hierarchy doesn't change that often etc

[AA-Turner]

Re-opening as not confirmed resolved

[arbitrarypunter]

Never thought about the cache. Ok I nuked the cache, that makes issue one go away and i can get a list of member_id's. Sorry about the noise there, i'll add a hook/note to self to remove the json whenever the module is updated. Still getting the second problem however where the Role Location seems to be trying to matched to a date field.

I rolled back to 0.9.6, nuked the json again and tried returning just the roles without the .membership_duration, but get the same error: Exception has occurred: ValueError time data 'Scouts @ 78th Lanarkshire 3rd Blantyre' does not match format '%d %b %Y' File "C:\Users\adamg\Dropbox (Personal)\Scouts\SSAS\scripts\testing.py", line 20, in service = people.roles(member_id)

So doesn't look like it was a problem that was introduced by the 0.10.0 release. I hadn't tried to return roles before so not sure if it has ever worked for me.

Can i send you anything that would help?

[AA-Turner]

Could you sned the literal output of people._scraper._get_member_profile_tab(membership_num, "Roles") please? should be a fairly long string.

Other things I can think of is that the table is off-by-one, but not sure as to why that could be

[arbitrarypunter]

@AA-Turner Requested information provided by email. I think your off-by-one is a good shout, and I think its related to a column of check boxes at the start of the roles table, which doesn't seem to be ever present.

[arbitrarypunter]

off-by-one fix confirmed working in 0.10.1 - can now retrieve roles for all members of my unit. Also can return the .membership_duration now as well. Thanks very much for the speedy addition and fixes - great work as always!