Error using `ProfileWithTies` and `iterative_methods`

[dmnapolitano]

Hi!

I'm working with the data set for the 2006 Mayoral Election in Burlington, VT which you can download from here. This was an instant run-off election where voters were asked to rank their top five out of six candidates but were able to assign multiple candidates the same rank and also skip rankings.

Unfortunately, when I load the data into a ProfileWithTies object (see code below) called prof, then do instant_runoff.display(prof), I receive the following error:

Traceback (most recent call last):
  File "/Users/diane/pref_voting_test/pref_voting_test.py", line 65, in <module>
    instant_runoff.display(prof)
  File "/Users/diane/miniconda3/envs/pref_voting/lib/python3.11/site-packages/pref_voting/voting_method.py", line 117, in display
    ws = self.__call__(edata, curr_cands = curr_cands, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/diane/miniconda3/envs/pref_voting/lib/python3.11/site-packages/pref_voting/voting_method.py", line 79, in __call__
    return self.vm(edata, curr_cands=curr_cands, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/diane/miniconda3/envs/pref_voting/lib/python3.11/site-packages/pref_voting/iterative_methods.py", line 110, in instant_runoff
    return _instant_runoff_basic(profile, curr_cands = curr_cands)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/diane/miniconda3/envs/pref_voting/lib/python3.11/site-packages/pref_voting/iterative_methods.py", line 33, in _instant_runoff_basic
    winners = [c for c in candidates 
              ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/diane/miniconda3/envs/pref_voting/lib/python3.11/site-packages/pref_voting/iterative_methods.py", line 34, in <listcomp>
    if _num_rank_first(rs, rcounts, cands_to_ignore, c) >= strict_maj_size]
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/diane/miniconda3/envs/pref_voting/lib/python3.11/site-packages/numba/core/dispatcher.py", line 468, in _compile_for_args
    error_rewrite(e, 'typing')
  File "/Users/diane/miniconda3/envs/pref_voting/lib/python3.11/site-packages/numba/core/dispatcher.py", line 409, in error_rewrite
    raise e.with_traceback(None)
numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
non-precise type pyobject
During: typing of argument at /Users/diane/miniconda3/envs/pref_voting/lib/python3.11/site-packages/pref_voting/voting_method.py (282)

File "../../../../../miniconda3/envs/pref_voting/lib/python3.11/site-packages/pref_voting/voting_method.py", line 282:
def isin(arr, val):
    <source elided>

@jit(nopython=True, fastmath=True)
^ 

This error may have been caused by the following argument(s):
- argument 0: Cannot type list element type <class 'pref_voting.rankings.Ranking'>

I'm not entirely sure but I have some guesses as to where the fix needs to be made. It seems like all the various lists required by instant_runoff (and related classes) are already numpy.arrays containing numeric data in Profile but not ProfileWithTies.

But, of course, the issue could be with the way I've created the ProfileWithTies object, although it can tell me the Condorcet winner and loser and also from pref_voting.scoring_methods import borda works properly:

import pandas
from pref_voting.profiles_with_ties import ProfileWithTies

UNRANKED = 6

df = pandas.read_csv("Burlington Mayor 2006.txt", sep="\t", header=None,
                                       names=["Ward Number", "Memory card number", "Ballot number", "1st Ranking", "2nd Ranking", "3rd Ranking", "4th Ranking", "5th Ranking"])

df = df.set_index(["Ward Number", "Memory card number", "Ballot number"])

df = df.fillna("")

rank_rows = []
for (i, row) in df.iterrows():
    new_row = {}
    for col in df.columns:
        rank = int(col[0])
        for c in row[col].split("="):
            if len(c.strip()) > 0:
                new_row[c] = rank
    if len(new_row) > 0:
        rank_rows.append(pandas.Series(new_row, name=i))

rank_df = pandas.DataFrame(rank_rows).fillna(UNRANKED)
rank_df.index.names = ["Ward Number", "Memory card number", "Ballot number"]
rank_df = rank_df[sorted(rank_df.columns)]

unique_ranks = rank_df.value_counts()
rank_counts = unique_ranks.values
unique_ranks = [dict(zip(rank_df.columns, x)) for x in unique_ranks.index]

prof = ProfileWithTies(unique_ranks, rcounts=rank_counts)

This is with Python 3.11 and the latest version of pref-voting installed via pip (1.13.26).

What do you think? 🤔

Thanks, Diane

CC openjournals/joss-reviews#7020

[wesholliday]

Hi Diane,

Thanks for your message. You can use instant_runoff_for_truncated_linear_orders to analyze the Burlington election. You can see an example here:

https://github.com/voting-tools/election-analysis/blob/main/burlington_vt_2009.ipynb

Fairly soon we will implement versions of instant runoff for an arbitrary ProfileWithTies (allowing ties in the middle of a ballot, which is not allowed in real political elections), which has been a topic of recent research.

Please let us know if you have any other questions!

Best, Wes

[wesholliday]

P.S. To clarify one point, it's true that in the Burlington election, some voters did assign multiple candidates the same rank. However, election officials consider that a mistake--an "overvote"--rather than a legitimate way of ranking candidates that the IRV algorithm will use to compute the winner. So to analyze the Burlington election at https://github.com/voting-tools/election-analysis/blob/main/burlington_vt_2009.ipynb, we first process the ballots so that all legitimate ballots have the form of truncated linear orders (i.e., there are no ties, but some candidates can be unranked; all unranked candidates are in effect considered tied for last place, which is the effect of prof.use_extended_strict_preference()).

Different jurisdictions have different rules for how to handle overvotes (see, e.g., https://github.com/voting-tools/election-analysis/blob/main/alaska_2022.ipynb) and other ballot anomalies, and unfortunately it is not always easy to find the exact rules.

[dmnapolitano]

Hi Wes,

Thanks, this is all really helpful! I can confirm that instant_runoff_for_truncated_linear_orders and the examples in the notebook work here (which makes sense -- the 2006 and 2009 elections are pretty similar).

Based on this, I assumed all profiles/graphs listed in #1 worked with all rules/methods listed there. In hindsight, this was a silly assumption, but could I suggest you add a sentence or two to clarify that?

I'll close this issue since you're already working on it / it was my confusion 🙂

[wesholliday]

Hi Diane,

Yes, we will definitely clarify this--thanks!

In the meantime, if you look at the documentation for a specific voting method, it should say under "PARAMETERS" what kind of "edata" the method accepts.

Best, Wes