KeyError: '24' when generating ESPN report

pbutler312 commented 4 years ago

Running a report for an ESPN league and I get "KeyError: '24'"

Here is the output of the report generator:

Running Fantasy Football Metrics Weekly Report application from within virtual environment...
Generate report for default league? (y/n) -> y
Generate report for default week? (y/n) -> y
2020-09-11 00:34:39,595 - report.builder - INFO - 
Generating Espn Fantasy Football report with settings:
    league id: 264523
    game id: nfl
    week: selected/default
    save_data: False
    refresh_web_data: False
    playoff_prob_sims: None
    break_ties: False
    dq_ce: False
    dev_offline: False
    test: False
on Sep 11, 2020...
2020-09-11 00:34:39,601 - report.builder - INFO - Retrieving fantasy football data from Espn API...
The first week of the season is not yet complete. Are you sure you want to generate a report for an incomplete week? (y/n) -> y
Traceback (most recent call last):
  File "main.py", line 253, in <module>
    options.get("test", False))
  File "main.py", line 166, in select_league
    test=test)
  File "/Users/myuser/Documents/fantasy-football/fantasy-football-metrics-weekly-report/report/builder.py", line 116, in __init__
    dev_offline=self.dev_offline
  File "/Users/myuser/Documents/fantasy-football/fantasy-football-metrics-weekly-report/dao/utils.py", line 144, in league_data_factory
    dev_offline
  File "/Users/myuser/Documents/fantasy-football/fantasy-football-metrics-weekly-report/dao/espn.py", line 87, in __init__
    self.matchups_by_week[str(week_for_matchups)] = self.league.box_scores(int(week_for_matchups))
  File "/Users/myuser/Documents/fantasy-football/fantasy-football-metrics-weekly-report/dao/espn.py", line 524, in box_scores
    box_data = [BoxScore(matchup, pro_schedule, positional_rankings, week) for matchup in schedule]
  File "/Users/myuser/Documents/fantasy-football/fantasy-football-metrics-weekly-report/dao/espn.py", line 524, in <listcomp>
    box_data = [BoxScore(matchup, pro_schedule, positional_rankings, week) for matchup in schedule]
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/ff_espn_api/box_score.py", line 9, in __init__
    self.home_lineup = [BoxPlayer(player, pro_schedule, positional_rankings, week) for player in home_roster]
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/ff_espn_api/box_score.py", line 9, in <listcomp>
    self.home_lineup = [BoxPlayer(player, pro_schedule, positional_rankings, week) for player in home_roster]
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/ff_espn_api/box_player.py", line 25, in __init__
    self.pro_pos_rank = positional_rankings[str(player['defaultPositionId'])][str(opp_id)]
KeyError: '24'
logout
Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.

[Process completed]

uberfastman commented 4 years ago

Ah yes, I encountered that tonight as well. If you try running it on a historical ESPN league (that is, a league from a previous year), it should work without issue. I believe there is some funky stuff going on with the data being returned by ESPN's API while we're still in "week 0" so to speak. Since there are no complete weeks yet I think some data is missing.

Anyway, I was going to wait until next week to confirm it's good to go once week 1 is complete, since strictly speaking my app is meant to be run after completed weeks, not during them.

But it's good to know I'm not the only one seeing it, so I'll leave this open until I confirm that things are working right for current season ESPN leagues.

pbutler312 commented 4 years ago

Ah, gotcha. Sounds good, thanks!

pbutler312 commented 4 years ago

I decided to try and do a run against 2019 season week 16 (our championship week) and I got a similar key error (I updated season to 2019 and current_week to 16 in the config.ini):

Running Fantasy Football Metrics Weekly Report application from within virtual environment...
Generate report for default league? (y/n) -> y
Generate report for default week? (y/n) -> y
2020-09-11 16:30:59,011 - report.builder - INFO - 
Generating Espn Fantasy Football report with settings:
    league id: 264523
    game id: nfl
    week: selected/default
    save_data: False
    refresh_web_data: False
    playoff_prob_sims: None
    break_ties: False
    dq_ce: False
    dev_offline: False
    test: False
on Sep 11, 2020...
2020-09-11 16:30:59,017 - report.builder - INFO - Retrieving fantasy football data from Espn API...
Traceback (most recent call last):
  File "main.py", line 253, in <module>
    options.get("test", False))
  File "main.py", line 166, in select_league
    test=test)
  File "/Users/myuser/Documents/fantasy-football/fantasy-football-metrics-weekly-report/report/builder.py", line 116, in __init__
    dev_offline=self.dev_offline
  File "/Users/myuser/Documents/fantasy-football/fantasy-football-metrics-weekly-report/dao/utils.py", line 144, in league_data_factory
    dev_offline
  File "/Users/myuser/Documents/fantasy-football/fantasy-football-metrics-weekly-report/dao/espn.py", line 94, in __init__
    for matchup in self.matchups_by_week[str(week_for_rosters)]:
KeyError: '14'
logout
Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.

[Process completed]

pbutler312 commented 4 years ago

Just tried to run for week 1 2019 as maybe since we were in the playoffs that's why it was an "incomplete week", but now get the following errors:

Generate report for default league? (y/n) -> y
2020-09-11 16:38:28,542 - report.builder - INFO - 
Generating Espn Fantasy Football report with settings:
    league id: 264523
    game id: nfl
    week: 1
    save_data: False
    refresh_web_data: False
    playoff_prob_sims: None
    break_ties: False
    dq_ce: False
    dev_offline: False
    test: False
on Sep 11, 2020...
2020-09-11 16:38:28,542 - report.builder - INFO - Retrieving fantasy football data from Espn API...
2020-09-11 16:38:51,442 - report.builder - INFO - ...retrieved all fantasy football data from Espn API in 0:00:22.899682

2020-09-11 16:38:51,442 - report.builder - INFO - Retrieving bad boy data from http://nflarrest.com website...
2020-09-11 16:38:51,660 - urllib3.connection - WARNING - Certificate did not match expected hostname: nflarrest.com. Certificate: {'subject': ((('countryName', 'US'),), (('stateOrProvinceName', 'California'),), (('localityName', 'San Francisco'),), (('organizationName', 'GitHub, Inc.'),), (('commonName', 'www.github.com'),)), 'issuer': ((('countryName', 'US'),), (('organizationName', 'DigiCert Inc'),), (('organizationalUnitName', 'www.digicert.com'),), (('commonName', 'DigiCert SHA2 High Assurance Server CA'),)), 'version': 3, 'serialNumber': '02493E07FA9E375A2DBBC61D94430FCF', 'notBefore': 'May  6 00:00:00 2020 GMT', 'notAfter': 'Apr 14 12:00:00 2022 GMT', 'subjectAltName': (('DNS', 'www.github.com'), ('DNS', '*.github.com'), ('DNS', 'github.com'), ('DNS', '*.github.io'), ('DNS', 'github.io'), ('DNS', '*.githubusercontent.com'), ('DNS', 'githubusercontent.com')), 'OCSP': ('http://ocsp.digicert.com',), 'caIssuers': ('http://cacerts.digicert.com/DigiCertSHA2HighAssuranceServerCA.crt',), 'crlDistributionPoints': ('http://crl3.digicert.com/sha2-ha-server-g6.crl', 'http://crl4.digicert.com/sha2-ha-server-g6.crl')}
Traceback (most recent call last):
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/urllib3/connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/urllib3/connectionpool.py", line 978, in _validate_conn
    conn.connect()
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/urllib3/connection.py", line 397, in connect
    _match_hostname(cert, self.assert_hostname or server_hostname)
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/urllib3/connection.py", line 407, in _match_hostname
    match_hostname(cert, asserted_hostname)
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/ssl.py", line 323, in match_hostname
    % (hostname, ', '.join(map(repr, dnsnames))))
ssl.SSLCertVerificationError: ("hostname 'nflarrest.com' doesn't match either of 'www.github.com', '*.github.com', 'github.com', '*.github.io', 'github.io', '*.githubusercontent.com', 'githubusercontent.com'",)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/urllib3/connectionpool.py", line 727, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/urllib3/util/retry.py", line 439, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: MyHTTPSConnectionPool(host='nflarrest.com', port=443): Max retries exceeded with url: /api/v1/team/arrests/ARI (Caused by SSLError(SSLCertVerificationError("hostname 'nflarrest.com' doesn't match either of 'www.github.com', '*.github.com', 'github.com', '*.github.io', 'github.io', '*.githubusercontent.com', 'githubusercontent.com'")))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main.py", line 253, in <module>
    options.get("test", False))
  File "main.py", line 166, in select_league
    test=test)
  File "/Users/myuser/Documents/fantasy-football/fantasy-football-metrics-weekly-report/report/builder.py", line 131, in __init__
    self.bad_boy_stats = self.league.get_bad_boy_stats(self.save_data, self.dev_offline, self.refresh_web_data)
  File "/Users/myuser/Documents/fantasy-football/fantasy-football-metrics-weekly-report/dao/base.py", line 222, in get_bad_boy_stats
    refresh=refresh
  File "/Users/myuser/Documents/fantasy-football/fantasy-football-metrics-weekly-report/calculate/bad_boy_stats.py", line 70, in __init__
    response = requests.get(nfl_arrest_api_team_base_url + team_abbr, headers=headers)
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/requests/api.py", line 76, in get
    return request('get', url, params=params, **kwargs)
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/requests/adapters.py", line 514, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: MyHTTPSConnectionPool(host='nflarrest.com', port=443): Max retries exceeded with url: /api/v1/team/arrests/ARI (Caused by SSLError(SSLCertVerificationError("hostname 'nflarrest.com' doesn't match either of 'www.github.com', '*.github.com', 'github.com', '*.github.io', 'github.io', '*.githubusercontent.com', 'githubusercontent.com'")))

uberfastman commented 4 years ago

Ah, that issue is one that I just finished fixing the other day! Please pull the latest from develop and try again. The NFL Arrest API that I was using is no longer available, so I had to revert to scraping the data myself as I'd done previously.

uberfastman commented 4 years ago

Also, @pbutler312 for future reference, the app does not support fantasy playoff weeks. There are just too many options across too many platforms to realistically handle all the edge cases that would come up. Sorry about that, I wish I did!

uberfastman commented 4 years ago

@pbutler312 closing this, as it seems things are good to go with the latest for ESPN leagues! Let me know if you have any other issues.

pbutler312 commented 4 years ago

still having issues unfortunately, I now receive this error (looks like an SSL cert verification issue for nflarrest.com)

Running Fantasy Football Metrics Weekly Report application from within virtual environment...
Generate report for default league? (y/n) -> y
Generate report for default week? (y/n) -> y
2020-09-15 15:53:10,394 - report.builder - INFO - 
Generating Espn Fantasy Football report with settings:
    league id: 264523
    game id: nfl
    week: selected/default
    save_data: False
    refresh_web_data: False
    playoff_prob_sims: None
    break_ties: False
    dq_ce: False
    dev_offline: False
    test: False
on Sep 15, 2020...
2020-09-15 15:53:10,400 - report.builder - INFO - Retrieving fantasy football data from Espn API...
2020-09-15 15:53:25,089 - report.builder - INFO - ...retrieved all fantasy football data from Espn API in 0:00:14.689397

2020-09-15 15:53:25,090 - report.builder - INFO - Retrieving bad boy data from http://nflarrest.com website...
2020-09-15 15:53:25,504 - urllib3.connection - WARNING - Certificate did not match expected hostname: nflarrest.com. Certificate: {'subject': ((('countryName', 'US'),), (('stateOrProvinceName', 'California'),), (('localityName', 'San Francisco'),), (('organizationName', 'GitHub, Inc.'),), (('commonName', 'www.github.com'),)), 'issuer': ((('countryName', 'US'),), (('organizationName', 'DigiCert Inc'),), (('organizationalUnitName', 'www.digicert.com'),), (('commonName', 'DigiCert SHA2 High Assurance Server CA'),)), 'version': 3, 'serialNumber': '02493E07FA9E375A2DBBC61D94430FCF', 'notBefore': 'May  6 00:00:00 2020 GMT', 'notAfter': 'Apr 14 12:00:00 2022 GMT', 'subjectAltName': (('DNS', 'www.github.com'), ('DNS', '*.github.com'), ('DNS', 'github.com'), ('DNS', '*.github.io'), ('DNS', 'github.io'), ('DNS', '*.githubusercontent.com'), ('DNS', 'githubusercontent.com')), 'OCSP': ('http://ocsp.digicert.com',), 'caIssuers': ('http://cacerts.digicert.com/DigiCertSHA2HighAssuranceServerCA.crt',), 'crlDistributionPoints': ('http://crl3.digicert.com/sha2-ha-server-g6.crl', 'http://crl4.digicert.com/sha2-ha-server-g6.crl')}
Traceback (most recent call last):
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/urllib3/connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/urllib3/connectionpool.py", line 978, in _validate_conn
    conn.connect()
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/urllib3/connection.py", line 397, in connect
    _match_hostname(cert, self.assert_hostname or server_hostname)
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/urllib3/connection.py", line 407, in _match_hostname
    match_hostname(cert, asserted_hostname)
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/ssl.py", line 323, in match_hostname
    % (hostname, ', '.join(map(repr, dnsnames))))
ssl.SSLCertVerificationError: ("hostname 'nflarrest.com' doesn't match either of 'www.github.com', '*.github.com', 'github.com', '*.github.io', 'github.io', '*.githubusercontent.com', 'githubusercontent.com'",)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/urllib3/connectionpool.py", line 727, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/urllib3/util/retry.py", line 439, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: MyHTTPSConnectionPool(host='nflarrest.com', port=443): Max retries exceeded with url: /api/v1/team/arrests/ARI (Caused by SSLError(SSLCertVerificationError("hostname 'nflarrest.com' doesn't match either of 'www.github.com', '*.github.com', 'github.com', '*.github.io', 'github.io', '*.githubusercontent.com', 'githubusercontent.com'")))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main.py", line 253, in <module>
    options.get("test", False))
  File "main.py", line 166, in select_league
    test=test)
  File "/Users/myuser/Documents/fantasy-football/fantasy-football-metrics-weekly-report/report/builder.py", line 131, in __init__
    self.bad_boy_stats = self.league.get_bad_boy_stats(self.save_data, self.dev_offline, self.refresh_web_data)
  File "/Users/myuser/Documents/fantasy-football/fantasy-football-metrics-weekly-report/dao/base.py", line 222, in get_bad_boy_stats
    refresh=refresh
  File "/Users/myuser/Documents/fantasy-football/fantasy-football-metrics-weekly-report/calculate/bad_boy_stats.py", line 70, in __init__
    response = requests.get(nfl_arrest_api_team_base_url + team_abbr, headers=headers)
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/requests/api.py", line 76, in get
    return request('get', url, params=params, **kwargs)
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/Users/myuser/.virtualenvs/fantasy-football-metrics-weekly-report/lib/python3.7/site-packages/requests/adapters.py", line 514, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: MyHTTPSConnectionPool(host='nflarrest.com', port=443): Max retries exceeded with url: /api/v1/team/arrests/ARI (Caused by SSLError(SSLCertVerificationError("hostname 'nflarrest.com' doesn't match either of 'www.github.com', '*.github.com', 'github.com', '*.github.io', 'github.io', '*.githubusercontent.com', 'githubusercontent.com'")))
logout
Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.
Deleting expired sessions...4 completed.

[Process completed]

pbutler312 commented 4 years ago

going to actually make a new issue for the above

uberfastman / fantasy-football-metrics-weekly-report

KeyError: '24' when generating ESPN report #90