search

BurntSushi / nflgame

An API to retrieve and read NFL Game Center JSON data. It can work with real-time data, which can be used for fantasy football.
http://pdoc.burntsushi.net/nflgame
The Unlicense
1.27k stars 412 forks source link

2017 week 1 data is missing #331

Open nbkincaid opened 6 years ago

nbkincaid commented 6 years ago

It's Friday during week 2 and only the data from the week 2 Thursday game is returned in the following:

import nflgame nflgame.games(2017) [<nflgame.game.Game object at 0x11197bf50>]

Is this intended behavior, or is this an issue? Is there a way to get the week 1 data still?

Thank you!

zag2me commented 6 years ago

My Site has an auto NFL importer but its broken and I couldn't work out why, maybe this is it!

http://www.thesportsdb.com/league/4391

Hope it gets fixed.

ochawkeye commented 6 years ago

Not an issue I'm seeing.

import nflgame
games = nflgame.games(2017)
print len(games)
30

That's 15 games for week 1 + 15 games played as of Sunday night in week 2.

Try updating 2017 with update_sched.py found in your nflgame directory.

C:\Python27\Lib\site-packages\nflgame>python update_sched.py --year 2017
Last updated: 2017-09-18 01:53:15.482000
Updating (2017, PRE, 0)...
Updating (2017, PRE, 1)...
Updating (2017, PRE, 2)...
Updating (2017, PRE, 3)...
Updating (2017, PRE, 4)...
Updating (2017, REG, 1)...
Updating (2017, REG, 2)...
Updating (2017, REG, 3)...
Updating (2017, REG, 4)...
Updating (2017, REG, 5)...
Updating (2017, REG, 6)...
Updating (2017, REG, 7)...
Updating (2017, REG, 8)...
Updating (2017, REG, 9)...
Updating (2017, REG, 10)...
Updating (2017, REG, 11)...
Updating (2017, REG, 12)...
Updating (2017, REG, 13)...
Updating (2017, REG, 14)...
Updating (2017, REG, 15)...
Updating (2017, REG, 16)...
Updating (2017, REG, 17)...
Updating (2017, POST, 1)...
Updating (2017, POST, 2)...
Updating (2017, POST, 3)...
Updating (2017, POST, 4)...
zag2me commented 6 years ago

Perfect! works again now.

Did I just miss something in the documentation? I had no idea this needed to be done. Thanks again.

ochawkeye commented 6 years ago

You didn't miss anything. Changing seasons has always proven to be a bit problematic.

I'm not positive on this, but I think this mostly has to do with the fact that - by design - the schedule is not rebuilt every time update_sched is run unless explicitly told to do so. Only the current week is updated by default if necessary. So if you miss week(s) at the start of a season those weeks are not attempted to be updated unless told to specifically with --year ####.

Not sure if it would resolve this entirely, but in addition to last_updated being updated/checked in schedule.json maybe we could also include there the schedule URL that was last used to update the schedule. If a different URL was used this time than last time, then instead of only updating the week, the entire current year would be updated.

jamesyh commented 6 years ago

I'm getting a weird issue now. It worked a few days ago for week 1 of 2017 but now nflgame.games(year=2017,week=1) returns:

Traceback (most recent call last): File "", line 1, in File "c:\Users\jamhill\Documents\Python\football\lib\site-packages\nflgame__init__.py", line 229, in games return list(games_gen(year, week, home, away, kind, started)) TypeError: 'NoneType' object is not iterable

But it works fine for other years and for week 3. Any ideas for why this would be happening?

BadLuckRestaurant commented 6 years ago

same problems for me, only for this year weeks I've this error Traceback (most recent call last): File "<pyshell#10>", line 1, in nflgame.combine(nflgame.games(2017,week=3)) File "C:\Python27\lib\site-packages\nflgame__init__.py", line 229, in games return list(games_gen(year, week, home, away, kind, started)) TypeError: 'NoneType' object is not iterable

BadLuckRestaurant commented 6 years ago

running pip install git+https://github.com/BurntSushi/nflgame.git instead of pip install nflgame it works!

derek-adair commented 6 years ago

This is almost certainly an issue importing the schedule.

What do you see when you run...

import nflgame nflgame.sched.last_updated

jamesyh commented 6 years ago

nflgame.sched.last_updated datetime.datetime(2017, 10, 5, 8, 56, 8, 704000)

jamesyh commented 6 years ago

BadLuckRestaurant's solution worked. Thanks!

eshickey commented 6 years ago

I think I'm missing something here. I'm running a script and getting the following error. I can pull data from prior to 2016, but nothing from 2017. I have run the update_sched with the following results: Python 2.7.14 (v2.7.14:84471935ed, Sep 16 2017, 20:25:58) [MSC v.1500 64 bit (AMD64)] on win32 Type "copyright", "credits" or "license()" for more information.

======= RESTART: C:\Python27\Lib\site-packages\nflgame\update_sched.py ======= Last updated: 2017-11-19 14:48:41.606000

But when I run the script, I still get the following error: Traceback (most recent call last): File "C:\Python27\Week 9.py", line 3, in games = nflgame.games(2017, week=10) File "C:\Python27\lib\site-packages\nflgame__init__.py", line 229, in games return list(games_gen(year, week, home, away, kind, started)) TypeError: 'NoneType' object is not iterable

I also tried to pull the file that BadLuckResturant mentions, and get an error trying to locate the file.

ariannedee commented 6 years ago

I had tried to pip install from git and update the schedule, as others have mentioned, but I was still having issues keeping the games up to date. Here is what I did to get it working reliably on a Heroku environment using pipenv.

Solution

There is a line in nflgame/live.py that is using the post season url and not the regular season url to update schedule.json 🙀.

So in nflgame/live.py, line 73 needs to be commented out and line 68 commented in.

This issue has been addressed by someone else in the open PR #317.

I believe if you do that, then it doesn't matter that you're using an older version of schedule.json since it will be updated dynamically from the URL (so using the latest git version doesn't matter). If the file isn't updating because it says it's already up to date, you can edit the timestamp at the very bottom of schedule.json to some time before the game it is you want to update. (For me it was MIA vs TB on Nov 19, 2017 that was missing, 2017111911).

I ended up forking the repo and pipenv installing that version in edit mode instead. pipenv install -e git+https://github.com/denimandsteel/nflgame.git#egg=nflgame

After that is installed, then you can run: python path/to/nflgame/update_sched.py --year 2017 or nfldb-update --update-schedules (if you're using nfldb)

jason-tilley commented 6 years ago

Thank you ariannedee! Uncommenting and commenting the appropriate lines fixed the problem with update_sched.py. Unfortunately, I'm still having issues with update_players.py. I get the following (maybe it's related to the Jr.):

`Loading games for REG 2017 week 11 Downloading team rosters... 32/32 complete. (100.00%) Done!

There were some errors during the download. Usually this is a result of an HTTP request timing out, which means the resulting "players.json" file is probably missing some data. An appropriate solution is to re-run the script until there are no more errors (or when the errors are problems on NFL.com side.)

Could not get player info from roster row:

72 Leno, Charles, Jr. OT ACT 6'3" 306 10/9/1991 4 Boise State

Exception:

Traceback (most recent call last): File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nflgame/update_players.py", line 419, in run roster.append(meta_from_soup_row(team, row)) File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nflgame/update_players.py", line 179, in meta_from_soup_row last_name, first_name = map(lambda s: s.strip(), name.split(',')) ValueError: too many values to unpack `

Sorry, I don't know how to disable the markdown.

jason-tilley commented 6 years ago

OK, if I install using:

sudo pip install -e git+https://github.com/denimandsteel/nflgame.git#egg=nflgame

on my Mac, it seems to be working.

ariannedee commented 6 years ago

@ForTozs Yeah, that Leno, Charles Jr. issue has been a problem for a while. There are a few PRs to address it but I put a fix for that in my fork as well. Glad it worked!

(If you add a newline above the dashes for the horizontal line, then the text above won't show up as a heading. I guess it's treating it as ReStructuredText rts headings.)

nrosjat commented 6 years ago

I just installed nflgame by

pip2 install nflgame

but I guess I have to apply some changes to update the last season?

import nflgame games = nflgame.games(2017) print len(games)

16

I ran python update_sched.py --year 2017 several times, but it didn't change the schedule at all (eventhough it said "updating (2017, Pre-Post ...) ). Any ideas what I could do?