Closed asottile closed 8 years ago
This seems to get pretty far:
--- scrape.py 2016-07-25 08:13:11.552197514 -0700
+++ scrape2.py 2016-07-25 10:50:13.407914238 -0700
@@ -405,6 +405,13 @@
poke.Longitude,
disappear_ms,
))
+
+ for fort in cell.Fort:
+ if fort.LureInfo.ActivePokemonId:
+ print((
+ fort.LureInfo.ActivePokemonId,
+ fort.LureInfo.LureExpiresTimestampMs,
+ ))
if data:
print('Upserting {} pokemon'.format(len(data)))
with sqlite3.connect('database.db') as db:
I wonder if we want to store these separately from the rest of the pokemon, or maybe a column that mentions they are lured
I can't tell if this is working for me or not. If it is, there are far more lured pokemon than wild pokemon, and the whole scan slows to a crawl processing all the data. Additionally, the Lure's all seem to expire within a couple minutes. Perhaps this is just how they store some of the information so that they can update the lured pokemon periodically throughout the life of the lure? Still, something strikes me as a bit off here. Hard to say. Maybe I'll need to go test it out in the wild =)
From what I could tell there is a ton of duplicate data (the prints above show the same pokemon over and over). With some set()
I can probably reduce this by a lot so the database inserts are quicker.
@aeaston the confusion may also be that LureExpiresTimestampMs
is mislabeled. I believe that should actually be something like LuredPokemonExpiresTimestampMs
These are stored in a different way than normal pokemon (on the pokestop itself). I'd post sample data but I'm having issues logging in :)