search

PokemonGoers / PokeData

In this project you will scrape as much data as you can get about the *actual* sightings of Pokemons. As it turns out, players all around the world started reporting sightings of Pokemons and are logging them into a central repository (i.e. a database). We want to get this data so we can train our machine learning models. You will of course need to come up with other data sources not only for sightings but also for other relevant details that can be used later on as features for our machine learning algorithm (see Project B). Additional features could be air temperature during the given timestamp of sighting, location close to water, buildings or parks. Consult with Pokemon Go expert if you have such around you and come up with as many features as possible that describe a place, time and name of a sighted Pokemon. Another feature that you will implement is a twitter listener: You will use the twitter streaming API (https://dev.twitter.com/streaming/public) to listen on a specific topic (for example, the #foundPokemon hashtag). When a new tweet with that hashtag is written, an event will be fired in your application checking the details of the tweet, e.g. location, user, time stamp. Additionally, you will try to parse formatted text from the tweets to construct a new “seen” record that consequently will be added to the database. Some of the attributes of the record will be the Pokemon's name, location and the time stamp. Additional data sources (here is one: https://pkmngowiki.com/wiki/Pok%C3%A9mon) will also need to be integrated to give us more information about Pokemons e.g. what they are, what’s their relationship, what they can transform into, which attacks they can perform etc.
Apache License 2.0
9 stars 6 forks source link

status of scraper/listener #190

Closed jonas-he closed 6 years ago

jonas-he commented 8 years ago

As Niantic has recently enforced an update to the API of their game (implemented some new security/ anti reverse engineering features) all websites that worked with actual game data are out of service as of now.

If it is the plan that our listeners will be used in the future then i guess that theres little hope that they will work. The owner of fastpokemap is currently working on breaking the API, but he said that he will not release how to do that. Apart from that we could still scrape crowd-sourced websites (which sadly i cant find any reliable one atm).

sacdallago commented 8 years ago

thanks for the heads up @jonas-he , this sucks! It's good though, that the prediction guys have had enough data to come up with predictions that exceed random. Hopefully Niantic won't change the approach on that alg though :cry:

goldbergtatyana commented 8 years ago

and if they will, we'll just retrain our machin learner 👍

MatthiasBaur commented 8 years ago

Actually we won't be able to preform on a modified spawning algorithm without contemporary data on it. But since most of Niantics effort seems to go into rooting out 3rd parties' sites, I believe that won't be for some time.

sacdallago commented 8 years ago

Yup. Where's the fun for developers Niantics, WHERE?! :D