search

PokemonGoers / PokeData

In this project you will scrape as much data as you can get about the *actual* sightings of Pokemons. As it turns out, players all around the world started reporting sightings of Pokemons and are logging them into a central repository (i.e. a database). We want to get this data so we can train our machine learning models. You will of course need to come up with other data sources not only for sightings but also for other relevant details that can be used later on as features for our machine learning algorithm (see Project B). Additional features could be air temperature during the given timestamp of sighting, location close to water, buildings or parks. Consult with Pokemon Go expert if you have such around you and come up with as many features as possible that describe a place, time and name of a sighted Pokemon. Another feature that you will implement is a twitter listener: You will use the twitter streaming API (https://dev.twitter.com/streaming/public) to listen on a specific topic (for example, the #foundPokemon hashtag). When a new tweet with that hashtag is written, an event will be fired in your application checking the details of the tweet, e.g. location, user, time stamp. Additionally, you will try to parse formatted text from the tweets to construct a new “seen” record that consequently will be added to the database. Some of the attributes of the record will be the Pokemon's name, location and the time stamp. Additional data sources (here is one: https://pkmngowiki.com/wiki/Pok%C3%A9mon) will also need to be integrated to give us more information about Pokemons e.g. what they are, what’s their relationship, what they can transform into, which attacks they can perform etc.
Apache License 2.0
9 stars 6 forks source link

Empty API result #154

Closed johartl closed 8 years ago

johartl commented 8 years ago

It seems like all API requests now return an empty data object. E.g. curl http://pokedata.c4e3f8c7.svc.dockerapp.io:65014/api/pokemon/

{
    "message": "Success",
    "data": []
}
jonas-he commented 8 years ago

@johartl we are currently rebuilding DB, so at the moment the collection for the pokemon is empty. But should be fixed soon (or maybe not because we exceeded the storage limit on the DB provider 😄)

johartl commented 8 years ago

Do you know when you will be done with this approximately? Thanks for the prompt response.

sacdallago commented 8 years ago

I've just stopped the scrapers since they are just interfering with your work, I guess.

jonas-he commented 8 years ago

@johartl fixed

jonas-he commented 8 years ago

@sacdallago when did you stop the scrapers? because there was no more data written to mlab since like the evening on the day you started them (aka Wednesday)

sacdallago commented 8 years ago

Mh. I stopped then when I commented earlier. Probably once the database rises an error the scrapers stop without dying (they enter in the so called zombie-state), aka the docker container didn't restart (it only restarts on non-exit 0) and was just running on empty?

Might be worth checking, could you open an issue for this?

jonas-he commented 8 years ago

@sacdallago i think the fact that there was no more data inserted is more related to the mlab limit