search

PokemonGoers / PokeData

In this project you will scrape as much data as you can get about the *actual* sightings of Pokemons. As it turns out, players all around the world started reporting sightings of Pokemons and are logging them into a central repository (i.e. a database). We want to get this data so we can train our machine learning models. You will of course need to come up with other data sources not only for sightings but also for other relevant details that can be used later on as features for our machine learning algorithm (see Project B). Additional features could be air temperature during the given timestamp of sighting, location close to water, buildings or parks. Consult with Pokemon Go expert if you have such around you and come up with as many features as possible that describe a place, time and name of a sighted Pokemon. Another feature that you will implement is a twitter listener: You will use the twitter streaming API (https://dev.twitter.com/streaming/public) to listen on a specific topic (for example, the #foundPokemon hashtag). When a new tweet with that hashtag is written, an event will be fired in your application checking the details of the tweet, e.g. location, user, time stamp. Additionally, you will try to parse formatted text from the tweets to construct a new “seen” record that consequently will be added to the database. Some of the attributes of the record will be the Pokemon's name, location and the time stamp. Additional data sources (here is one: https://pkmngowiki.com/wiki/Pok%C3%A9mon) will also need to be integrated to give us more information about Pokemons e.g. what they are, what’s their relationship, what they can transform into, which attacks they can perform etc.
Apache License 2.0
9 stars 6 forks source link

Integration of HashPokemonGo (Project C) #162

Closed sockeqwe closed 8 years ago

sockeqwe commented 8 years ago

Adds:

samitsv commented 8 years ago

@sockeqwe could you look into this problem, i tried to run it with production mongodbUrl, it contains the username and password in the mongodb url, so when I ran it, it gave me 

Error: More than 1 database name in URL
    at module.exports (/Users/samit/PokeData/node_modules/hashpokemongo/node_modules/mongodb/lib/url_parser.js:128:13)
    at connect (/Users/samit/PokeData/node_modules/hashpokemongo/node_modules/mongodb/lib/mongo_client.js:290:16)
    at Function.MongoClient.connect (/Users/samit/PokeData/node_modules/hashpokemongo/node_modules/mongodb/lib/mongo_client.js:114:3)
    at Object.module.exports.start.MongoClient.connect.Rx.Observable.interval.map.counter.do.pokemon.flatMap.pokemon.flatMap.emptyArrayObservable.flatMap.tweets.filter.tweet.map [as start] (/Users/samit/PokeData/node_modules/hashpokemongo/sentimentTwitter/TwitterSentimentsMiner.js:22:17)
    at Object.module.exports.insertToDb (/Users/samit/PokeData/app/controllers/filler/hashPokemonGo.js:34:40)
    at NativeConnection.<anonymous> (/Users/samit/PokeData/scripts/listen.js:33:18)
    at emitNone (events.js:67:13)
    at NativeConnection.emit (events.js:166:7)
    at open (/Users/samit/PokeData/node_modules/mongoose/lib/connection.js:516:11)
    at /Users/samit/PokeData/node_modules/mongoose/lib/connection.js:522:7
    at /Users/samit/PokeData/node_modules/mongoose/node_modules/mongodb/lib/db.js:1504:5
    at handleCallback (/Users/samit/PokeData/node_modules/mongoose/node_modules/mongodb/lib/utils.js:96:12)
    at _callback (/Users/samit/PokeData/node_modules/mongoose/node_modules/mongodb/lib/db.js:1420:5)
    at /Users/samit/PokeData/node_modules/mongoose/node_modules/mongodb/lib/db.js:1463:7
    at /Users/samit/PokeData/node_modules/mongoose/node_modules/mongodb/node_modules/mongodb-core/lib/topologies/server.js:1416:5
    at finish (/Users/samit/PokeData/node_modules/mongoose/node_modules/mongodb/node_modules/mongodb-core/lib/auth/scram.js:157:16)
samitsv commented 8 years ago

@sockeqwe any progress with the database url? did you try to connect with remote database?

sockeqwe commented 8 years ago

Not yet, will try that in the evening

samitsv notifications@github.com schrieb am Fr., 23. Sep. 2016, 15:32:

@sockeqwe https://github.com/sockeqwe any progress with the database url? did you try to connect with remote database?

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/PokemonGoers/PokeData/pull/162#issuecomment-249193680, or mute the thread https://github.com/notifications/unsubscribe-auth/AAjnrtF8lnuYpXPSKgC0KbM8nwpMl9qlks5qs9TUgaJpZM4KC6JQ .

sockeqwe commented 8 years ago

@samitsv So I set up a demo database on mlab and I am able to connect to. The url mongodb://username:password@ds023435.mlab.com:23435/fooDatabase works.

So compared to your database url: mongodb://username:password@xxxxx.mlab.com:63725/pokemongo/PokeData you have an additional /pokemongo/DatabaseName whereas I have only /DatabaseName in my test (without /pokemongo).

Are your sure your url is correct?

sockeqwe commented 8 years ago

@samitsv Ok, so I have found the bug. The issue is that the url you retrun from getMongoDbUrl() is wrong:

mongodb://username:password@xxxxx.mlab.com:63725/pokemongo/PokeData (currently used url, returned from getMongoDbUrl() )

should be

mongodb://username:password@xxxxx.mlab.com:63725/pokemongo (connecting to this url works)

getMongoDbUrl() is always appending /PokeData (but shouldn't, at least not in production)

samitsv commented 8 years ago

@sockeqwe thanks .. corrected it and now it works fine

sockeqwe commented 8 years ago

thanks for your help!

swathi-ssunder commented 8 years ago

@sacdallago - After this merge, the API link is not available. The builds (both Travis and Docker) seemed fine and completed without errors.

swathi-ssunder commented 8 years ago

@sockeqwe - In the file, HasMap is required.. However, the module name is hashmap. This might fail on case-sensitive file systems and in fact it does fail. Could you please correct this?