PokemonGoers / PokeData

In this project you will scrape as much data as you can get about the *actual* sightings of Pokemons. As it turns out, players all around the world started reporting sightings of Pokemons and are logging them into a central repository (i.e. a database). We want to get this data so we can train our machine learning models. You will of course need to come up with other data sources not only for sightings but also for other relevant details that can be used later on as features for our machine learning algorithm (see Project B). Additional features could be air temperature during the given timestamp of sighting, location close to water, buildings or parks. Consult with Pokemon Go expert if you have such around you and come up with as many features as possible that describe a place, time and name of a sighted Pokemon. Another feature that you will implement is a twitter listener: You will use the twitter streaming API (https://dev.twitter.com/streaming/public) to listen on a specific topic (for example, the #foundPokemon hashtag). When a new tweet with that hashtag is written, an event will be fired in your application checking the details of the tweet, e.g. location, user, time stamp. Additionally, you will try to parse formatted text from the tweets to construct a new “seen” record that consequently will be added to the database. Some of the attributes of the record will be the Pokemon's name, location and the time stamp. Additional data sources (here is one: https://pkmngowiki.com/wiki/Pok%C3%A9mon) will also need to be integrated to give us more information about Pokemons e.g. what they are, what’s their relationship, what they can transform into, which attacks they can perform etc.
Apache License 2.0
9 stars 6 forks source link

POST / PUT requests #105

Closed sacdallago closed 8 years ago

sacdallago commented 8 years ago

Hey guys,

from the post on GGroups it appears that you want to open up the API to post requests. For me this is a big deal and it changes the API quite a lot... Just to give you a few points:

So I want to ask: how many post/put/delete requests do you think you will need and for what? If it turns out opening to something else then GET is not necessary, I suggest you to let it be :)

jonas-he commented 8 years ago

@sacdallago this idea came up, because it was requested that we provide methods for uploading icons for pokemon, see #72 . From my side it would be a lot easier if they just send us a ZIP and we integrate it into the DB.

sacdallago commented 8 years ago

Yes. Please do that

jonas-he commented 8 years ago

Requested icons from PokeMap group.

swathi-ssunder commented 8 years ago

@sacdallago - Currently, we only have the option of searching by one field in the apis. Example: http://pokedata.c4e3f8c7.svc.dockerapp.io:65014/api/pokemon/id/1 gets the Pokemon with id 1. There is the option to filter by other fields such as type, gender etc.

However, we need to support filtering by multiple parameters.

  1. We can do this using a GET request with Query parameters like /api/pokemon?gender=:gender&type-:type . But this may not be ideal when there are too many parameters and the query string gets too long.
  2. We can use a POST request of the form /api/pokemon/search and the parameters sent as key-value pairs in the Request body. Also, since the request would still be read-only, there should be no associated issues.

We would prefer the second approach, but would like to know if there are any other concerns regarding this.

sacdallago commented 8 years ago

This is an understandable problem.

The standard answer would be no. (No as in GET is to get, POST is to update/create/ eventually delete). I would also be for the no.

I know that in this case the POST would not operate in CUD of CRUD, but still: it is semantically wrong and conceptually misleading.

I am not saying you must not do it, I'm just saying: try to convince me, if you can :D I can convince you that you can't! :D

Would you sacrifice sticking to standards to avoid having long URLs that in most cases are anyway machine generated? Additionally: I don't get the claim:

But this may not be ideal when there are too many parameters and the query string gets too long.

Why wouldn't it be ideal? :). It's the way it is :)

Also: the query object in express is anyway an object, similar to the body of a post request, so at the end of the day there's no real huge difference there if you map query attribute to object/schema attribute. You would need to perform validation checks (attribute existence, attribute type,..) on either the body or the query, so again: no difference.

If the concern is the URL length do a boundary test: what is the worst case? URLs can be of 2k char so I guess it's hard to reach even with 20 filter queries :)

If finally the problem is encoding (URL encoding), express does automatic decoding of variables inside the query string (as long as they conform to standards).

If you can beat this: http://cell.dallago.us/ppi?p=P04637&p=P05549&p=P09874&p=P35232&p=P19525&p=P27695&p=P28482&p=P17252&p=P04271&p=Q06609&p=P27694&p=P31350&p=Q15843&p=P32745&p=Q14186&p=Q01094&p=P32780&p=P00519&p=P04637&p=P09884&p=P49459&p=Q07817&p=P20226&p=Q92793&p=P51587&p=P35354&p=P41235&p=P55199&p=Q16637&p=P48729&p=P38646&p=O15111&p=P48730&p=P78527&p=Q13526&p=P45983&p=P63104&p=Q99816&p=Q9UK53&p=P51959&p=Q12824&p=Q05086&p=Q99973&p=P60484&p=P27361&p=Q9UQR1&p=O15350&p=Q99608&p=Q99986&p=Q15831&p=O43474&p=Q93009&p=P51948&p=O15151&p=Q14974&p=P61086&p=P45984&p=P53779&p=Q9H4B4&p=Q99873&p=Q66K89&p=O14757&p=P18847&p=Q9UER7&p=P51532&p=Q9H3D4&p=Q16665&p=O75569&p=Q9UNH5&p=O60729&p=Q13330&p=Q9NVC6&p=O95997&p=Q15648&p=P61201&p=P54132&p=Q14191&p=O95967&p=Q7LG56&p=Q92994&p=Q15573&p=Q53T94&p=Q15572&p=Q13043&p=P49841&p=Q9NZC7&p=Q12888&p=P61981&p=Q9H2X6&p=P19544&p=Q8N114&p=Q8IWT3&p=Q13315&p=Q86Z02&p=Q9BVP2&p=Q9HBM6&p=P08238&p=Q92831&p=Q92905&p=Q8N2W9&p=Q9UNS2&p=P10828&p=Q96PM5&p=Q13535&p=P49642&p=P63279&p=P51946&p=Q12873&p=Q8WUF5&p=Q96A56&p=Q96ST3&p=Q9UNL4&p=Q03701&p=Q9BT78&p=Q99856&p=Q9UKY1&p=Q9NS56&p=Q13625&p=Q13227&p=Q96S44&p=Q9GZV1&p=Q9BTV7&p=Q13098&p=Q9UM63&p=O75925&p=Q8WTS6&p=Q14676&p=Q8WYH8&p=Q8TDN4&p=Q8N9N5&p=Q7L5N1&p=Q9UBW8&p=Q99627&p=P26447&p=P38398&p=P29590&p=P07384&p=P68400&p=P67870&p=Q99728&p=P17844&p=Q15392&p=P24941&p=Q8NHY2&p=P67775&p=O94776&p=Q92754&p=Q9Y3D8&p=P28360&p=P29034&p=Q99612&p=P12429&p=P60953&p=Q03013&p=P11274&p=P00491&p=Q15181&p=P63162&p=P04183&p=P17028&p=P21673&p=Q16363&p=P50453&p=O14641&p=Q15102&p=Q13190&p=P42773&p=P55774&p=P20042&p=O00231&p=P36405&p=Q14061&p=P48643&p=Q96M61&p=P49888&p=P84090&p=P04792&p=O95376&p=Q9H0Q3&p=Q15013&p=Q9C0J8&p=Q9BX70&p=Q99547&p=P68133&p=P20023&p=P29965&p=Q15796&p=P84022&p=O15126&p=P15559&p=Q5JVS0&p=P41218&p=Q00535&p=P20248&p=Q02447&p=P50750&p=P08047&p=Q09472&p=Q00987&p=Q05397&p=P05109&p=P20338&p=Q9BWC9&p=Q9BQD7&p=Q8NA92&p=Q8TBK6&p=O43715&p=P11387&p=P11388&p=Q02880&p=Q92993&p=P53350&p=Q8IW41&p=P30419&p=O60551&p=Q86TM6&p=P06748&p=Q13547&p=Q92769&p=O15379&p=P18146&p=O96017&p=P55209&p=A8MT69&p=P23511&p=P25208&p=P22736&p=P63165&p=Q00005&p=P36873&p=Q9NRG4&p=O43257&p=Q86XK2&p=Q7Z6Z7&p=Q96KB5&p=Q16611&p=P13639&p=P29320&p=P42680&p=Q06187&p=P14174&p=P03372&p=P19447&p=Q03468&p=P04150&p=P07900&p=P42858&p=Q16666&p=P10415&p=P67809&p=P30307&p=P62136&p=P09429&p=P26583&p=P25963&p=P19338&p=Q9NXR7&p=P46736&p=P31947&p=P38936&p=Q86X55&p=Q7Z2E3&p=P18887&p=P62913&p=Q92922&p=Q9H422&p=Q13569&p=P16220&p=P35251&p=P18074&p=P60510&p=P21675&p=O43422&p=P50613&p=P11142&p=Q00688&p=Q8N488&p=Q13155&p=P09936&p=O15519&p=Q96J02&p=P00374&p=Q8IX12&p=O75493&p=P61289&p=P34932&p=P49757&p=P50454&p=Q9ULJ6&p=Q8NF64&p=Q15326&p=O95571&p=Q9HCK8&p=P53355&p=P10599&p=P16083&p=Q96G74&p=P51784&p=Q53GS9&p=Q8TAF3&p=Q9UM07&p=P62328&p=P51813&p=P17676&p=Q8N9B5&p=Q9BUJ2&p=Q96PM9&p=Q15759&p=P62081&p=P55060&p=Q14999&p=P02749&p=P13497&p=O75928&p=Q5VTR2&p=O75150&p=O14503&p=O14576&p=O14920&p=O14980&p=P04731&p=P10275&p=P11021&p=P30153&p=P34931&p=P46821&p=P63151&p=Q13362&p=Q96GM8&p=Q9BY41&p=Q9UBE0&p=Q9H160&p=Q13972&p=Q9Y675&p=Q16594&p=P49848&p=O60936&p=P49674&p=P25490&p=Q96Q15&p=P61088&p=P42226&p=Q9BV47&p=Q9NVN8&p=O00255&p=P61964&p=Q01484&p=Q15291&p=Q15596&p=P11473&p=P01106&p=Q99966&p=P06400&p=Q9HBJ7&p=P06703&p=P62258&p=Q96KQ4&p=O95863&p=Q9UHC7&p=P13693&p=P23297&p=Q8TAQ5&p=P61978&p=O43524&p=Q15672&p=P18283&p=O60285&p=Q86WX3&p=Q8WWV6&p=O75190&p=P33981&p=P11766&p=P54725&p=Q8N680&p=P24844&p=P34925&p=P42704&p=Q9BZG8&p=Q13283&p=Q96KQ7&p=O75832&p=O60675&p=Q9UL18&p=Q9H0F5&p=Q9Y3T9&p=O00458&p=O75486&p=Q5VWQ8&p=Q9UHR4&p=Q96RL1&p=O15164&p=Q9Y5A7&p=O95071&p=P46777&p=Q96BD6&p=Q9P2P5&p=P26358&p=Q9HCI7&p=P62195&p=Q96K58&p=Q9Y4A5&p=Q5U5Z8&p=P49336&p=P12755&p=Q96QH2&p=Q9BYW2&p=Q969K3&p=O95365&p=Q14694&p=O43189&p=Q9Y294&p=Q96EB6&p=Q9Y297&p=P42345&p=P12956&p=Q86Y07&p=Q9Y468&p=Q6EMB2&p=Q12962&p=O75478&p=Q7Z4G4&p=P43246&p=Q8TDY2&p=Q92925&p=Q06945&p=Q2M296&p=Q13029&p=Q9BZE4&p=P43356&p=P16104&p=Q8IZT6&p=O95155&p=O75376&p=P40337&p=Q13363&p=Q13503&p=P62829&p=Q8WZ73&p=Q9UKV8&p=Q96GD4&p=Q96J94&p=P55854&p=Q9NZI8&p=P40925&p=O75529&p=O15304&p=Q9H9B1&p=Q9H0E9&p=O00505&p=Q8NC51&p=P17096&p=Q92830&p=Q9UKV0&p=Q9UN86&p=O94864&p=O14867&p=P24522&p=P61224&p=Q9H9J4&p=Q15424&p=P20073&p=Q14151&p=Q92890&p=Q9BQA5&p=Q96SB4&p=P18754&p=P52701&p=P08069&p=P15036&p=P14921&p=P55036&p=O60341&p=Q7Z392&p=Q96BN2&p=Q13263&p=O14497&p=Q9Y6Q9&p=P06493&p=Q12980&p=Q96FW1&p=Q9UQL6&p=O14965&p=P61254&p=Q15788&p=Q15466&p=O96019&p=Q7Z3B3&p=Q6PCD5&p=P17980&p=P01019&p=Q99466&p=O75182&p=Q6P3S6&p=O95251&p=O75528&p=Q49MG5&p=Q9NR12&p=Q92794&p=Q9H4L4&p=Q9BVJ6&p=Q01105&p=Q16539&p=Q9UNE7&p=P23396&p=P0CG47&p=Q96PU4&p=O60763&p=O60678&p=Q9HCE7&p=O15169&p=Q9H7Z6&p=P61956&p=P14373&p=P36956&p=Q9NPI1&p=Q92841&p=Q9UBF1&p=Q12772&p=P10809&p=Q8WVJ9&p=P30405&p=Q13216&p=P51668&p=P46783&p=Q68DV7&p=P61077&p=Q13572&p=Q9Y385&p=O43293&p=O15105&p=Q9UNF1&p=Q6B0I6&p=Q13233&p=P07948&p=Q15303&p=Q15170&p=O94992&p=Q9UBK2&p=Q01664&p=Q9Y222&p=Q9Y4B6&p=P03950&p=P01112&p=O15205&p=Q15797&p=Q15717&p=Q05655&p=Q9UKT5&p=Q15714&p=P62837&p=P12004&p=Q969V5&p=P49792&p=Q76N89&p=O14737&p=Q16881&p=P61081&p=Q15542&p=Q99497&p=P02511&p=Q07955&p=P63146&p=Q9H0W5&p=Q96GM5&p=Q15139&p=O15524&p=Q01844&p=Q9NZJ0&p=Q13619&p=Q9NRC8&p=Q07812&p=P04406&p=Q14790&p=Q9UPR3&p=P63208&p=Q9UJ04&p=P19784&p=Q9UPZ9&p=P82650&p=Q13077&p=Q13472&p=Q13509&p=O95714&p=Q13200&p=O95816&p=Q13885&p=P07437&p=Q07666&p=O75970&p=P46379&p=P49761&p=Q53H96&p=Q9NP81&p=Q96RR1&p=Q96S59&p=Q96RU2&p=P27348&p=Q587J7&p=Q92540&p=Q8TEW0&p=Q96JN8&p=Q9BRJ7&p=Q5T9A4&p=Q9NVI7&p=Q9NTK5&p=P40227&p=Q9NZC9&p=P49902&p=Q8N3Y1&p=Q8N3R9&p=P49368&p=O43847&p=O15355&p=P62191&p=O60568&p=Q8NBJ5&p=Q9H1Z9&p=P17987&p=Q9Y3P9&p=Q9H6S0&p=P78371&p=P50991&p=P50990&p=O95758&p=Q99684&p=Q99832&p=Q16401&p=P62993&p=P62979&p=P62877&p=Q9HCM9&p=Q06587&p=P15941&p=Q9Y2Z0&p=Q9H3M7&p=Q504Q3&p=P07355&p=Q9H8V3&p=Q5T447&p=Q92759&p=P35226&p=Q9BVS4&p=P55072&p=Q99496&p=Q9BQ15&p=Q9BZR9&p=Q9BQG0&p=Q9H2S5&p=Q9BVI0&p=Q9BVK6&p=P62854&p=Q96L91&p=P21580&p=Q5S007&p=O75164&p=Q6UXN9&p=Q9NRR4&p=Q9P0U4&p=O75147&p=O15047&p=Q9UBL3&p=P68104&p=O43623&p=Q9Y676&p=Q9Y2Q9&p=O60783&p=Q14764&p=Q9Y3D9&p=Q9Y399&p=Q9UNX3&p=Q16531&p=Q8NFQ8&p=O95229&p=Q96B26&p=Q02952&p=O60573&p=Q13620&p=Q9UKN8&p=Q9Y5Q9&p=Q9UBL6&p=P50914&p=O43290&p=O00425&p=P25685&p=Q13435&p=Q99848&p=O43819&p=O60343&p=Q15008&p=Q96PE2&p=P25705&p=Q96N67&p=P11940&p=Q3ZCQ8&p=Q6Y7W6&p=Q9Y2S7&p=P31689&p=O43837&p=P52597&p=P31943&p=P22626&p=Q00839&p=P09651&p=P84074&p=Q8IWR1&p=Q9BRQ8&p=Q53ET0&p=Q9ULT8&p=Q9BQE3&p=P46100&p=P22087&p=O60488&p=P02794&p=Q9BQ67&p=P27708&p=P53801&p=P35249&p=P40938&p=Q09028&p=Q16576&p=Q14257&p=Q8IY67&p=Q07820&p=Q13164&p=Q08209&p=P63098&p=P62851&p=P42677&p=P62847&p=P08708&p=P62263&p=P62841&p=P46781&p=P62241&p=Q96M02&p=Q8N8S7&p=P05388&p=P05387&p=P61247&p=P15880&p=P62753&p=P18124&p=Q02878&p=P36578&p=P39023&p=P30050&p=P62424&p=P62917&p=P26373&p=P61313&p=P62750&p=P46778&p=Q02543&p=P84098&p=Q07020&p=P62899&p=P83731&p=P61353&p=P62888&p=P61513&p=Q96EI5&p=Q9UBE8&p=Q96P16&p=Q53EZ4&p=Q96EY7&p=Q8TEB7&p=Q96A35&p=P62906&p=Q9UKJ3&p=Q92552&p=P52272&p=Q14684&p=O94952&p=Q15024&p=Q5T4S7&p=Q9Y520&p=P49720&p=Q92843&p=Q9HBE1&p=P55083&p=P22061&p=P46019&p=Q93100&p=P15735&p=Q9BYJ9&p=Q7L0Y3&p=Q15365&p=Q13573&p=Q9NYK5&p=P60709&p=P62736&p=Q12979&p=O75340&p=Q96L21&p=Q99536&p=Q9NS37&p=Q8NDX9&p=P68371&p=O60884&p=P61962&p=Q07912&p=Q13950&p=P48634&p=Q86W92&p=P78406&p=O14654&p=O15055&p=P46940&p=Q13501&p=Q96DU7&p=Q6NZ67&p=Q96DF8&p=Q7KZ85&p=P82675&p=P82933&p=Q8IXM3&p=Q96DV4&p=P62318&p=P82663&p=Q8TD20&p=P50552&p=Q99615&p=P68366&p=P23258&p=Q05516&p=Q13724&p=Q71U36&p=Q01831&p=P02786&p=Q96BR9&p=Q8NBN7&p=Q9Y6M4&p=Q16589&p=O60566&p=P31749&p=Q13976&p=Q9Y6P5&p=O15297&p=P07996&p=P58004&p=O75293&p=P05121&p=P78368&p=P27701&p=P17936&p=Q658P3&p=O14763&p=P11802&p=Q13794&p=O95819&p=O95257&p=O15264&p=Q9NSU2&p=P54760&p=P12931&p=Q9NY65&p=Q9UHB6&p=O00571&p=Q16643&p=P09497&p=P09496&p=Q00610&p=Q9UII4&p=O60841&p=Q14498&p=Q9UL15&p=Q07065&p=Q9BTZ2&p=Q562R1&p=P61221&p=Q9Y2W1&p=P05141&p=Q8ND56&p=Q6GPH4&p=P55084&p=P17066&p=P54652&p=P14866&p=Q92985&p=O15391&p=Q969P6&p=Q9UM11&p=P21333&p=P40121&p=P47756&p=Q05682&p=P62879&p=P41182&p=P06396&p=P78347&p=Q9GZQ8&p=Q04206&p=P98179&p=P39019&p=P62269&p=P62249&p=P62277&p=Q9BQA1&p=Q13247&p=Q9NR11&p=P62701&p=P46779&p=Q8TAT5&p=Q99576&p=Q8N5Z5&p=O00746&p=Q15018&p=Q15746&p=Q9UM54&p=O00159&p=O14974&p=P35579&p=Q9ULV4&p=P02545&p=P61626&p=Q86T24&p=Q13049&p=O14744&p=Q86V81&p=Q9H444&p=P15884&p=Q9P0U3&p=Q9ULC6&p=Q96AH0&p=O15550&p=P08670&p=P09493&p=P67936&p=P06753&p=P08218&p=O60481&p=Q9UKG1&p=P46108&p=Q9Y618&p=P16885 which actually gerates a problem, then we can start talking POST :D

swathi-ssunder commented 8 years ago

GET apis being implemented for search by multiple parameters. Refer #131.