TrevorEdwards / bigredapp-android

Informational Android app for Cornell University.
https://play.google.com/store/apps/details?id=is.genki.bigredapp.android&hl=en
MIT License
14 stars 8 forks source link

Show menus for cafés by scraping the online order system #9

Closed lidavidm closed 8 years ago

lidavidm commented 9 years ago

View the source of this URL: https://cornell.webfood.com/xmlstoremenu.dca?dx=165547880&s=21&op=0

It's the menu for Goldie's.

You can find menus yourself by visiting https://cornell.webfood.com/, then choosing "Frequently Asked Questions", "Browse a Menu", and going from there. While you're doing this open the dev tools and watch the network requests - look for any request to an xmlstoremenu.dca. (I used Chrome, I'm not sure if Firefox shows requests that a Flash plugin makes)

lidavidm commented 9 years ago

Upon further investigation, the dx= parameter doesn't do anything (maybe it's the session). s= controls the store. https://cornell.webfood.com/xmlstart.dca?mr=506101224 has data listing all the store IDs. Using this, I got to the menu for Nastie's (https://cornell.webfood.com/xmlstoremenu.dca?s=1&op=0) even though the flash plugin won't let you choose that cafe.

lidavidm commented 9 years ago

Man, I thought this would be harder :/

ghost commented 9 years ago

Haha that's awesome. So I wonder if we can find the requests to order so we can put it natively in the app? I would check but too uninspired to create the Frequent Diner account atm ^_^ Also previously for general information / descriptions of the menus (though not the full menus themselves), I had used this link: http://living.sas.cornell.edu/dine/wheretoeat/ (e.g click on Nasties, scroll past calendar). It also has GPS coordinates (which would be nice for map / location stuff) and some other random details.

lidavidm commented 9 years ago

Huh...so that links to https://netnutrition.cbord.com/nn-prod/cornell, which gives nutrition info for menu items. We could scrape that as well.

ghost commented 9 years ago

Oh yeah that is awesome. Wish they would do for more of them too.. Also, I found that this is where Kevin's API is getting the menus (see source). This is interesting because when Big Red App was first made that menu link didn't exist, so I used this one, which is still running. Checking the network requests, I just see the menu_start.jspx return, which returns all the data at once, and then client side just uses javascript to get the different ones, as opposed to the first link, which has a post method (see menus.cfm in the requests). The only nice thing about the one I used to use is that it has brunches too, ha.

lidavidm commented 9 years ago

The one Kevin uses also has brunch, but doesn't always display the option.

ghost commented 9 years ago

I see.. I could have found that out in multiple places (the API, the source, the website) but somehow all 3 slipped by me, I think I got blinded by when I originally made the list of meals within the app..

mrkev commented 9 years ago

o: This looks sweet David! If I find some more free time I could try to scrape some of this.

ghost commented 9 years ago

(Just pitching in) that would be sweet ^_^

TrevorEdwards commented 8 years ago

Resolved in https://github.com/TrevorEdwards/bigredapp-android/commit/0c257165ef02dfd49bf0a12718b8a6dbd9246540 .