New Backend, Heroku Node.js

[aneesh-neelam]

I have recently ported the VITacademics backend to Node.js and have hosted it on Heroku.

The project repository is here This backend and webapp is hosted here

API reference for Vellore Campus:

Check out the sample output data for previous and current semesters in this directory

• /api/vellore/login/manual?regno=(regno): Returns the bitmap of the captcha image
• /api/vellore/login/submit?regno=(regno)&dob=(dob)&captcha=(captcha): Submits and logs into academics with the given credentials
• /api/vellore/data/first?regno=(regno): Returns the Timetable, consolidated Course Data with Attendance and Marks in JSON format
• /api/vellore/data/refresh?regno=(regno): Returns consolidated Course Data with Attendance and Marks in JSON format

  Status

• Scraping and Storing Timetable, Attendance and Marks for Vellore campus in a consolidated format is complete, output data is here
• Automated Captcha Parsing is not yet implemented, assigned this to @karthikb351
• Sharing Timetable with Friends (Token System) is not yet implemented, will do this soon
• The Backend for the Chennai campus is still under development, @biocross wants me to give this a low priority :-P
• The WebApp is also still under development, Google Polymer/Material Design :-D
• Notices System for classes not yet implemented, maybe later

I've also ensured that the attendance details are correct even if course has been dropped and taken again or if taken late into Add/Drop. I'm also successfully scraping and storing Lab and PBL marks, as you can see from the sample output data Also, added support for the new slots introduced. (H, K and Saturday)

Let me know what you guys think. @battlex2010, @vibhorvarshneya

If you guys do like it and want to use it, might I suggest that you use the beta channel of the Android app to try this out, to limit the number of users initially and also for load testing

I understand this may require a lot of rework since I have a different output data format. However, if correct attendance details and Lab, PBL marks aren't enough to convince you, know that this Heroku hosted Node.js backend shall always have full uptime (Heroku) and can handle a large number of simultaneous users, being Node.js

[saurabhsjoshi]

Thanks for this! I'll work on this on a separate branch and then we can update the app on beta channel!

Wait you guys have classes on Saturday too!! @biocross wtf

Also what exactly do you mean by "Scraping and Storing" are saving it on Mongo? How much capacity do we have on that?

And in your Timetable response can make this

"Mon": ",,5647,5087,,,,,5115,,,"

a JSON array?

Great work! :+1:

[aneesh-neelam]

We have around 512 MB on MongoLab and MongoHQ each for free. @karthikb351 has assured me that all VITacademics data can be stored in a single 512 MB MongoDB database on MongoLab itself, which I am using now. If not, we can always use a combination of MongoLab and MongoHQ.

I can definitely make the Timetable response a JSON array. Please raise the issue here. I'll work on it soon. Any other issues or bugs, please raise them there.

And yeah, just a couple of days ago new slots until 9:00 pm on weekdays and on Saturday have been introduced. However, I have written my scraper such that it will elegantly scale without any changes required in the code even if they revert back to the old timetable format or make additional slots, except on Sunday. Sunday would require additional 3 lines of code.

And thanks. Hope to see this in action. :+1:

[biocross]

This is a lot of work that Aneesh has done.

We are looking for a phased roll out, for example shifting like a 1000 users to this see how much heroku can handle.

Let me know if it'll be easy to port the android beta to this.

Saturday classes are not a priority for now.

[saurabhsjoshi]

Not easy but will need to rework most part of VITxAPI and Data Handler

Also another immediate problem I see is we will have to make everyone login again (But I think we can us /firsttime to solve this)

[aneesh-neelam]

Its '/first'

[saurabhsjoshi]

And also I won't release app to Beta until captcha less login is done.

[aneesh-neelam]

I've assigned captcha less login on Node.js to @karthikb351 I'm waiting on him to finish it as soon as possible.

[karthikb351]

Finished implementing Captcha Parsing on Node https://github.com/aneesh-neelam/VITacademics/commit/d34bea7da5ae823f811e94ce5c1cad176bbfc2be.

Aneesh should integrate it soon with the dev and rel apps

[aneesh-neelam]

It's done. Both Vellore and Chennai campuses are now fully functional, with automated Captcha Parsing. Updated Backend deployed to dev and rel.

Check updated API Documentation when porting client apps to this.