search

nusmodifications / nusmods

🏫 Official course planning platform for National University of Singapore.
https://nusmods.com
MIT License
586 stars 319 forks source link

Unify efforts for NUS initiatives #75

Closed yeesian closed 6 years ago

yeesian commented 10 years ago

Off the top of my head, we already have:

and probably some others that I have missed. It might be worth the effort to unify some of these initiatives, and to provide a consistent API. I am opening this thread to initiate discussion.

Here's a proposal about how we can proceed: as a first step, it'll be a good idea to be able to separate the view from the content. E.g. for instance,

This will make it easier for others to re-use your data (e.g. NUSMorge and ModCrasher). This means that cron jobs (e.g. those in ModMaven) should be available through NUSModsAPI; perhaps file an issue or PR?

Why go through all the effort? As ModCrasher puts it:

By building on top of NUSMods API's JSON file, we don't have to reinvent the wheel and write our own crawler and focus instead on user-facing features that make our app a pleasure to use.

More importantly, it's often difficult to identify the appropriate codebase to redirect new developers (who might be interested in contributing), much less to find successors. Having a clear and consistent organization allows for easier discovery of what's available (or not), whether it's a crawler, or a user-facing feature they wish to contribute.

Here's what such an organization might look like, with NUSModifications as the github organization:

cc: @weiqingtoh, @ahbeng, @yangshun, @ashrayjain, @yyjhao, @khankuan, @ejamesc, @fuzzie360, @infinitus, @ngzhian, @ymichael, @cgcai, @darora, @jcheng31, @yanhan, @jsyeo

yangshun commented 10 years ago

Nice aggregation :thumbsup:

ejamesc commented 10 years ago

https://docs.google.com/document/d/1u-z1ZN8DqqTOb2YTbOUEE0CqS6j8S9WjeM7P4oX-oyg/edit?usp=drive_web

@ahbeng started an effort a month ago.

fuzzie360 commented 10 years ago

Hello Yee Sian and Yang Shun. Please be aware that the stalker "python API" has no history and relational functionality. At the same time the data I have collected is not legally distributable openly. I am working on a token based API at the moment to separate the frontend and the backend.

Slightly related: I have some proposals for a steam style reviews section but that is a matter for another day

yangshun commented 10 years ago

@fuzzie360 that's partly why we didn't include Stalker in the Google doc started by @ahbeng . But it's certainly handy for "internal" use hehe.

fuzzie360 commented 10 years ago

I am thinking along the lines of "verified, anonymous" review entries which I believe is a totally clean and legal use case for a public view.

yeesian commented 10 years ago

@ejamesc: in fact, this (github) issue is the outgrowth of one of the comment threads in that document; although you're probably aware of that as well

thanks @fuzzie360 for surfacing legal issues with some of the data. It is important to bring them up early -- we do need clear policies on what should be made available, and what shouldn't -- rather than to ignore "the elephant in the room".

The bigger point about the python API is the desire to have programmatic access to the data/services, rather than through a sleek website UI.

fuzzie360 commented 10 years ago

That's nice but the third party API is really just an activedirectory/ldap abstraction. My point is "true stalker API proper" is being worked on and you might be interested in certain data mined information such "module similarity" and "cohort statistics" and etc. otherwise the IVLE LAPI is a superior alternative as it can give more information than I can collect without permission.

ashrayjain commented 10 years ago

On a side note, all the parsing and consolidating logic in ModMaven has already been ported over to NUSMods. The generated data is available through NUSModsAPI.

thebengeu commented 10 years ago

@fuzzie360 you're talking about exposing the aggregated bits, which would be legal right?

fuzzie360 commented 10 years ago

Yes. Also keep in mind that nusmods is operating in a different context because it is public. And if stalker has a "public section" then it will surely indemnify nusmods of willful malice.

thebengeu commented 10 years ago

Here's a proposal about how we can proceed: as a first step, it'll be a good idea to be able to separate the view from the content.

I think as a zeroth step, we could start by having a simple index of already existing initiatives, which would provide more visibility to interested contributors and also an avenue by which new initiatives could be promoted. @yyjhao that's part of Project Ouroboros' aims right?

I'm not so sure about unifying under various NUS* names or having too centralized an organization though - I've some concerns that the homogeneous appearance might have an effect on stifling creativity.

yeesian commented 10 years ago

Actually I'm not too serious about the various NUS* names myself, except to illustrate discoverability. Having a simple index might be a better approach.

The point of a github organization, if any, is to establish norms for promoting consistent APIs across the various services. But you're right: if it runs the risk of stifling initiatives, it might be better not to have it

thebengeu commented 10 years ago

Started off by transferring NUSMods and NUSModsAPI completely to the NUSModifications GitHub organization. (Previously the versions there were forks.)

yangshun commented 6 years ago

NUSModifications has been a decent point of unification of NUS-related projects over the past few years. Most student projects do not stand the test of time; the only ones surviving so far (to my knowledge) is NUSMods and NUSWhispers, which are already part of NUSModifications. I believe it is ok to close this issue now.