Dependency graph based online learning system

[KOLANICH]

Project description

1 Create a model of knowledge of all areas as a directed graph of dependencies of concepts, and these concepts should be as small and as reusable as possible. For example for understanding the topic physics:phonons you should understand physics:normal modes of lattice, physics:secondary quantization, physics:lattice potential energy power expansion first. To understand physics:normal modes of lattice you need understanding physics:Brillouin zone and physics:lattice potential energy power expansion again first. For understanding physics:Brillouin zone you should understand signal processing:aliasing first. It's not physics but it is closely relevant to the topic! There are lots of such kind interlinks between areas, for example you need math:windowed Fourier transform to understand physics:Heisenberg uncertainty relation or physics:natural line width or physics:time dependent perturbation theory or physics:single slit difraction.

2 When we have such a graph we can assign to each concept a list of URIs to web resources devoted to such concepts, such as video lectures like the ones on Coursera or Khan Academy or books or lectures in PDFs (many lecturers in the universities have put their lectures and books online for their students, but they are usable and downloadable for anyone else). I guess the creation of list of such URIs can be partially automated.

3 A person fills own profile: marks the learned topics. Than he selects the topics he wanna learn from the graph and the system computes the path to the roots, removes the concepts he already knows and gives him the grap of topics he needs to learn with the links to the resources. When he learns a topic he marks it as learned.

Such process should give a better understanding than using a single source of lectures because every lecturer omits some important topics and links.

[Haroenv]

This seems like too hard of a problem, since you’d not be able to quantise all you know

[KOLANICH]

I don't understand what you mean. Usually each resource (for example a video lecture) has own topic. In rare cases a resource have more than a single topic. But it should not be a problem if the system gives a user a link to the resource with more than one topic.

[Haroenv]

Yeah but you don’t learn everything from the same platform, and often it’s not strictly necessary to understand everything something builds on, but you can understand something from different directions. Anyway, I was just giving my opinion, don’t hesitate trying it :p

[KOLANICH]

I usually dump here random ideas (as you see, I generate them constantly) I personally have no plans (because I have no resources to implement all my ideas) to implement, but which may be useful if someone have implemented them.

[FurkanKambay]

Microsoft has "learning paths": https://azure.microsoft.com/en-us/documentation/learning-paths/appservice-webapps/ https://azure.microsoft.com/en-us/documentation/learning-paths/appservice-mobileapps/ Like these two, there are others here: https://azure.microsoft.com/en-us/documentation/learning-paths/

I don't know if this strategy can be applied to everything or not, but it'd actually be pretty good. Some things aren't related as "must learn" but kind of as "learning this will help better understanding this topic". But I'm sure these type of things can also be implemented.

I personally like it.

[knoxxs]

This is also one of my most precious idea from about 10 years now. Have a lot of input on it. Let see if enough people are ready to give it a try then would love to discuss further.

[sks4903440]

I always feel the need for such a system whenever I start learning a new topic. Would love to contribute

[gtestault]

What about instead of a dependency graph we use a concept graph. In such a graph we could just link topics that are related with edges, it is much easier to build since anyone can contribute to the knowledge graph (without being a field expert).

An edge between each concept in this graph could have a dynamic cost. When someone learns a new concept he marks which learning paths (edges) where the most useful for his learning. If a learning path is useful for a lot of people its cost will eventually become very low.

Finally we could include User prior knowledge by running a pathing algorithm with a heuristic function on the graph. A possible heuristic function could be: "User ranks topics in an ordered list with topics he thinks he will learn the fastest."

Here is a link to an interesting discussion which highlights problems of building a knowledge dependency graph.

[KOLANICH]

An edge between each concept in this graph could have a dynamic cost. When someone learns a new concept he marks which learning paths (edges) where the most useful for his learning. If a learning path is useful for a lot of people its cost will eventually become very low.

Ant colony optimization algorithm with people as ants. Nice idea, but with big challenges.

What if someone wiling to ruin the website just for fun (I have done similar things for JS-based browser games highscore tables several times long ago by just sending the best highscore+small number with my nickname and only once I have encounter a game doing the full simulation on server side) wrote a bot voting automatically for useless paths? It is not Wikipedia, a user leaves no more than a vote, I guess we cannot detect and revert this kind of abuse at all.

I guess it may be better to derive this this info from sequence of sets of university lectures scattered around the web, but I don't know now how to do that. The naïve idea is to detect topics in each paragraph with LDA, think about it as about learning path and apply to the graph in the following way: ith topic assummes all preceding toopics as dependencies, but this would give us tonns of false positives and make a mess. A better approach is to try detect links between topics from text. There are 2 ways.

• Clear references like formula, image and topics headers references.
• unclear references via usage of terms.

But initially I haven't thought about this graph as about machine learning task and assummed that the whole graph would be created manually by experts.

Finally we could include User prior knowledge by running a pathing algorithm with a heuristic function on the graph. A possible heuristic function could be: "User ranks topics in an ordered list with topics he thinks he will learn the fastest."

Hmm, don't know. My initial vision of this system was as a teacher, guide and superviser. I say to the software what I want to learn and the software guides me through learning. User's guesses may be irrelevant since the user is just learning, if he has relevant guesses, probably he has already learned the topic.

In such a graph we could just link topics that are related with edges, it is much easier to build since anyone can contribute to the knowledge graph (without being a field expert).

If you state the task this way we may assume that all topics are linked to each ones but with different weight, which would give us a huge dense matrix.

[deklanw]

This already exists and is somewhat developed for machine learning topics (and a bit for related fields). https://www.metacademy.org/ https://github.com/metacademy

It's not really active, though.

[KOLANICH]

@deklanw Thank you, looks good though I have not very familiar with it now.

[KOLANICH]

@deklanw, Thank you, it seems like what we need, but currently it is too raw and glitchy to be usable now. And it looks like it is abandoned :( I guess it makes sense to fork it than to work from scratch.

[KOLANICH]

@Oxoron, I have read your habrapost, you may find the things discussed in this thread useful. Especially the link to metacademy, it really needs improvement.

[amirouche]

Very good idea.

Here is a tentative roadmap for becoming a web developer.

One way to solve this problem would be to use a software like the one that powers wikidata.org

[midzdotdev]

This is a fantastic idea! Count me in!

We could also expand this to eventually scrape the Internet and organise information automatically. I have a feeling this is already being done rather successfully by companies such as Google and IBM. It's a shame it's not open source.