search

Developer-DAO / academy-turbo

D_D Academy is an open-source education platform created by and for Developer DAO.
https://academy.developerdao.com
MIT License
15 stars 10 forks source link

Ensure learner credentials are worth something #210

Open elPiablo opened 5 months ago

elPiablo commented 5 months ago

We want to have (e.g. NFT) credentials that have some real proof that a learner has really learned something.

For that we need to do a few things (please collaborate for understanding intentions before spending lots of time on coding.):

Update the programmatic/DB handling of a question to reflect the following:

Ensure that the learner can't cheat. Some ideas are:

The final result we want to capture. Why? Each question used in all of our quizzes capture different types of knowledge from learner, and we want to capture it all:

Markkos89 commented 5 months ago

Hello piablo. While reading the text some questiones came up:

To clarify:

I believe that before tackling some topics we should change the way the questions and quiz questions and answers are stored. re: change json files to database tables

kempsterrrr commented 5 months ago

Interesting discussion here.

It feels it's a valuable to have some decent protections against cheating though as @Markkos89 has said I think it would be very hard to make it impossible and it may add a lot of complexity. Given we're reliant on them answering these questions themselves if they can't get it right and they want to the NFT I imagine in a decent number of cases people are going to just google the answer, or control F on the page to check for it. This is what I'd do.

More important for me is not alloying this to be "botted" so we can start offering CODE rewards for completing courses.

  • [ ] learner has 1 chance to do a checkpoint question set
  • [ ] in final quiz can only move 'back' and 'forward' one time to check and confirm answers, and if they leave the modal environment, their score gets recorded and minted

I can see the thinking behind this but I'm unsure if it provides a good experience for the user/learner. If they can't make it passed a check point what happens... they can't finish the lesson, ever?

I feel tracking lesson or track completion would be sufficient > score.

peteruche21 commented 5 months ago

speaking on CHEATING and how the QUIZ is CAPTURED

giving the learner one chance attempt to the quiz could be translated to giving the learner one chance attempt to a quiz POINT, but many chances to re-attempt the quiz. we could say if you attempt the quiz for the first time and pass, you get points for the quiz which could sum up to a total at the end. The more point the more probable you can get the NFT at the end.

I am trying to imaging it in a point system, where each checkpoint carries a number of points that can only be claimed on first try. Clicking outside the quiz modal could mean loosing the points, opening a new tab could also mean loosing the points.

But means that the learner could attempt the quiz as many times as possible; decide to loose the points and continue.

Markkos89 commented 4 months ago

speaking on CHEATING and how the QUIZ is CAPTURED

giving the learner one chance attempt to the quiz could be translated to giving the learner one chance attempt to a quiz POINT, but many chances to re-attempt the quiz. we could say if you attempt the quiz for the first time and pass, you get points for the quiz which could sum up to a total at the end. The more point the more probable you can get the NFT at the end.

I am trying to imaging it in a point system, where each checkpoint carries a number of points that can only be claimed on first try. Clicking outside the quiz modal could mean loosing the points, opening a new tab could also mean loosing the points.

But means that the learner could attempt the quiz as many times as possible; decide to loose the points and continue.

This feeds my idea to gamify the learner experience and introduce a score based on XP, a leaderboard and learner levels.

score: xp to receive after completing a quiz/checkpoint q/warmup q XP: the total experience points leaderboard: create learners leaderboard learner levels: introduce learner levels that are going to be changing based on the XP

elPiablo commented 4 months ago

Hey folks I just wanted to get all these ideas into one issue - not for the purpose of trying to solve them in one issue, because there are too many, but as far as pedagogy go, the features cover one over-arching concept => learner questions and their purpose at each stage for how they engage with the content, and the rewards and motivation they get from the challenges each question type gives them.

So the issue with all its checkboxes is like a mini-data base for me to get an overview on how to solve the problems, and check them off as we go in trying to solve them. It will also be a public place to have others understand what I'm trying to achieve. E.g. content authors. Obviously they would need to eventually be broken down into multiple issues/tasks. Just getting the convo going on the thought process.

Markkos89 commented 4 months ago

Good to know. I'll suggest Notion docs for working on these kind of topics. Maybe github is more dev oriented, or it might be only my perspective.

Good thing is that because of this issue, this score topic came to the table like @peteruche21 said, adding my thoughts like:

introduce a score based on XP, a leaderboard and learner levels.

score: xp to receive after completing a quiz/checkpoint q/warmup q XP: the total experience points leaderboard: create learners leaderboard learner levels: introduce learner levels that are going to be changing based on the XP

Should I go to a new issue or a Notion page to discuss further details and get to a more solid idea state? Thank you

elPiablo commented 4 months ago
  • What is a prediction question? Do you have any working example

There are more, Marcos, but I found these ones quite quickly: https://academy.developerdao.com/tracks/nft-solidity/3 (second, third and fifth checkpoint sets => each set last question) https://academy.developerdao.com/tracks/nft-solidity/4 (warm-up question set; first checkpoint set; => each set last question)

elPiablo commented 4 months ago
  • Which domain would be responsible to update the quiz-builder app?

Dev team stewarding it out to the world hehe.

We first need to solve some of the other questions before we get to updating the app. Maybe Chiin would like to do. In any case, I thought it was one that you lads could but on a to-do task board and advertise it 😁

elPiablo commented 4 months ago
  • Storing checkpoint/quiz answer would be unnesary as they will be the same always (there is only one way to pass a quiz)
  • There is no score recorded/minted, just storing "completed/not completed" status without learner's answers If I understand you correctly, I'll answer with this.

The idea of giving a test is not to expect 100% from the learner. It's to show the learner what they absorbed. It defeats the whole purpose of testing yourself if there is only one outcome. There has to be a challenge for the learner to overcome, otherwise they don't really need to try.

elPiablo commented 4 months ago
  • It's hard to prevent from cheating. (Right now a learner could open a new browser tab with the content able to read and the quiz in another tab.)

This the main thing we have to solve. Cheating.

elPiablo commented 4 months ago

giving the learner one chance attempt to the quiz could be translated to giving the learner one chance attempt to a quiz POINT, but many chances to re-attempt the quiz. we could say if you attempt the quiz for the first time and pass, you get points for the quiz which could sum up to a total at the end. The more point the more probable you can get the NFT at the end.

Could you give two example scenarios so we can compare them? Just so I'm 100% sure what you mean. Could be some lift off here hehe 🚀

elPiablo commented 4 months ago

I am trying to imaging it in a point system, where each checkpoint carries a number of points that can only be claimed on first try. Clicking outside the quiz modal could mean loosing the points, opening a new tab could also mean loosing the points.

But means that the learner could attempt the quiz as many times as possible; decide to loose the points and continue.

I like your line of thinking here too. It really encourages the learner to 'digest the content well' as they progress through the lesson. 🚀

elPiablo commented 4 months ago

Given we're reliant on them answering these questions themselves if they can't get it right and they want to the NFT I imagine in a decent number of cases people are going to just google the answer, or control F on the page to check for it.

"Sorry, learner, it's D_D Academy for learning skills to build web3, and not a marketplace for collectibles. If that's why you're here, fill your boots, but if someone looks later at your collection, and you can't answer the questions that you claim to have got right, then you won't look as smart as the NFT that we designed for you 🤣 " would be my response.

The reason I came to our wonderful devs was to find solutions to these problems, because I have faith in them.

I want to create the necessary environment that create dopamine in learners when they overcome the challenges we put in front of them, and get the rewards, i.e. they've really learned something and proved it by getting the answers right. NFTs are a bonus.

Markkos89 commented 4 months ago

Ok if dev domain team will be responsible for updating the utility app then I will include it on the admin all rather to have it outside the repo or in a different link.

We don't have tooooo many lessons completed in v1 but in case a learner wants to cheat then the should go to GitHub and in the correct folder there is all the answers in different JSON files, lol. So one point should to be store this sensitive data in a protected database or use EAS. Because we don't reach too many quizzes completed maybe we don't have cheaters yet but they can arrive....