search

cschaffner / InformationTheory

Documents for Information Theory used in Master of Logic course Information Theory at the UvA
Other
34 stars 13 forks source link

Homework #1 #4

Closed cschaffner closed 7 years ago

cschaffner commented 8 years ago

I had a quick look at the homework set. I think it is fine, maybe a bit on the easy side. I would like them to prove the weak law of large number themselves, but as I do the symbol codes and arithmetic codes first, there's still time for that in next week's homework (before using it in the AEP). In 2, we could draw the entropy diagram and ask them to fill in all numbers. The title of Problem 3 is a bit strange. Problem 5 should include a reference to the definition of relative entropy, as I will probably not talk about it in the lectures.

Also, can we change the exercise.sty template in a way that it becomes easy for them to directly insert their answers into the source? So we can tell them to download the .tex file and directly start editing there (with the obvious advantage of already having all the shortcuts defined, diagrams etc., and the obvious disadvantage the our source has to actually be clean, but that should probably be the case anyway, as it is on github...)

Maybe it would be fun to have some more challenging problem as well for those who dare. It does not have to be part of the homework, we could just put it on moodle as "Challenge of the week". For this week it could be: Find a discrete distribution with an infinite amount of entropy. I'm not sure if I can come up with a nice challenge every week, but it should be possible. ;-)

cschaffner commented 8 years ago

Also, there will be only 6 weekly homework grades (in the preamble)

Yfke commented 8 years ago

I have updated exercise 2 (and the other small things you mentioned). It now asks for the computations of all the measures in the entropy diagram. I've also put a macro for the (2-variable) entropy diagram in the exercises.sty template, and created an "empty" solutions tex-file, which I'll put up on Moodle together with the exercises.sty. Maybe you need to check with Bas if it is ok to publish his style file online?

Challenge of the week sounds nice (but, of course, extra work :wink: ). Do you want people to submit them (which means we have to check them), or just have a discussion topic for every challenge? If it's not graded, how do you want to create incentive to do them? Bonus points, or some sort of prize at the end?