Suitability of Pelita for a Master level programming lecture?

[fmaussion]

I just had a look at pelita and I am truly amazed by the potential of the tool to teach OO concepts in python!

I am planning a new lecture in scientific programming for atmospheric scientists (https://orawww.uibk.ac.at/public/lfuonline_lv.details?sem_id_in=18S&lvnr_id_in=707716), and I am considering Pelita as an end-of-semester group project. Before I dive into the topic in more detail, I have a couple of questions you may have answers to.

How suitable do you think the tool is for a repeated lecture (once a year) at university level? Master students are different from your targeted audience of PhD Students: some of them will try to copy solutions from others / earlier courses / internet. Do you think there is a way to mitigate and/or control this somehow?

From what I understand you usually let your students work on the topic for a couple of days. It Pelita also suited for, say, a month-long group project? (not full time of course).

Thanks for your help!

[otizonaizit]

Hey! Cool that you want to use pelita for a course!

Depending a bit on what you want to do in the course pelita may or may not be suitable for a month-long group project.

In our experience the code students are able to write after 2 days is very rudimentary, so there is definitely room for having more time and doing somethings better, especially in terms of following best software development pratices, for example proper testing. If you want your students to spend sometime implementing some more adventurous strategies, then I think there is the definitely potential of filling up a month worth of time.

Regarding OO in particular, we are in the process of cleaning up the API. As part of this work, we plan to offer a completely procedural interface for implementing a player, so that students do not need to be able to write their own classes anymore. A minimal understanding of the Python object model is still required, for example when using methods and attributes on objects, but there will be no need to write any class to implement a client.

The ongoing API refactoring is happening on branch new_api.

In case your course starts soon, we can agree on keeping the master branch untouched for a while, or maybe we could do a release to freeze the status-quo, and you could use that release in your course, while we merge improvements from new_api to master. @Debilski : what do you think?

The plan is to finish the refactoring before the next school, which is planned in September 2018.

Let us know!

[otizonaizit]

How suitable do you think the tool is for a repeated lecture (once a year) at university level? Master students are different from your targeted audience of PhD Students: some of them will try to copy solutions from others / earlier courses / internet. Do you think there is a way to mitigate and/or control this somehow?

What we do is that we have private repos on GitHub for the groups, and we set permissions so that they are only readable by group members. By using the network game facilities in pelita they can still play against each other without seeing the code. As an academic you can get private repos for free from GitHub.

This of course does not save you from the situation that some students from previous years would just pass over a clone of the group repo to current students, or sharing it on GitHub under a different name or on a completely different platform altogether. but I guess this is a problem you would have independent of the software you choose to use, don't you think so?

[Debilski]

Hi, yeah, it is very cool that you want to use Pelita indeed!

@Debilski : what do you think?

Not sure we technically have to freeze it, since we have a published, installable version of Pelita on PyPI which will still have to OO API.

But of course, if we know that there are users ‘out there’, we will try not to introduce any sudden API changes and document everything in a changelog. :)

In general, I would not worry too much about this for this year’s course but keep in mind that next year there might be a few incompatibilities and probably a little less object oriented programming. On the other hand, though, if your course produces some nice pull requests for a better object oriented API that is also easily understandable, we might also consider keeping them both for a while. :)

By using the network game facilities in pelita they can still play against each other without seeing the code.

This reminds me that we (I) should definitely update the documentation on how to do this.

[fmaussion]

Thanks for the quick and positive answers! Indeed, please don't accommodate your development timeline for me: I will go back to an older version myself if I feel it's necessary.

Regarding your comments:

Regarding OO in particular, we are in the process of cleaning up the API. As part of this work, we plan to offer a completely procedural interface for implementing a player, so that students do not need to be able to write their own classes anymore.

May I ask what is the reason for this change? OO concepts are very important even for scientists, since all of them will at least use them, while some will need them if they develop library code

This of course does not save you from the situation that some students from previous years would just pass over a clone of the group repo to current students, or sharing it on GitHub under a different name or on a completely different platform altogether. but I guess this is a problem you would have independent of the software you choose to use, don't you think so?

Yes, you are right - this is definitely not limited to your tool. If this really becomes an issue I'll have to look for plagiarism detection software somewhere. This is not my primary concern (yet).

But of course, if we know that there are users ‘out there’, we will try not to introduce any sudden API changes and document everything in a changelog. :)

I would highly welcome this, but of course this is quite a burden for you (small tip: if you plan to do so, a reminder in form of a PR template is quite helpful). From now on I'm watching the repo and I'll be aware of your changes, so feel free to decide whatever is best for you.

[otizonaizit]

May I ask what is the reason for this change? OO concepts are very important even for scientists, since all of them will at least use them, while some will need them if they develop library code

The reason for implementing a completely procedural API in pelita (which doesn't mean that we are going to drop the OO API, by the way) comes from the very practical reason that during our summer school most of the participants know how to call methods and set attributes on object instances, but do not know how write their own classes. Understandable somehow, because most of them did not have any programming courses in their academic career.

As to why we don't teach OOP at the summer school, this is a completely different issue, and one that can easily degenerate in a flame war ;-) . For a start, you can read the FAQ "Why aren't you teaching Object Oriented Programming?" at https://python.g-node.org/wiki/faq

[Debilski]

Good point with the PR template. Will definitely look into it.

The main reason for us changing the OO interface is that even though we agree that the OO concepts may be useful for scientists, we notice in our summer school that there are less and less scientists confronted with it that we would have to do a complete beginner’s introduction for OO programming to get most of them even started and it does not really fit into the already crowded schedule for us. Of course, the Python summer school may be a bit biased in that regard: If you are a scientist who learned Java (and OO concepts), of course you would be less likely to come to a Python school.

[fmaussion]

Haha, these are good arguments. I'll try to teach OOP and if I see this is a complete failure I'll revisit my statement ;)

(btw I learned and used Java for my master thesis: what a burden I have to carry now! ;)

[fmaussion]

Folks,

I've been playing around with Pelita for a couple of days and I liked the experience a lot. I found the learning curve to be pretty steep but manageable: after one day and a couple of "aha" moments I was able to create an aggressive player targeting harvesters and a harvester a bit more clever than SmartEatingPlayer.

I believe it's going to be quite tough for my students though, but I'll give it a try anyway. My plan is to introduce the game to them tomorrow and give them three weeks of time until the tournament.

I'm a little bit in a rush right now but I'll come up with concrete suggestions/PRs soon!

I would be grateful if you could send me the groupN template repository (it isn't on github anymore). In particular, I'd like the students to be able to run tests, which I believe is part of your template.

I would also appreciate some pointers about how to use the network game facilities -- this looks awesome!

[Debilski]

Cool. I hope it will go well for your students. The template is also in the repo: https://github.com/ASPP/pelita/tree/master/doc/source/groupN but for the most recently tested version you could use this one as a starting point:

https://github.com/ASPP/pelita-player-2018-melbourne

I wrote down the reasoning behind the Makefile in this document https://github.com/Debilski/pelita/blob/cc335f9d77cc0d9697f29df44fe8e774cd915272/TEACHING.md (#463). Basically it forces the students to always be in the correct folder for both running pelita and running the tests.

There is also a bit of explanation for how to do a network game in this document but I can give you more details on this as well. (We also have a Dockerfile if you want to avoid running your students code unrestricted on your computer :) )

[Debilski]

Network games can be run both with a centralised server and in a peer-to-peer mode.

Peer-to-peer mode

For the peer-to-peer mode we assume that teams A and B (with computers A and B) want to play against each other and they want to watch it on machine C (C can be identical to either A or B in which case one could just run a local game for this team).

C must be reachable by A and B with its IP address or its name. (Let’s assume c-machine.uni.private) Additionally, there must be two arbitrary but fixed tcp ports available on C. One port each is communicated to A (eg. 50001) and B (eg. 50002) who then run the command:

$ pelita-player team-of-a/ tcp://c-machine.uni.private:50001 # machine A

$ pelita-player team-of-b/ tcp://c-machine.uni.private:50002 # machine B

on their respective computers. pelita-player now tries to connect to C using a zeromq tcp connection and just waits until C is reachable. We therefore have to tell C to start a game and assign tcp ports 50001 and 50002 as blue and red teams (instead of the usual path to a python module). This is done with

$ pelita tcp://0.0.0.0:50001 tcp://0.0.0.0:50002

Machine C will now open the usual Tk window, set up a gamemaster and run the game.

Centralised mode

The problem with the peer-to-peer mode is that the pelita-player will have to be restarted every time and additionally, that the IP address and ports from the viewer will have to be communicated to all peers. This mode therefore does not scale very well.

The centralised mode now assumes that there is a network player to be run on machine N and team A wants to play against it. We now assume a free port on N (eg. 50000) and set up the remote player on N with:

$ pelita-player --remote team-of-N/ tcp://0.0.0.0:50000

Team A can now play against N with

$ pelita team-of-a/ remote:tcp://0.0.0.0:50000

The remote protocol will then connect to N and request the start of a new player instance on N that will be used in the game against A. It is also possible to run multiple games against N at the same time (not only from A but also from B, …) or after another. Additional teams can be added to N quite easily by just assigning a new free port.

One bug in the current implementation is however that unfinished matches on N will live on forever waiting on a new universe to react to. Therefore it might be a good idea to put the pelita-player --remote part inside a container. (Even more so, if you plan on automatically running the student’s code on one of your machines in this way.) Otherwise you will have to regularly check the ps output and maybe kill some of those Python processes.

Hope this is enogh information to get you started. I will also copy this text to the docs some time. :)

[fmaussion]

Thanks a lot for the info, this is great! I think I won't use this feature this time, but will consider it for next semester.

The students received their instructions last Monday (http://fabienmaussion.info/scientific_programming/html/26-Pelita.html) and have three weeks time to write their players. Since they have quite a lot of other things on their plate at the end of the semester my guess is that they will probably spend (much?) less than 3 full days on it, let's see how it goes/

[Debilski]

Yeah, I already looked into the instructions. We might steal back some of the examples.

Are you planning to use our tournament script at the end of your course or do you make the competition manually? (There is still bug #451 which kills the UI currently during tournament but it is fixable.)

[fmaussion]

We might steal back some of the examples.

Sure! I guess that with a little more work they can become even nicer.

Are you planning to use our tournament script at the end of your course or do you make the competition manually?

My plan was to test it out next week and decide based on the outcome. From your experience, how long does a full tournament take with 5 teams?

[Debilski]

I think the tournament time depends on how many slow implementations you will have. Some games can then take five minutes or more (the only hope is that the slow implementations are also not so good in defending and the other team simply finishes early). Theoretically even longer times are possible of course. Still, some games are done in a minute. I don’t have the stats with me but for 14 games (full tournament; however, draws in knock-out stage will be replayed so it might be more) you should estimate 30–45 minutes and maybe even more.

[fmaussion]

The students are going to send me their players this week end, and the tournament will take place next Monday!

I decided not to use the tournament mode but to use some online tournament app instead (https://kickertool.de).

Do you have any advice for the tournament? Should I set up something differently than a default game call, i.e. to avoid dead end mazes? Right now I would tend to set a slower game speed, did you do this as well?

[otizonaizit]

We use layouts without dead ends for the tournament (but that should be the default also when you run pelita manually, right?). On the other end, if your students programmed their players training on layouts with deadends it would be unfair to change this: I think layouts with dead ends encourage certain strategies that are not useful in the no-dead-end case. We decided to use no-dead-end by default because of this… Regarding the speed of the game, my impression is that after the students have played with pelita day after day, they are pretty fast at interpreting the state of the game, and the default speed is good enough. But of course, YMMV.

[fmaussion]

Thanks! Yes right, the default is without dead end and should stay like this.

[Debilski]

The students are definitely faster than me with parsing the game state during the tournament. Maybe it’s the age.

[fmaussion]

Dear pelita-devs!

We just had our tournament this morning. It went perfectly, it was a lot of fun! The students were really happy (and I was, too).

Their solutions are simple but nice: I can anonymize them and provide them to you if you if you are interested.

A few comments after playing around with the game myself over the week-end and seeing the students interact with it:

• I like the OO layer, and it helped me to make a point about when OO in python is useful and when not. For most of my students it will be the last time they write classes themselves, and I'm glad they did so by programming a player
• I have the diffuse impression that things in the pelita API are a little bit more confusing than they should. Again, this impression comes out of only a couple of weeks experience, but I'll try to open some issues later this week for discussion.

Thanks again for this great educational tool! Closing this for now, next pelita tournament for me will be February 2019 ;-)

[Debilski]

On 25.06.2018 11:17, Fabien Maussion wrote:

We just had our tournament this morning. It went perfectly, it was a lot of fun! The students were really happy (and I was, too).

That’s great to hear. Very happy it worked for you.

Their solutions are simple but nice: I can anonymize them and provide them to you if you if you are interested.

Did you log the games for replays? (We probably should make the possibility of this more pronounced in the docs. Also it is slightly broken atm #453)

I’d really like seeing different solutions though (especially when they come from a completely different course).

A few comments after playing around with the game myself over the week-end and seeing the students interact with it:

• I like the OO layer, and it helped me to make a point about when OO in python is useful and when not. For most of my students it will be the last time they write classes themselves, and I'm glad they did so by programming a player

In general I agree; the primary reason for us to remove it is pragmatics though. We hope it the non-OO version will be easier to understand; it won’t necessarily be more elegant.

• I have the diffuse impression that things in the pelita API are a little bit more confusing than they should. Again, this impression comes out of only a couple of weeks experience, but I'll try to open some issues later this week for discussion.

Please go ahead with opening more issues. I think one of the biggest problems right now is that we have multiple APIs that are overlapping and make the OO approach even harder. My personal priority is also fixing (= completely makeover) the API rather than the OO model.

Thanks again for this great educational tool! Closing this for now, next pelita tournament for me will be February 2019 ;-)

Looking forward to your improvements and thanks for trying it out. :)

[Debilski]

Thanks for the link. Is there any specific reason you put the init.py files into each of the groups’ folders?

[fmaussion]

Is there any specific reason you put the init.py files into each of the groups’ folders?

Yes, I told my students to do so: with them the Makefile is obsolete: pytest can be run from the root folder, and spyder-unittest can also be taught to run pytest from the current working directory (i.e the root).

Is there another way to avoid using make ?

[Debilski]

Is there another way to avoid using make ?

By setting the PYTHONPATH explicitly probably. Or if you moved the tests inside the team folder.

Honestly, I did not expect the simple __init__.py solution to actually work from inside the test folder as well, which it apparently does. It is more robust than I thought. :)

[fmaussion]

Personally I would rather move the tests in the team folder for the students. I don't like giving the team a package structure (because it isn't really a package) and I definitely don't like setting PYTHONPATH explicitly.

I would have done so before giving them the link to the melbourne template but once I realized it was too late ;)