Use this knowledge of Camisole to try to write "bull-proof multi-language script launcher" for my teaching activites

[Naereen]

I want to write a wrapper script like run-camisoled, that can read a file in Python/OCaml/C, and safely pass it to Camisole VM, and pretty-print its JSON results!

For my teaching next year this would be veryyy useful!. TODO: read https://github.com/mcandre/linters and https://en.wikipedia.org/wiki/List_of_tools_for_static_code_analysis and about sandboxing in general

Overview of the goal

• Imagine you gave a Python exam to a large group of student (15-400).
• You don't have any automated tests, you'll basically have to run every file, and then open it to add some comments, and finally grade it.
• Students are expected to include examples and tests in their files, to save their figures to local image files, etc.
• Oh, and at least one student has written a very evil program that will read your entire home folder, compress it and send it over the network to his Dropbox folder, then will delete and burn your laptop to ashes. :bomb: one of your student is a secret Nazi hacker!

So the setup is like this:

$ ls TP1_etudiants/
Etudiant1.py Etudiant2.py ... Etudiant49.py
$ python EtudiantHackeur.py
==> all your data belogns to us!
==> destroying your laptop
==> cry and just quit your job already, old man!

To avoid such catastrophy, the solution would be

$ run-camisoled EtudiantHackeur.py
print output of this script, but ran from a FULLY secure isolated environment!

I want to have this "multi-language" launcher script that can execute Python3, OCaml 4.05+, C11 (and more) code, in a safe and secure environment.

A few reasons for that:

• I should assume that every file submitted by a student is a very dangerous virus, and any attempt to execute/interpret/compile it locally will make my computer explode ;
• Every student's file can generate a few figures, or temporary files, and I don't want to have ~139 figures and ~153 temporary files after having executed 50 Python scripts in the same folder!
  • one easy solution is to move each file to its sub-folder!
• I don't want to think about which languages it is written in, because for the same exam some students will have the choice between C, OCaml and Python!
• I want to keep a local database, in every folder like this, for each student at each time I run their script, showing what their script shows.

So my solution should:

• [ ] be open-source, easily installable (of course) ;

• [ ] launch the Camisole VM if not already available ;

• [ ] execute the code based on its MIME type or file extension (.py -> "python", .ml -> "ocaml", .c -> "c", etc) ;

• [ ] pretty print the results, giving lots of information: compilation & execution time, stdout & stderr output (pretty printed accurately?)

• [ ] possibly (but not by default) copy back all files generated by the script, from the VM storage to the local folder? or at least allow to view/read them?

• [ ] store the results in a local file? (a local database? a text file? Markdown or org-mode file?)

• [ ] Maybe it's not important that this script can handle multi-files, this is easy to do with a for loop outside the script:

  $ ls TP1_etudiants/
  Etudiant1.ml Etudiant2.ml ... Etudiant49.ml
  $ for i in TP1_etudiants/*.ml; do
  clear; echo $i; hr
  run-camisoled $i
  # this will print a lot of information about student $i, and you have a break to read them, maybe open the file in VSCode, to add annotation etc
  read;
  done

• [ ] BUT I would LOVE that I could give multiple files and it just executes them, like if I provided a library.ml file (prior to exam) to each student, some student wrote etudiant37.ml and then I want to test their performance using tests.ml

  $ ocamlopt library.ml etudiant37.ml tests.ml

Bonus idea?

• [ ] run a linter on each file, but from a safe environment?
  • [ ] Can someone hack you if you just lint their file? See https://stackoverflow.com/questions/66287065/how-unsafe-can-it-be-to-read-ot-to-use-a-linter-of-an-untrusted-piece-of-code
• [ ] run the file maybe 100 times and get mean+-std execution time, maybe to allow an automated tool to identify the slowest codes and notify the students that something is wrong ?
• [ ] interface it with awesome testing like on ocaml-learn-public, to have coverage test for each student like
  • Etudiant1: 10/10
  • Etudiant2: 4/10
  • ...
  • Etudiant1: 10/10

Features for just OCaml

• [ ] what about using ocamlprof to add execution counter everywhere in the file?
• [ ] using merlin, mascot (not updated since 2012) or other linter?

Features for just Python

• [ ] there are a lot of Python linters, pylint, pyflakes etc: which one to choose?
• [ ] automatic line/memory profiling of each code??

Features for just C

• [ ] I don't know much, but I also guess there are a lot of C linters
• [ ] automatic line/memory profiling of each code??

Tools to use?

• Camisole or Docker for safe execution?
• Mega-linter or GitHub's super'linter (no C, no OCaml) for unified linting?

[Naereen]

See https://stackoverflow.com/questions/66287065/how-unsafe-can-it-be-to-read-ot-to-use-a-linter-of-an-untrusted-piece-of-code

[Naereen]

... I just discovered that python3 (and pypy3) command line have this option:

-I     : isolate Python from the user's environment (implies -E and -s)