search

ARC-Lab-UF / testbench-runner

Interactive testbench runner via Modelsim. Integrates with Canvas assignments.
2 stars 1 forks source link
modelsim python3 tcl vhdl

readme

testbench-runner

An interactive testbench runner for HDL. Uses ModelSim under the hood. Integrates with Canvas assignments for easy grading.

Table of contents

Prerequisites

1. ModelSim

Install ModelSim via this link

The tool will verify ModelSim is installed and an appropriate version on startup.

2. Python

You can download Python from this link.

You can verify Python is working properly by running python -V in a terminal window. It should print something similar to:

$ python -V
Python 3.10.1

Note: DD_Grader has been tested with Python 3.10.1 and is not guaranteed to work on other versions.

Download

There are two ways to download the tool. For those who only plan to use the tool (and will not contribute), follow option 1. For those interested in contributing, follow option 2.

Download from the Releases page (recommended) - Download a released version as a zip file. - Be sure to also download a supported release of the lab testbenches you plan to use. EEL4712C Digital Design's are [here](https://github.com/ARC-Lab-UF/digital-design-grading-tbs).
Clone the repository Use the following command to clone the repository: `git clone --recurse-submodules https://github.com/ARC-Lab-UF/testbench-runner.git` - The `--recurse-submodules` flag will also clone the `lab-testbenches/` private repository for you, if you have access to it. - Optionally, use the SSH url rather than the HTTPS url (shown above).

Usage

1. Select Students to Grade

There are two methods to select students to grade: By section number, or via a list of names.

Select by Section Number To use the `--section` flag, a file named `all_students.csv` must be downloaded to the project directory. This file contains information about students in the course, including each students' section number. ### Download Student Data 1. Navigate to the course gradebook on Canvas. 2. Click "Actions > Export" as shown in the below image. ![Image showing the highlighted export button the user should press in the Canvas gradebook to download a CSV of students in the course](images/export-students.png) Save this file as `all_students.csv` in the root directory (the same directory as this README file). It should be structured similarly to `all_students_example.csv`, but may include additional information, like assignment grades. When running the grader, use `--section
` to specify a section number you wish to grade. Section numbers are 5-digit numbers seen in the parentheses on a students' section ID string. For example, in the class/section identifier `EEL4712C-0001(11624)`, `11624` is the section number.


Select by Student List To choose specific students for grading, make a text file with the students' names. Use the same format as `students_example.txt`, which is ` [Middle name(s)] ` per line. Finally, use the default `students.txt` filename, or specify the file path in the CLI args via `--student-list `.


2. Download Canvas Submissions

On the Canvas Assignment page, press "Download submissions".

Image showing where to find the "Download Submissions" button on the Canvas Assignment page

This will collect all student submissions and download them in one file titled submissions.zip.

Copy submissions.zip into the project directory, or specify the zip archive path in the CLI args via --submissions <path>.

3. Run grader.py

To run the testbenches for Lab 1 with all default values (a "submissions.zip" folder and a "students.txt" file in the project root directory), run the following command:

python grader.py --lab 1

If you want to use a section, you must provide the --section flag:

python grader.py --lab 1 --section 12345

Be sure to supply values for --submissions, --student-list, or --all-students-file if they do not use the default file name.

Other options and flags

submissions.zip path

To specify a path for the submissions.zip file, use --submissions <path>. This can be useful if you want to maintain multiple labs' worth of submissions. You can rename the file to submissions_lab2.zip for example, and specify this name in the CLI args.

Using the Interactive Testbench Runner

After running a command to start the grader, the program will print the students it was able to find in the submissions.zip file. Then, ModelSim will begin running, and a prompt will appear:

----------------------------------------
(1/9) Student name
- Press return to move to next student
- Press r to rerun simulation
- Press g to rerun simulation in the ModelSim GUI
- Press n to skip this student
- Press q to quit
----------------------------------------
> <type your response here>

Pressing return will either run the simulation for the first time, or move to the next student.

If multiple testbenches are used to grade a particular lab, the simulator will ask the user to press Enter to continue on to the next testbench.