ARC-Lab-UF / testbench-runner

Interactive testbench runner via Modelsim. Integrates with Canvas assignments.
2 stars 1 forks source link

Modelsim template overhaul #10

Closed benjamin051000 closed 1 year ago

benjamin051000 commented 1 year ago

Do not modify original modelsim project files. Copy everything needed into temp directories, and optionally delete them afterwards.

benjamin051000 commented 1 year ago

First step is to combine tcl-templates and modelsim-templates into their own lab-templates directory... Or something like that.

benjamin051000 commented 1 year ago

Apparently the work directories, while they're supposed to be ignored, seem to have important stuff in them... opening the project doesn't work without them. I guess that makes sense.

benjamin051000 commented 1 year ago

Now that I think of it, why don't we just create new projects each time? What's the reason behind starting with an old project?

benjamin051000 commented 1 year ago

Let's try to see if we can't get it working without the modelsim templates... Just make a new project each time! All the script does anyway is add files and compile/run, I don't think there's anything special about the current setup.

benjamin051000 commented 1 year ago

One issue with this approach is using the GUI for multiple runs may require ModelSim to close and reopen several times (which is really slow). Perhaps a solution to this could be:

benjamin051000 commented 1 year ago

Projects are now all isolated as per the spec. One thing that must be addressed is projects that error out. They just close instantly instead of giving you an ability to "retry". On one hand, this is fine, because it's quick to rerun the grader, or you can just open that modelsim project file in the GUI. Intervention is most likely required anyway in that case, since it's likely an issue with the students' submitted code.

I'm honestly not even sure if the original grader handled errors properly. It was set to "resume" on errors, which from what I can tell means essentially ignore them. One nice thing you could do was modify the files, save them, and hit "retry" to recompile and rerun the sim, and see if that fixes the problem. This feature should probably be added back.

benjamin051000 commented 1 year ago

Yes. Issues with errors when compiling/simulating are gonna make this tough to sell. Otherwise I have to manually open and run each one. Two options for this:

  1. When a simulation finishes (failed or completed), offer to run it again or move on.
  2. Using the main menu concept, allow re-running specific students
benjamin051000 commented 1 year ago

For now, I may just leave that alone. Let's just attach the tcl script data to StudentData class and move on for now.

benjamin051000 commented 1 year ago

Well, it wasn't that hard to have the python code rerun the sim until you're happy with it. It shuts down and reopens modelsim as a new process each time.

Not sure if anything will be affected by multiple runs, since modelsim creates a new project each time. A potential issue could be that the modelsim temp files already exist... they may need to be deleted each time, too. Not sure.

benjamin051000 commented 1 year ago

Doesn't seem to be an issue! Looks good for now. The last thing would be to update the TCL to actually print the name of the upcoming TB. But that can be its own issue. This is pretty much done.

benjamin051000 commented 1 year ago

Nah, let's not change the workflow. Keep the keybindings the same as before.

benjamin051000 commented 1 year ago

much much better.

benjamin051000 commented 1 year ago

Sorry in advance to those who do everything in the --gui...