ucbds-infra / otter-grader

A Python and R autograding solution
https://otter-grader.readthedocs.io
BSD 3-Clause "New" or "Revised" License
126 stars 66 forks source link

Tests Passed Locally But Fail in Gradescope #884

Open codingPie opened 4 days ago

codingPie commented 4 days ago

I have been using Otter grader for over 10 coding exercises without any problem. My next coding exercise is on charts and graphs. I plan to manually grade the work. I set up most questions with the following settings (I had to remove the space after "#" to prevent it from becoming a heading):

BEGIN QUESTION

name: q1 manual: true

In addition, I have 2 questions for autograding as q0 and q00. They work well on my computer but cannot work on Gradescope. I keep getting the following error and could not find out why. I have my file attached to this question. The file contains only the 2 autograded questions and 1 of the manually graded question. I placed the data loading code at different places. None worked. This is one version. Please help. Coding_Exercise_13-Copy1.ipynb.zip

q0 - 1 result: ❌ Test case failed Trying: assert get_fruit_shape() == (12, 9) Expecting nothing


    Line 1, in q0 0
    Failed example:
        assert get_fruit_shape() == (12, 9)
    Exception raised:
        Traceback (most recent call last):
          File "/root/miniforge3/envs/otter-env/lib/python3.12/doctest.py", line 1368, in __run
            exec(compile(example.source, filename, "single",
          File "<doctest q0 0[0]>", line 1, in <module>
            assert get_fruit_shape() == (12, 9)
                   ^^^^^^^^^^^^^^^^^
          File "/tmp/ipykernel_31/2329667314.py", line 4, in get_fruit_shape
            return fruit_sale_df.shape
                   ^^^^^^^^^^^^^
        NameError: name 'fruit_sale_df' is not defined
chrispyles commented 3 days ago

NameErrors usually come from an error being thrown in the same cell where that variable is defined. You can run Otter in debug mode on Gradescope following the steps here with their debug via SSH feature to see what/if an error is thrown. If that doesn't work, let me know I can look further into this.

codingPie commented 2 days ago

OK, I had difficulty connect to SSH today. Need to go back to the computer with the authentication key to check this later.

codingPie commented 23 hours ago

@chrispyles I finally managed to login to SSH but could not get anything useful. No error was thrown and I got the same message of public tests all failed. I turned on the log and read the log file. You can see that even th log file says that all test cases parsed. It would be great if you can look into this. Thanks.

otter.logs.Log([ otter.logs.LogEntry(event_type=EventType.INIT, success=True, timestamp=2024-11-13T20:24:41.175918+00:00), otter.logs.LogEntry(event_type=EventType.CHECK, question=q0, success=True, timestamp=2024-11-13T20:24:57.606077+00:00), otter.logs.LogEntry(event_type=EventType.CHECK, question=q00, success=True, timestamp=2024-11-13T20:25:00.980963+00:00), otter.logs.LogEntry(event_type=EventType.BEGIN_EXPORT, success=True, timestamp=2024-11-13T20:29:25.903023+00:00) ])

chrispyles commented 20 hours ago

@codingPie can you send the autograder zip file you uploaded to gradescope as well? feel free to send via slack if you don't want the hidden tests shared publicly.

codingPie commented 3 hours ago

@chrispyles Thank you. Here it is Coding_Exercise_13-autograder_2024_11_13T12_49_05_531161.zip