Umuzi-org / Tilde

Open source agile, collaborative learning platform.
52 stars 34 forks source link

Make "command description" DRY so it doesn't need to be rewritten in each language #668

Open kingraphaii opened 10 months ago

kingraphaii commented 10 months ago

Related issues: N/A

Description:

New automarker lib setup will require or allow configs to set command description within functional tests, no need for repeating it in adapters for each flavour :)

Example usage:

file: consume_github_api_186/functional_tests/test_get_pull_requests.py

def test_returns_empty_array_when_no_matching_prs(tester):
    command = get_pull_requests(
        owner="Umuzi-org",
        repo="ACN-syllabus",
        start_date="2022-12-25",
        end_date="2022-12-25",
    )
+   tester.register_command_description("Bla bla bla")
    tester.run_command(command)

But...

Setting command description in adapters allowed us to describe how certain functions were called, according to what we expected from each language.

JS ex.

console.log(
  `call getPullRequests with arguments ${JSON.stringify({
    owner,
    repo,
    startDate,
    endDate,
  })}`
);

Python ex.

print(
    f"call get_pull_requests with arguments owner={owner}, repo={repo}, start_date={start_date}, end_date={end_date}"
)

if..else blocks won't look good here, still thinking of a cleaner way we can use, or is something like the following pseudo desc. acceptable?:

tester.register_command_description(f"call 'get pull requests' with arguments owner = {...}")

Screenshots/videos

N/A

I solemnly swear that:

kingraphaii commented 10 months ago

At this point the lib prioritizes tester.register_command_description(), if thats not set it checks the adapter confs. In summary:

For your case, I'm assuming you had neither <command_description> nor tester.register_command_description() in the tests?

I did push another commit to safely check for command_description on the test runner itself, but it's a different issue I addressed than what you encountered.

Ngoakor12 commented 10 months ago

I have this one in the functional tests:

# automarker-2-config/coding_aptitude_mini_course_task1_756/functional_tests/test_task_1.py
import importlib
import adapter.adapter as adapter

importlib.reload(adapter)

from adapter.adapter import task1

def test_runs_correctly(tester):
    command = task1()
    tester.run_command(command, assert_no_import_side_effects=False)
    tester.register_command_description("call task1 :)")
    tester.assert_returned(None)
    tester.assert_printed("Hello World!")

In adapter:

// /automarker-2-config/coding_aptitude_mini_course_task1_756/javascript/adapter/task1.js

console.log("<setup>");
console.log("</setup>");
// console.log("<command_description>");
// console.log("call task1");
// console.log("</command_description>");
console.log("<import_learner_code>");
const { task1 } = require("../../task1");
console.log("</import_learner_code>");
console.log("<running>");
const result = task1();
console.log("</running>");
console.log("<returned>");
console.log(JSON.stringify(result));
console.log("</returned>");

Does it matter where you define it in the functional tests?

kingraphaii commented 10 months ago

Oh, yeah. I see where the issue is, should have documented it better. We register description before running command :)

Ngoakor12 commented 10 months ago

That makes sense in hindsight.