Submission: pygtracker (Python)

Submitting Author:

Javairia (@jraza19),
Jianru (@jianructose),
Vu (@tdkhanhvu),
Yanhua (@yhchen20)

Package Name: pygtracker One-Line Description of Package: pygtracker provides UBC MDS instructors with tools to record, analyze and adjust grades for students in MDS program. Repository Link: pygtracker Version submitted: 0.1.7 Editor: Tiffany Timbers(@ttimbers ) Reviewer 1: @eboylestanley Reviewer 2: @arashshams Archive: TBD
Version accepted: TBD

Description

pygtracker is a Python package that allows UBC MDS lecturers to record, analyze and adjust grades for students in a particular program. Users can record grades from each course, generate a summary report to determine which class is more challenging than the rest, or identify students who may need help. Finally, the package can suggest ways to adjust the grades for students to ensure the average grades match predefined benchmarks.

Scope

Please indicate which category or categories this package falls under:
- [ ] Data retrieval
- [ ] Data extraction
- [ ] Data munging
- [ ] Data deposition
- [ ] Reproducibility
- [ ] Geospatial
- [ ] Education
- [ ] Data visualization*
- [x] Other (Teaching analytics)

* Please fill out a pre-submission inquiry before submitting a data visualization package. For more info, see notes on categories of our guidebook.

Explain how the and why the package falls under these categories (briefly, 1-2 sentences):

The package pygtracker does not explicitly fall under any of the categories listed above. After the discussion with the editor, we concluded it could fall into the category as other (teaching analytics for education technology). However, pygtracker does include some data extraction and munging, as well as for workflow automation purpose.

Who is the target audience and what are scientific applications of this package?

The target audience for pygtracker is intended for instructors and teaching team in UBC MDS program specifically. The application of pygtracker leans towards more practical side rather than scientific ones, which includes teaching and learning analytics. The teaching team is able to adjust the grades for students to ensure the final average grades match the expected benchmarks of student performance.
Are there other Python packages that accomplish the same thing? If so, how does yours differ?

There are existing packages to manage students' grades in the Python ecosystem. For example, grades-report is a package purely generating statistics from all grades. In addition, edx-bulk-grades aids lecturers in grading and modifying students' grades in bulk. However, these packages are lacking useful features that are unique to the MDS program, such as identifying students / courses that are much different from their peers or adjusting grades to make sure they conform to MDS standard. As a result, we come up with this useful tool which helps UBC MDS lecturers have a better control of student performance.
If you made a pre-submission enquiry, please paste the link to the corresponding issue, forum post, or other discussion, or @tag the editor you contacted:

Technical checks

For details about the pyOpenSci packaging requirements, see our packaging guide. Confirm each of the following by checking the box. This package:

[x] does not violate the Terms of Service of any service it interacts with.
[x] has an OSI approved license.
[x] contains a README with instructions for installing the development version.
[x] includes documentation with examples for all functions.
[x] contains a vignette with examples of its essential functions and uses.
[x] has a test suite.
[x] has continuous integration, such as Travis CI, AppVeyor, CircleCI, and/or others.

Publication options

[ ] Do you wish to automatically submit to the Journal of Open Source Software? If so:

JOSS Checks

- [ ] The package has an **obvious research application** according to JOSS's definition in their [submission requirements][JossSubmissionRequirements]. Be aware that completing the pyOpenSci review process **does not** guarantee acceptance to JOSS. Be sure to read their submission requirements (linked above) if you are interested in submitting to JOSS. - [ ] The package is not a "minor utility" as defined by JOSS's [submission requirements][JossSubmissionRequirements]: "Minor ‘utility’ packages, including ‘thin’ API clients, are not acceptable." pyOpenSci welcomes these packages under "Data Retrieval", but JOSS has slightly different criteria. - [ ] The package contains a `paper.md` matching [JOSS's requirements][JossPaperRequirements] with a high-level description in the package root or in `inst/`. - [ ] The package is deposited in a long-term repository with the DOI: *Note: Do not submit your package separately to JOSS*

Are you OK with Reviewers Submitting Issues and/or pull requests to your Repo Directly?

This option will allow reviewers to open smaller issues that can then be linked to PR's rather than submitting a more dense text based review. It will also allow you to demonstrate addressing the issue via PR links.

[x] Yes I am OK with reviewers submitting requested changes as issues to my repo. Reviewers will then link to the issues in their submitted review.

Code of conduct

[x] I agree to abide by pyOpenSci's Code of Conduct during the review process and in maintaining my package should it be accepted.

P.S. *Have feedback/comments about our review process? Leave a comment here

Editor and Review Templates

Editor and review templates can be found here

Assigning @eboylestanley & @arashshams as reviewers.

Package Review

Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide

[X] As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor before starting your review).

Documentation

The package includes all the following forms of documentation:

[X] A statement of need clearly stating problems the software is designed to solve and its target audience in README
[X] Installation instructions: for the development version of package and any non-standard dependencies in README
[X] Vignette(s) demonstrating major functionality that runs successfully locally
[X] Function Documentation: for all user-facing functions
[X] Examples for all user-facing functions
[X] Community guidelines including contribution guidelines in the README or CONTRIBUTING.
[X] Metadata including author(s), author e-mail(s), a url, and any other relevant metadata e.g., in a setup.py file or elsewhere.

Readme requirements The package meets the readme requirements below:

[X] Package has a README.md file in the root directory.

The README should include, from top to bottom:

[X] The package name
[X] Badges for continuous integration and test coverage, a repostatus.org badge, and any other badges. If the README has many more badges, you might want to consider using a table for badges: see this example. Such a table should be more wide than high. (Note that the badge for pyOpenSci peer-review will be provided upon acceptance.)
[X] Short description of goals of package, with descriptive links to all vignettes (rendered, i.e. readable, cf the documentation website section) unless the package is small and there’s only one vignette repeating the README.
[X] Installation instructions
[ ] Any additional setup required (authentication tokens, etc)
[X] Brief demonstration usage
[X] Direction to more detailed documentation (e.g. your documentation files or website).
[X] If applicable, how the package compares to other similar packages and/or how it relates to other packages
[X] Citation information

Usability

Reviewers are encouraged to submit suggestions (or pull requests) that will improve the usability of the package as a whole. Package structure should follow general community best-practices. In general please consider:

[X] The documentation is easy to find and understand
[X] The need for the package is clear
[ ] All functions have documentation and associated examples for use
- Documentation missing examples - see review note

Functionality

[X] Installation: Installation succeeds as documented.
[X] Functionality: Any functional claims of the software been confirmed.
[ ] Performance: Any performance claims of the software been confirmed.
[X] Automated tests: Tests cover essential functions of the package and a reasonable range of inputs and conditions. All tests pass on the local machine.
[X] Continuous Integration: Has continuous integration, such as Travis CI, AppVeyor, CircleCI, and/or others.
[X] Packaging guidelines: The package conforms to the pyOpenSci packaging guidelines.

For packages co-submitting to JOSS

Click to expand!

- [ ] The package has an **obvious research application** according to JOSS's definition in their [submission requirements](http://joss.theoj.org/about#submission_requirements). *Note:* Be sure to check this carefully, as JOSS's submission requirements and scope differ from pyOpenSci's in terms of what types of packages are accepted. The package contains a `paper.md` matching [JOSS's requirements](http://joss.theoj.org/about#paper_structure) with: - [ ] **A short summary** describing the high-level functionality of the software - [ ] **Authors:** A list of authors with their affiliations - [ ] **A statement of need** clearly stating problems the software is designed to solve and its target audience. - [ ] **References:** with DOIs for all those that have one (e.g. papers, datasets, software).

Final approval (post-review)

[ ] The author has responded to my review and made changes to my satisfaction. I recommend approving this package.

Estimated hours spent reviewing: 1hr

Review Comments

Hi pygtracker Team!

I really enjoyed reviewing your package. I thought the purpose was very clear and easy to understand.

It might be nice to include a small example in the documentation of each function. This would improve the documentation as a user could quickly see an example when reviewing the documentation on readthedocs.
The usage section could be expanded to include all the class functions, specifically calculate_final_grade does not have a demonstration. I think it could fit nicely with the current example using the toy dataset.
For the suggest_grade_adjustment function, I would suggest to eliminate assignments that were not present for the course, i.e. there are no milestones in course 511. It might confuse the user to see these assignments with no grades. It could be unclear if these a legitimate grades received by the student or not.
I tried tracker.suggest_grade_adjustment("522", benchmark_course=100) but received a KeyError: 0 result. After reviewing the function code, line 409 temp = temp[temp[0] != 0] # filter assessment with 0 is hardcoded to only work with the first course code in the self.course list.
Might consider adding exception handling for course codes that do not exist for the suggest_grade_adjustment function.

Hi @eboylestanley ,

Thanks for the very thorough feedbacks! Our group will look into further based on your suggestions.

Package Review

Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide

[x] As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor before starting your review).

Documentation

The package includes all the following forms of documentation:

[x] A statement of need clearly stating problems the software is designed to solve and its target audience in README
[x] Installation instructions: for the development version of package and any non-standard dependencies in README
[x] Vignette(s) demonstrating major functionality that runs successfully locally
[x] Function Documentation: for all user-facing functions
[x] Examples for all user-facing functions
[x] Community guidelines including contribution guidelines in the README or CONTRIBUTING.
[x] Metadata including author(s), author e-mail(s), a url, and any other relevant metadata e.g., in a setup.py file or elsewhere.

Readme requirements The package meets the readme requirements below:

[x] Package has a README.md file in the root directory.

The README should include, from top to bottom:

[x] The package name
[x] Badges for continuous integration and test coverage, a repostatus.org badge, and any other badges. If the README has many more badges, you might want to consider using a table for badges: see this example. Such a table should be more wide than high. (Note that the badge for pyOpenSci peer-review will be provided upon acceptance.)
[x] Short description of goals of package, with descriptive links to all vignettes (rendered, i.e. readable, cf the documentation website section) unless the package is small and there’s only one vignette repeating the README.
[x] Installation instructions
[ ] Any additional setup required (authentication tokens, etc)
[x] Brief demonstration usage
[x] Direction to more detailed documentation (e.g. your documentation files or website).
[x] If applicable, how the package compares to other similar packages and/or how it relates to other packages
[x] Citation information

Usability

[x] The documentation is easy to find and understand
[x] The need for the package is clear
[ ] All functions have documentation and associated examples for use

Functionality

[x] Installation: Installation succeeds as documented.
[x] Functionality: Any functional claims of the software been confirmed.
[ ] Performance: Any performance claims of the software been confirmed.
[x] Automated tests: Tests cover essential functions of the package and a reasonable range of inputs and conditions. All tests pass on the local machine.
[x] Continuous Integration: Has continuous integration, such as Travis CI, AppVeyor, CircleCI, and/or others.
[x] Packaging guidelines: The package conforms to the pyOpenSci packaging guidelines.

For packages co-submitting to JOSS

[ ] The package has an obvious research application according to JOSS's definition in their submission requirements.

Note: Be sure to check this carefully, as JOSS's submission requirements and scope differ from pyOpenSci's in terms of what types of packages are accepted.

The package contains a paper.md matching JOSS's requirements with:

[ ] A short summary describing the high-level functionality of the software
[ ] Authors: A list of authors with their affiliations
[ ] A statement of need clearly stating problems the software is designed to solve and its target audience.
[ ] References: with DOIs for all those that have one (e.g. papers, datasets, software).

Final approval (post-review)

[ ] The author has responded to my review and made changes to my satisfaction. I recommend approving this package.

Estimated hours spent reviewing: 2

Review Comments

Hi pygtracker team,

Congratulations on completing your package. I think you did a great job there and I really like the idea behind your package and how it can be helpful in the UBS MDS program. I enjoyed reviewing your work. Below you can find my suggestions which I hope they come useful for you guys:

It might be better if you elaborate a little more on how different your package is from similar packages in Python Ecosystem and maybe illustrate the package's unique application with an example (i.e. something simpler and less technical than the one in the Usage section) so that the audience get a better understanding on how "UBC MDS lecturers have a better control of student performance".
Great job with going above and beyond with writing seven functions with only four people on the team, however I noticed a small issue. Some courses like 524 or 532 on graded on other metrics like team-work-reflection or peer-review. Is there any reason for not including them in the "assessment_list"?
It might be a good idea to include examples with your functions or the GradeTracker Class in the "pygtracker.py", just to make it more understandable and easy to follow.
Great job with setting up so many tests. I really liked the exception-handling approach in your "test_pygtracker.py". They cover many edge cases, however I may have to agree with the first reviewer regarding exception-handling for the suggest_grade_adjustment function.
Nice job with adding keywords in your package About section. I have to keep that in mind to implement in our package too.

Congrats again and well done.

Arash

UBC-MDS / software-review-2021