cardsort: analyzing data from open card sorting tasks

[katoss]

Submitting Author: (@katoss) All current maintainers: (@katoss) Package Name: cardsort One-Line Description of Package: A python package to analyse data from open card sorting tasks Repository Link: https://github.com/katoss/cardsort Version submitted: 0.2.2 Editor: @Batalex Reviewer 1: @Robaina Reviewer 2: @khynder Archive: JOSS DOI: N/A Version accepted: v 0.2.36 Date accepted (month/day/year): 08/16/2023

---

Code of Conduct & Commitment to Maintain Package

• [x] I agree to abide by pyOpenSci's Code of Conduct during the review process and in maintaining my package after should it be accepted.
• [x] I have read and will commit to package maintenance after the review as per the pyOpenSci Policies Guidelines.

Description

• Include a brief paragraph describing what your package does:

Card sorting is a standard research method in the human computer interaction field to design information architectures. Thecardsort package analyzes and visualizes data from open card sorting tasks. Using csv data in the format returned by the kardsort tool (or any other tool outputting the same columns), it outputs a dendrogram based on hierarchical cluster analysis and pairwise similarity scores. It can also return category labels to learn which labels study participants gave to combinations of cards from emerging clusters.

Scope

• Please indicate which category or categories. Check out our package scope page to learn more about our scope. (If you are unsure of which category you fit, we suggest you make a pre-submission inquiry):

  • [ ] Data retrieval
  • [x] Data extraction
  • [x] Data processing/munging
  • [ ] Data deposition
  • [ ] Data validation and testing
  • [x] Data visualization[^1]
  • [ ] Workflow automation
  • [ ] Citation management and bibliometrics
  • [ ] Scientific software wrappers
  • [ ] Database interoperability

Domain Specific & Community Partnerships

- [ ] Geospatial
- [ ] Education
- [ ] Pangeo

Community Partnerships

If your package is associated with an existing community please check below:

• [ ] Pangeo
  • [ ] My package adheres to the Pangeo standards listed in the pyOpenSci peer review guidebook

[^1]: Please fill out a pre-submission inquiry before submitting a data visualization package.

• For all submissions, explain how the and why the package falls under the categories you indicated above. In your explanation, please address the following points (briefly, 1-2 sentences for each):

  • Who is the target audience and what are scientific applications of this package?

The target audience are researchers or practitioners in human computer interaction and user experience. Open card sorting is a popular user research method used to understand how people order and conceptualize information, in order to design information architectures. In order to make sense of the data, clusters are often visualized in form of a dendrogram. To do so, pairwise similarity scores need to be calculated for all cards, followed by a hierarchical cluster analysis. This functionality is provided by the cardsort package. It also offers functionality to return category labels, in order to learn which labels study participants gave to combinations of cards in the emerging clusters.

• Are there other Python packages that accomplish the same thing? If so, how does yours differ?

I did not find any python packages, nor other open source tools, that accomplish this, which was my motivation to make this package, when I was running a card sorting study in the course of my PhD. While research articles describe the steps of this analysis (i.e. pairwise similarity scores --> hierarchical cluster analysis --> dendrogram, see articles linked in question above), I did not find any open source tools that help to do this analysis. I used the widely used kardsort.com tool to collect the data, which refers to rather dated, closed source Windows software for analysis, which underpins my assumption of a lack of open source tools. The approach is rather simple, making use of scipy for hierarchical cluster analysis and dendrogram, adding a custom function to create the similarity scores, and putting everything in an easy-to-use pipeline. Nevertheless, in doing so, I think the cardsort package can help remove barriers for the application of this user research method (easy to use even for python beginners, no need to use closed source software that only runs on windows or expensive subscription tools). In any case I would have liked to have this package when I started my study :)

• If you made a pre-submission enquiry, please paste the link to the corresponding issue, forum post, or other discussion, or @tag the editor you contacted:

101

Technical checks

For details about the pyOpenSci packaging requirements, see our packaging guide. Confirm each of the following by checking the box. This package:

• [x] does not violate the Terms of Service of any service it interacts with.
• [x] uses an OSI approved license.
• [x] contains a README with instructions for installing the development version.
• [x] includes documentation with examples for all functions.
• [x] contains a tutorial with examples of its essential functions and uses.
• [x] has a test suite.
• [x] has continuous integration setup, such as GitHub Actions CircleCI, and/or others.

Publication Options

• [ ] Do you wish to automatically submit to the Journal of Open Source Software? If so:

JOSS Checks

- [ ] The package has an **obvious research application** according to JOSS's definition in their [submission requirements][JossSubmissionRequirements]. Be aware that completing the pyOpenSci review process **does not** guarantee acceptance to JOSS. Be sure to read their submission requirements (linked above) if you are interested in submitting to JOSS. - [ ] The package is not a "minor utility" as defined by JOSS's [submission requirements][JossSubmissionRequirements]: "Minor ‘utility’ packages, including ‘thin’ API clients, are not acceptable." pyOpenSci welcomes these packages under "Data Retrieval", but JOSS has slightly different criteria. - [ ] The package contains a `paper.md` matching [JOSS's requirements][JossPaperRequirements] with a high-level description in the package root or in `inst/`. - [ ] The package is deposited in a long-term repository with the DOI: *Note: Do not submit your package separately to JOSS*

Are you OK with Reviewers Submitting Issues and/or pull requests to your Repo Directly?

This option will allow reviewers to open smaller issues that can then be linked to PR's rather than submitting a more dense text based review. It will also allow you to demonstrate addressing the issue via PR links.

• [x] Yes I am OK with reviewers submitting requested changes as issues to my repo. Reviewers will then link to the issues in their submitted review.

Confirm each of the following by checking the box.

• [x] I have read the author guide.
• [x] I expect to maintain this package for at least 2 years and can help find a replacement for the maintainer (team) if needed.

Please fill out our survey

• [x] Last but not least please fill out our pre-review survey. This helps us track submission and improve our peer review process. We will also ask our reviewers and editors to fill this out.

P.S. Have feedback/comments about our review process? Leave a comment here

Editor and Review Templates

The editor template can be found here.

The review template can be found here.

[khynder]

Checked!

[Robaina]

so sorry for the delay... checked!

[Batalex]

🎉 cardsort has been approved by pyOpenSci! Thank you @katoss for submitting cardsort and many thanks to @Robaina and @khynder for reviewing this package! 🐈‍⬛

Author Wrap Up Tasks

There are a few things left to do to wrap up this submission:

• [x] Activate Zenodo watching the repo if you haven't already done so.
• [x] Tag and create a release to create a Zenodo version and DOI.
• [x] Add the badge for pyOpenSci peer-review to the README.md of cardsort. The badge should be [![pyOpenSci](https://tinyurl.com/y22nb8up)](https://github.com/pyOpenSci/software-review/issues/102).
• [x] Please fill out the post-review survey. All maintainers and reviewers should fill this out.

@Robaina, @khynder we would greatly appreciate if you could fill the post-review survey as well!
If any of you have any feedback for us about the review process please feel free to share it here. We are always looking to improve our process and documentation in the peer-review-guide.

[Robaina]

Congrats @katoss!

[katoss]

Amazing! I just added the badge and did the survey. Thanks a lot for spending the time on reviewing everything so thoroughly @Batalex @Robaina and @khynder! The review was really helpful and instructive, and improved the package a lot :)

[Batalex]

Congrats @katoss, I am glad this review went so well. I am truly pleased that we were able to add value to your work. Best of luck with your thesis! 🤞 I will be closing this submission issue now.

We offer authors to join us on our slack / discourse. Let me know if you are interested in joining the slack.
We can also promote cardsort with a blog post, if you feel like you have not written enough with your thesis 😁 This is what it could look like.

[katoss]

Amazing, thanks @Batalex :) And yes, I would like to join the slack. Could you send me an invite? A blog article sounds like a good idea, I will try to write a draft some time this month.

I saw that the PR to update the pyOpenSci website with the cardsort package is still open and has no reviewer assigned. Is normal that it just takes a while, or do you think it would be better if I send a reminder?

[Batalex]

I just invited you 🐈‍⬛

Is normal that it just takes a while, or do you think it would be better if I send a reminder?

It seems that we have an issue with pre-commit.ci, but it should be fixed soon

[lwasser]

hey there @katoss 👋 thank you for mentioning this issue in our website update pr!! i saw it. it took me a while to merge as there were some bugs in the build that i wanted to fix before merging. it would have broken the website! but generally these will get merged more quickly - as that build becomes a bit more stable.

cardsort is now listed on our website!! 🚀 and we look forward to seeing you around slack!!

thank you everyone for your work on this pr!! and @Batalex for leading the charge on the review ✨ !!!