[PRE REVIEW]: ReLax: Efficient and Scalable Recourse Explanation Benchmarking using JAX

[editorialbot]

Submitting author: !--author-handle-->@BirkhoffG<!--end-author-handle-- (Hangzhi Guo) Repository: https://github.com/BirkhoffG/jax-relax/ Branch with paper.md (empty if default branch): joss Version: v0.2.4 Editor: !--editor-->@Fei-Tao<!--end-editor-- Reviewers: @GarrettMerz, @duhd1993 Managing EiC: Arfon Smith

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/ed8b635440789ebe1ff496021455df1e"><img src="https://joss.theoj.org/papers/ed8b635440789ebe1ff496021455df1e/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/ed8b635440789ebe1ff496021455df1e/status.svg)](https://joss.theoj.org/papers/ed8b635440789ebe1ff496021455df1e)

Author instructions

Thanks for submitting your paper to JOSS @BirkhoffG. Currently, there isn't a JOSS editor assigned to your paper.

@BirkhoffG if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). You can search the list of people that have already agreed to review and may be suitable for this submission.

Editor instructions

The JOSS submission bot @editorialbot is here to help you find and assign reviewers and start the main review. To find out what @editorialbot can do for you type:

@editorialbot commands

[editorialbot]

Hello human, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

[editorialbot]

Software report:

github.com/AlDanial/cloc v 1.88  T=0.15 s (544.4 files/s, 159641.0 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          38           1009            734           5796
Jupyter Notebook                30              0           9144           5784
TeX                              1            171             13           1277
YAML                             8             19              8            228
Markdown                         3             88              0            223
CSS                              1             10              6             58
INI                              1              1              0             37
Sass                             1              3              4             17
SVG                              1              0              0              1
-------------------------------------------------------------------------------
SUM:                            84           1301           9909          13421
-------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository

[editorialbot]

Wordcount for paper.md is 1177

[editorialbot]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[editorialbot]

Five most similar historical JOSS papers:

survxai: an R package for structure-agnostic explanations of survival models Submitting author: @AleksandraDabrowska Handling editor: @yochannah (Retired) Reviewers: @hiendn, @dirmeier Similarity score: 0.7955

High-performance neural population dynamics modeling enabled by scalable computational infrastructure Submitting author: @a9p Handling editor: @emdupre (Active) Reviewers: @richford, @tachukao Similarity score: 0.7946

modelStudio: Interactive Studio with Explanations for ML Predictive Models Submitting author: @hbaniecki Handling editor: @terrytangyuan (Retired) Reviewers: @acolum, @expectopatronum Similarity score: 0.7895

FreqAI: generalizing adaptive modeling for chaotic time-series market forecasts Submitting author: @robcaulk Handling editor: @Fei-Tao (Active) Reviewers: @ady00, @shagunsodhani Similarity score: 0.7858

Efficiently Learning Relative Similarity Embeddings with Crowdsourcing Submitting author: @stsievert Handling editor: @ajstewartlang (Active) Reviewers: @hoechenberger, @stain, @jorgedch Similarity score: 0.7854

⚠️ Note to editors: If these papers look like they might be a good match, click through to the review issue for that paper and invite one or more of the authors before considering asking the reviewers of these papers to review again for JOSS.

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1145/3375627.3375812 is OK
- 10.1145/3351095.3375624 is OK
- 10.1145/3447548.3467333 is OK
- 10.1145/3375627.3375812 is OK
- 10.1145/3580305.3599290 is OK
- 10.1109/CVPR.2009.5206848 is OK
- 10.18653/v1/N19-1423 is OK

MISSING DOIs

- 10.2139/ssrn.3254511 may be a valid DOI for title: The European Union general data protection regulation: what it is and what it means
- 10.1109/access.2019.2909068 may be a valid DOI for title: Badnets: Evaluating backdooring attacks on deep neural networks
- 10.1609/aaai.v36i6.20571 may be a valid DOI for title: Efficient robust training via backward smoothing
- 10.24963/ijcai.2018/520 may be a valid DOI for title: Curriculum adversarial training
- 10.1109/access.2021.3051315 may be a valid DOI for title: A survey of contrastive and counterfactual explanation generation methods for explainable artificial intelligence
- 10.1145/3603195.3603204 may be a valid DOI for title: The hidden assumptions behind counterfactual explanations and principal reasons
- 10.1109/dsaa.2018.00018 may be a valid DOI for title: Explaining explanations: An overview of interpretability of machine learning
- 10.3386/w23180 may be a valid DOI for title: Human decisions and machine predictions
- 10.1145/3236386.3241340 may be a valid DOI for title: The mythos of model interpretability
- 10.1038/s42256-019-0048-x may be a valid DOI for title: Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead
- 10.1145/2487575.2487579 may be a valid DOI for title: Accurate intelligible models with pairwise interactions
- 10.1214/15-aoas848 may be a valid DOI for title: Interpretable classifiers using rules and bayesian analysis: Building a better stroke prediction model
- 10.1145/3306618.3314229 may be a valid DOI for title: Faithful and customizable explanations of black box models
- 10.2139/ssrn.3063289 may be a valid DOI for title: Counterfactual explanations without opening the black box: Automated decisions and the GDPR
- 10.1093/bioinformatics/btq134 may be a valid DOI for title: Permutation importance: a corrected feature importance measure
- 10.1609/aaai.v32i1.11771 may be a valid DOI for title: Deep Learning for Case-Based Reasoning Through Prototypes: A Neural Network That Explains Its Predictions
- 10.1145/3313831.3376219 may be a valid DOI for title: Interpreting Interpretability: Understanding Data Scientists’ Use of Interpretability Tools for Machine Learning
- 10.1145/3290605.3300234 may be a valid DOI for title: Human-centered tools for coping with imperfect algorithms during medical decision-making
- 10.1145/3290605.3300831 may be a valid DOI for title: Designing theory-driven user-centric explainable AI
- 10.1609/hcomp.v7i1.5280 may be a valid DOI for title: Human evaluation of models built for interpretability
- 10.1016/j.artint.2018.07.007 may be a valid DOI for title: Explanation in artificial intelligence: Insights from the social sciences
- 10.1609/aaai.v33i01.33013681 may be a valid DOI for title: Interpretation of neural networks is fragile
- 10.1145/3287560.3287574 may be a valid DOI for title: Explaining explanations in AI
- 10.1609/aaai.v34i01.5427 may be a valid DOI for title: AdaCare: Explainable Clinical Health Status Representation Learning via Scale-Adaptive Feature Extraction and Recalibration
- 10.1007/978-3-030-81907-1_9 may be a valid DOI for title: How to Design AI for Social Good: Seven Essential Factors
- 10.1609/aaai.v30i2.19070 may be a valid DOI for title: Deploying PAWS: Field Optimization of the Protection Assistant for Wildlife Security.
- 10.1145/2783258.2788620 may be a valid DOI for title: A machine learning framework to identify students at risk of adverse academic outcomes
- 10.1145/3351095.3372850 may be a valid DOI for title: Explaining machine learning classifiers through diverse counterfactual explanations
- 10.1109/cvpr.2019.00880 may be a valid DOI for title: Learning to explain with complemental examples
- 10.24963/ijcai.2018/761 may be a valid DOI for title: Bridging the Gap Between Theory and Practice in Influence Maximization: Raising Awareness about HIV among Homeless Youth.
- 10.1007/978-3-030-56485-8_3 may be a valid DOI for title: Random forests
- 10.1145/1772690.1772758 may be a valid DOI for title: A contextual-bandit approach to personalized news article recommendation
- 10.1016/j.knosys.2011.07.021 may be a valid DOI for title: A collaborative filtering approach to mitigate the new user cold start problem
- 10.1145/170036.170072 may be a valid DOI for title: Mining association rules between sets of items in large databases
- 10.1007/978-3-030-86520-7_40 may be a valid DOI for title: Interpretable counterfactual explanations guided by prototypes
- 10.1609/aaai.v32i1.11491 may be a valid DOI for title: Anchors: High-Precision Model-Agnostic Explanations.
- 10.1109/cvpr.2016.90 may be a valid DOI for title: Deep residual learning for image recognition
- 10.7551/mitpress/10761.003.0012 may be a valid DOI for title: Adversarial Perturbations of Deep Neural Networks
- 10.1016/j.eswa.2007.12.020 may be a valid DOI for title: The comparisons of data mining techniques for the predictive accuracy of probability of default of credit card clients
- 10.1109/iccv.2015.123 may be a valid DOI for title: Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
- 10.1145/3287560.3287566 may be a valid DOI for title: Actionable recourse in linear classification
- 10.1017/s0269888921000102 may be a valid DOI for title: Contrastive explanation: A structural-model approach
- 10.1145/3366423.3380087 may be a valid DOI for title: Learning model-agnostic counterfactual explanations for tabular data
- 10.1145/3583780.3615040 may be a valid DOI for title: RoCourseNet: Robust Training of a Prediction Aware Recourse Model
- 10.1080/10691898.2018.1434342 may be a valid DOI for title: “Should This Loan be Approved or Denied?”: A Large Dataset with Class Assignment Guidelines
- 10.1109/test.2018.8624792 may be a valid DOI for title: Influence-directed explanations for deep convolutional networks
- 10.1145/3065386 may be a valid DOI for title: ImageNet Classification with Deep Convolutional Neural Networks
- 10.1007/s10115-007-0095-1 may be a valid DOI for title: Forecasting skewed biased stochastic ozone days: analyses, solutions and beyond
- 10.1021/ci4000213 may be a valid DOI for title: Quantitative structure–activity relationship models for ready biodegradability of chemicals
- 10.1145/3534678.3539065 may be a valid DOI for title: Rax: Composable Learning-to-Rank using JAX
- 10.1109/cvpr52688.2022.02070 may be a valid DOI for title: Scenic: A JAX library for computer vision research and beyond
- 10.1145/3580305.3599343 may be a valid DOI for title: Feature-based Learning for Diverse and Privacy-Preserving Counterfactual Explanations

INVALID DOIs

- None

[BirkhoffG]

Here is a list of potential reviewers:

• hbaniecki
• JHoelli
• expectopatronum
• xiaohk
• ApocalyVec
• allie-tatarian
• mikemahoney218
• alessandroBombelli

[arfon]

@BirkhoffG – thanks for your submission to JOSS. We're currently managing a large backlog of submissions and the editor most appropriate for your area is already rather busy.

For now, we will need to waitlist this paper and process it as the queue reduces. Thanks for your patience!

While we wait, please take a look at your BibTeX file here. Either remove the entries you're not citing in the paper, or add the DOIs where they match. Thank you!

[arfon]

@editorialbot invite @Fei-Tao as editor

👋 @Fei-Tao – would you be willing to edit this submission for JOSS?

[editorialbot]

Invitation to edit this submission sent!

[Fei-Tao]

Hi @arfon, I would like to edit this submission. Thanks for your invitation.

[Fei-Tao]

@editorialbot assign me as editor

[editorialbot]

Assigned! @Fei-Tao is now the editor

[Fei-Tao]

Hi @bbortey9, @Saran-nns, @GarrettMerz, @WangKehan573, @duhd1993, I hope this message finds you well. If you'd like to review this submission, please comment here! Feel free to unsubscribe from this issue if you are not interested. Thank you for your time!

[GarrettMerz]

Hi, all-

I'm happy to help! I'm slightly less familiar with JAX compared to Pytorch/TF, so it might be optimal to have at least one other set of eyes on this in case I miss anything, but I'm happy to review!

-Garrett

[duhd1993]

Thanks for the invite. I'd be happy to take a look. I have experience with JAX.

[Fei-Tao]

Hi @GarrettMerz, @duhd1993, thanks so much for agreeing to review this submission. I will add you to the reviewer list.

[Fei-Tao]

@editorialbot add @GarrettMerz as reviewer

[editorialbot]

@GarrettMerz added to the reviewers list!

[Fei-Tao]

@editorialbot add @duhd1993 as reviewer

[editorialbot]

@duhd1993 added to the reviewers list!

[Fei-Tao]

@editorialbot start review

[editorialbot]

OK, I've started the review over in https://github.com/openjournals/joss-reviews/issues/6567.

[Fei-Tao]

Thanks everyone! I have started the review process. Looking forward to your reviews in https://github.com/openjournals/joss-reviews/issues/6567.