Ethics section discussion and proposals

[Benjamin-Lee]

As @SiminaB mentioned in #252, we should discuss the ethical use of deep learning, particularly with respect to biological and biomedical sciences. There are two major topics we need to figure out:

1. What ethical issues are important to cover?
2. Where should the discussion go?

[rasbt]

I think covering ethical issues can potentially be interesting and useful, thanks for thinking about that @SiminaB . I think that the focus should be on biology-related ethics issues that are particular to employing deep learning, to avoid getting into broad discussions and to keep the article focused. The target audience is likely biologists trying to employ deep learning (rather than deep learning folks getting into bio), so issues that arise when biologists shift from traditional methods too deep learning might be particularly interesting. Issues that are privacy-related (which we slightly touched on already) and maybe issues that arise to different model interpretability constraints.

[SiminaB]

I totally agree it should be biology-focused! A few things I was thinking about (and I 100% defer to @juancarmona - I am not trained in this, literally am familiar with these things from Twitter discussions, seminars, and the like):

• How can DL inadvertently be used to perpetuate existing stereotypes eg racist and sexist ones, and what can be done about it (first step is to realize that this can be an issue, of course)? We know this can happen either because of the training set (eg training set consists exclusively of individuals of European descent, then model is used on a more diverse population) or because of predictions being incorrectly interpreted due to confounding (eg the training set has doctors and nurses and most doctors are men and most nurses are women, therefore going forward gender is either explicitly or implicitly used to play an outsized role in predicting career choice.) Real direct harm can occur when using face recognition to arrest suspects if a DL approach performs better with individuals of European descent vs. Black individuals, but even if it's not such a direct individual-specific prediction, it can lead to errors that get perpetuated in the literature, coupled with a false confidence that "it must be correct, since DL is clearly an unbiased way to do about things." One way why this is potentially more problematic with DL vs. traditional ML is because DL models may have less interpretability so it can be unclear what features influence prediction.

• Issues about whether predictions should even be done in some cases - eg is there ever a good reason to predict sexual orientation from faces? Just because we can do something doesn't mean we should.

• Possible solutions - Timnit Gebru suggests the use of "model cards" https://dl.acm.org/doi/abs/10.1145/3287560.3287596 that can "encourage ... transparent reporting" of ML models.

Model cards are short documents accompanying trained machine learning models that provide benchmarked evaluation in a variety of conditions, such as across different cultural, demographic, or phenotypic groups (e.g., race, geographic location, sex, Fitzpatrick skin type [15]) and intersectional groups (e.g., age and race, or sex and Fitzpatrick skin type) that are relevant to the intended application domains. Model cards also disclose the context in which models are intended to be used, details of the performance evaluation procedures, and other relevant information.

Another solution is of course making sure that the field is open to a diverse set of individuals, both across genders/ethnicities etc and across different fields (biology, CS, but also ethics, other humanities disciplines).

None of these are DL-specific (could be applicable to ML in general), though many of the scenarios with recognized issues come from image recognition, which tends to be pretty DL-heavy.

Again, I will defer to @juancarmona - these were just my thoughts when suggesting this!

[rasbt]

Good points! I can see how this can be an issue when working with human subjects or samples (like DNA or RNA) from humans and there is a lack of diversity in the population. I think we can mention that as a general problem, and maybe we can also find a good biological study (ideally using DL) where the authors took great care in sampling that is representative of the population.

EDIT: regarding point

Just because we can do something doesn't mean we should.

That just reminds me of an example related to contexts where diverse and representative samples are important. I.e., I remember certain efforts towards life expectancy predictions based on face images and epigenetic data, which can/is being used by life insurance underwriters.

EDIT 2:

Just flipped through the titles of papers that cited Mitchell et al.'s model card paper @SiminaB linked above ( https://dl.acm.org/doi/abs/10.1145/3287560.3287596). Was thinking maybe we can find an interesting biology application of model cards we can describe. The following ones seem to be potentially bio-related (haven't read them, yet, will try to check them in the next couple of days to see whether there is potentially something interesting in there):

• What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use, https://arxiv.org/abs/1905.05134
• "The human body is a black box": supporting clinical decision-making with deep learning Share on, https://dl.acm.org/doi/abs/10.1145/3351095.3372827?casa_token=K_27-e_GutoAAAAA:vLAhub83-Hu-CduyCdDLq0vLU_TlM432jQBjNapcEQo4ZTZHCPN_tekUCzTIlcs73f--f-8Gt9rY
• BRAVE-NET: Fully Automated Arterial Brain Vessel Segmentation In Patients with Cerebrovascular Disease, https://www.researchgate.net/publication/344443099_BRAVE-NET_Fully_Automated_Arterial_Brain_Vessel_Segmentation_In_Patients_with_Cerebrovascular_Disease
• Risk of Training Diagnostic Algorithms on Data with Demographic Bias, https://arxiv.org/abs/2005.10050

[fmaguire]

While it may seem self-evident to folks that are in groups/institutions that do pay attention to it, given the lack of familiarity with Research Ethics Review/Institutional Review Boards in a lot of Bio and CS folks (compared to med/psych/sociology etc) I think it might be worth having a sentence in the ethics section along the lines of: "Ensure that where necessary your proposed project is compliant with the research ethics approval policies of your institution (e.g., institutional review boards)."

While IRBs are a bit of a blunt object and can be frustrating, there is a terrifying amount of deep learning research that seems to act like think don't even exist!

[rasbt]

"Ensure that where necessary your proposed project is compliant with the research ethics approval policies of your institution (e.g., institutional review boards)."

I like the suggestion. It is short and on point.

[Benjamin-Lee]

While it may seem self-evident to folks that are in groups/institutions that do pay attention to it, given the lack of familiarity with Research Ethics Review/Institutional Review Boards in a lot of Bio and CS folks (compared to med/psych/sociology etc) I think it might be worth having a sentence in the ethics section along the lines of: "Ensure that where necessary your proposed project is compliant with the research ethics approval policies of your institution (e.g., institutional review boards)."

While IRBs are a bit of a blunt object and can be frustrating, there is a terrifying amount of deep learning research that seems to act like think don't even exist!

@fmaguire, just to follow up with this, @juancarmona's PR (#299) does include this recommendation.