Abstract
FrOoDo is an easy-to-use and flexible framework for Out-of-Distribution detection tasks in digital pathology. It can be used with PyTorch classification and segmentation models, and its modular design allows for easy extension. The goal is to automate the task of OoD Evaluation such that research can focus on the main goal of either designing new models, new methods or evaluating a new dataset. The code can be found at https://github.com/MECLabTUDA/FrOoDo.
Keyword: expected calibration error
There is no result
Keyword: overconfident
There is no result
Keyword: overconfidence
There is no result
Keyword: confidence
Personalised recommendations of sleep behaviour with neural networks using sleep diaries captured in Sleepio
Authors: Alejo Nevado-Holgado, Colin Espie, Maria Liakata, Alasdair Henry, Jenny Gu, Niall Taylor, Kate Saunders, Tom Walker, Chris Miller
Abstract
SleepioTM is a digital mobile phone and web platform that uses techniques from cognitive behavioural therapy (CBT) to improve sleep in people with sleep difficulty. As part of this process, Sleepio captures data about the sleep behaviour of the users that have consented to such data being processed. For neural networks, the scale of the data is an opportunity to train meaningful models translatable to actual clinical practice. In collaboration with Big Health, the therapeutics company that created and utilizes Sleepio, we have analysed data from a random sample of 401,174 sleep diaries and built a neural network to model sleep behaviour and sleep quality of each individual in a personalised manner. We demonstrate that this neural network is more accurate than standard statistical methods in predicting the sleep quality of an individual based on his/her behaviour from the last 10 days. We compare model performance in a wide range of hyperparameter settings representing various scenarios. We further show that the neural network can be used to produce personalised recommendations of what sleep habits users should follow to maximise sleep quality, and show that these recommendations are substantially better than the ones generated by standard methods. We finally show that the neural network can explain the recommendation given to each participant and calculate confidence intervals for each prediction, all of which are essential for clinicians to be able to adopt such a tool in clinical practice.
On Interactive Explanations as Non-Monotonic Reasoning
Abstract
Recent work shows issues of consistency with explanations, with methods generating local explanations that seem reasonable instance-wise, but that are inconsistent across instances. This suggests not only that instance-wise explanations can be unreliable, but mainly that, when interacting with a system via multiple inputs, a user may actually lose confidence in the system. To better analyse this issue, in this work we treat explanations as objects that can be subject to reasoning and present a formal model of the interactive scenario between user and system, via sequences of inputs, outputs, and explanations. We argue that explanations can be thought of as committing to some model behaviour (even if only prima facie), suggesting a form of entailment, which, we argue, should be thought of as non-monotonic. This allows: 1) to solve some considered inconsistencies in explanation, such as via a specificity relation; 2) to consider properties from the non-monotonic reasoning literature and discuss their desirability, gaining more insight on the interactive explanation scenario.
Taming Multi-Output Recommenders for Software Engineering
Abstract
Recommender systems are a valuable tool for software engineers. For example, they can provide developers with a ranked list of files likely to contain a bug, or multiple auto-complete suggestions for a given method stub. However, the way these recommender systems interact with developers is often rudimentary -- a long list of recommendations only ranked by the model's confidence. In this vision paper, we lay out our research agenda for re-imagining how recommender systems for software engineering communicate their insights to developers. When issuing recommendations, our aim is to recommend diverse rather than redundant solutions and present them in ways that highlight their differences. We also want to allow for seamless and interactive navigation of suggestions while striving for holistic end-to-end evaluations. By doing so, we believe that recommender systems can play an even more important role in helping developers write better software.
Adaptive Temperature Scaling for Robust Calibration of Deep Neural Networks
Authors: Sergio A. Balanya, Juan Maroñas, Daniel Ramos
Abstract
In this paper, we study the post-hoc calibration of modern neural networks, a problem that has drawn a lot of attention in recent years. Many calibration methods of varying complexity have been proposed for the task, but there is no consensus about how expressive these should be. We focus on the task of confidence scaling, specifically on post-hoc methods that generalize Temperature Scaling, we call these the Adaptive Temperature Scaling family. We analyse expressive functions that improve calibration and propose interpretable methods. We show that when there is plenty of data complex models like neural networks yield better performance, but are prone to fail when the amount of data is limited, a common situation in certain post-hoc calibration applications like medical diagnosis. We study the functions that expressive methods learn under ideal conditions and design simpler methods but with a strong inductive bias towards these well-performing functions. Concretely, we propose Entropy-based Temperature Scaling, a simple method that scales the confidence of a prediction according to its entropy. Results show that our method obtains state-of-the-art performance when compared to others and, unlike complex models, it is robust against data scarcity. Moreover, our proposed model enables a deeper interpretation of the calibration process.
The Unnecessity of Assuming Statistically Independent Tests in Bayesian Software Reliability Assessments
Abstract
When assessing a software-based system, the results of statistical inference on operational testing data can provide strong support for software reliability claims. For inference, this data (i.e. software successes and failures) is often assumed to arise in an independent, identically distributed (i.i.d.) manner. In this paper we show how conservative Bayesian approaches make this assumption unnecessary, by incorporating one's doubts about the assumption into the assessment. We derive conservative confidence bounds on a system's probability of failure on demand (pfd), when operational testing reveals no failures. The generality and utility of the confidence bounds are demonstrated in the assessment of a nuclear power-plant safety-protection system, under varying levels of skepticism about the i.i.d. assumption.
DNNShield: Dynamic Randomized Model Sparsification, A Defense Against Adversarial Machine Learning
Authors: Mohammad Hossein Samavatian, Saikat Majumdar, Kristin Barber, Radu Teodorescu
Abstract
DNNs are known to be vulnerable to so-called adversarial attacks that manipulate inputs to cause incorrect results that can be beneficial to an attacker or damaging to the victim. Recent works have proposed approximate computation as a defense mechanism against machine learning attacks. We show that these approaches, while successful for a range of inputs, are insufficient to address stronger, high-confidence adversarial attacks. To address this, we propose DNNSHIELD, a hardware-accelerated defense that adapts the strength of the response to the confidence of the adversarial input. Our approach relies on dynamic and random sparsification of the DNN model to achieve inference approximation efficiently and with fine-grain control over the approximation error. DNNSHIELD uses the output distribution characteristics of sparsified inference compared to a dense reference to detect adversarial inputs. We show an adversarial detection rate of 86% when applied to VGG16 and 88% when applied to ResNet50, which exceeds the detection rate of the state of the art approaches, with a much lower overhead. We demonstrate a software/hardware-accelerated FPGA prototype, which reduces the performance impact of DNNSHIELD relative to software-only CPU and GPU implementations.
DSLA: Dynamic smooth label assignment for efficient anchor-free object detection
Authors: Hu Su, Yonghao He, Jiabin Zhang, Wei Zou, Bin Fan
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Abstract
Anchor-free detectors basically formulate object detection as dense classification and regression. For popular anchor-free detectors, it is common to introduce an individual prediction branch to estimate the quality of localization. The following inconsistencies are observed when we delve into the practices of classification and quality estimation. Firstly, for some adjacent samples which are assigned completely different labels, the trained model would produce similar classification scores. This violates the training objective and leads to performance degradation. Secondly, it is found that detected bounding boxes with higher confidences contrarily have smaller overlaps with the corresponding ground-truth. Accurately localized bounding boxes would be suppressed by less accurate ones in the Non-Maximum Suppression (NMS) procedure. To address the inconsistency problems, the Dynamic Smooth Label Assignment (DSLA) method is proposed. Based on the concept of centerness originally developed in FCOS, a smooth assignment strategy is proposed. The label is smoothed to a continuous value in [0, 1] to make a steady transition between positive and negative samples. Intersection-of-Union (IoU) is predicted dynamically during training and is coupled with the smoothed label. The dynamic smooth label is assigned to supervise the classification branch. Under such supervision, quality estimation branch is naturally merged into the classification branch, which simplifies the architecture of anchor-free detector. Comprehensive experiments are conducted on the MS COCO benchmark. It is demonstrated that, DSLA can significantly boost the detection accuracy by alleviating the above inconsistencies for anchor-free detectors. Our codes are released at https://github.com/YonghaoHe/DSLA.
Guidance on the Safety Assurance of Autonomous Systems in Complex Environments (SACE)
Authors: Richard Hawkins, Matt Osborne, Mike Parsons, Mark Nicholson, John McDermid, Ibrahim Habli
Subjects: Software Engineering (cs.SE); Systems and Control (eess.SY)
Abstract
Autonomous systems (AS) are systems that have the capability to take decisions free from direct human control. AS are increasingly being considered for adoption for applications where their behaviour may cause harm, such as when used for autonomous driving, medical applications or in domestic environments. For such applications, being able to ensure and demonstrate (assure) the safety of the operation of the AS is crucial for their adoption. This can be particularly challenging where AS operate in complex and changing real-world environments. Establishing justified confidence in the safety of AS requires the creation of a compelling safety case. This document introduces a methodology for the Safety Assurance of Autonomous Systems in Complex Environments (SACE). SACE comprises a set of safety case patterns and a process for (1) systematically integrating safety assurance into the development of the AS and (2) for generating the evidence base for explicitly justifying the acceptable safety of the AS.
What do Deep Neural Networks Learn in Medical Images?
Abstract
Deep learning is increasingly gaining rapid adoption in healthcare to help improve patient outcomes. This is more so in medical image analysis which requires extensive training to gain the requisite expertise to become a trusted practitioner. However, while deep learning techniques have continued to provide state-of-the-art predictive performance, one of the primary challenges that stands to hinder this progress in healthcare is the opaque nature of the inference mechanism of these models. So, attribution has a vital role in building confidence in stakeholders for the predictions made by deep learning models to inform clinical decisions. This work seeks to answer the question: what do deep neural network models learn in medical images? In that light, we present a novel attribution framework using adaptive path-based gradient integration techniques. Results show a promising direction of building trust in domain experts to improve healthcare outcomes by allowing them to understand the input-prediction correlative structures, discover new bio-markers, and reveal potential model biases.
Keyword: scaling
DAS: Densely-Anchored Sampling for Deep Metric Learning
Authors: Lizhao Liu, Shangxin Huang, Zhuangwei Zhuang, Ran Yang, Mingkui Tan, Yaowei Wang
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Abstract
Deep Metric Learning (DML) serves to learn an embedding function to project semantically similar data into nearby embedding space and plays a vital role in many applications, such as image retrieval and face recognition. However, the performance of DML methods often highly depends on sampling methods to choose effective data from the embedding space in the training. In practice, the embeddings in the embedding space are obtained by some deep models, where the embedding space is often with barren area due to the absence of training points, resulting in so called "missing embedding" issue. This issue may impair the sample quality, which leads to degenerated DML performance. In this work, we investigate how to alleviate the "missing embedding" issue to improve the sampling quality and achieve effective DML. To this end, we propose a Densely-Anchored Sampling (DAS) scheme that considers the embedding with corresponding data point as "anchor" and exploits the anchor's nearby embedding space to densely produce embeddings without data points. Specifically, we propose to exploit the embedding space around single anchor with Discriminative Feature Scaling (DFS) and multiple anchors with Memorized Transformation Shifting (MTS). In this way, by combing the embeddings with and without data points, we are able to provide more embeddings to facilitate the sampling process thus boosting the performance of DML. Our method is effortlessly integrated into existing DML frameworks and improves them without bells and whistles. Extensive experiments on three benchmark datasets demonstrate the superiority of our method.
enpheeph: A Fault Injection Framework for Spiking and Compressed Deep Neural Networks
Authors: Alessio Colucci, Andreas Steininger, Muhammad Shafique
Subjects: Neural and Evolutionary Computing (cs.NE); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
Abstract
Research on Deep Neural Networks (DNNs) has focused on improving performance and accuracy for real-world deployments, leading to new models, such as Spiking Neural Networks (SNNs), and optimization techniques, e.g., quantization and pruning for compressed networks. However, the deployment of these innovative models and optimization techniques introduces possible reliability issues, which is a pillar for DNNs to be widely used in safety-critical applications, e.g., autonomous driving. Moreover, scaling technology nodes have the associated risk of multiple faults happening at the same time, a possibility not addressed in state-of-the-art resiliency analyses. Towards better reliability analysis for DNNs, we present enpheeph, a Fault Injection Framework for Spiking and Compressed DNNs. The enpheeph framework enables optimized execution on specialized hardware devices, e.g., GPUs, while providing complete customizability to investigate different fault models, emulating various reliability constraints and use-cases. Hence, the faults can be executed on SNNs as well as compressed networks with minimal-to-none modifications to the underlying code, a feat that is not achievable by other state-of-the-art tools. To evaluate our enpheeph framework, we analyze the resiliency of different DNN and SNN models, with different compression techniques. By injecting a random and increasing number of faults, we show that DNNs can show a reduction in accuracy with a fault rate as low as 7 x 10 ^ (-7) faults per parameter, with an accuracy drop higher than 40%. Run-time overhead when executing enpheeph is less than 20% of the baseline execution time when executing 100 000 faults concurrently, at least 10x lower than state-of-the-art frameworks, making enpheeph future-proof for complex fault injection scenarios. We release enpheeph at https://github.com/Alexei95/enpheeph.
Adaptive Temperature Scaling for Robust Calibration of Deep Neural Networks
Authors: Sergio A. Balanya, Juan Maroñas, Daniel Ramos
Abstract
In this paper, we study the post-hoc calibration of modern neural networks, a problem that has drawn a lot of attention in recent years. Many calibration methods of varying complexity have been proposed for the task, but there is no consensus about how expressive these should be. We focus on the task of confidence scaling, specifically on post-hoc methods that generalize Temperature Scaling, we call these the Adaptive Temperature Scaling family. We analyse expressive functions that improve calibration and propose interpretable methods. We show that when there is plenty of data complex models like neural networks yield better performance, but are prone to fail when the amount of data is limited, a common situation in certain post-hoc calibration applications like medical diagnosis. We study the functions that expressive methods learn under ideal conditions and design simpler methods but with a strong inductive bias towards these well-performing functions. Concretely, we propose Entropy-based Temperature Scaling, a simple method that scales the confidence of a prediction according to its entropy. Results show that our method obtains state-of-the-art performance when compared to others and, unlike complex models, it is robust against data scarcity. Moreover, our proposed model enables a deeper interpretation of the calibration process.
Is current research on adversarial robustness addressing the right problem?
Authors: Ali Borji
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR); Machine Learning (cs.LG)
Abstract
Short answer: Yes, Long answer: No! Indeed, research on adversarial robustness has led to invaluable insights helping us understand and explore different aspects of the problem. Many attacks and defenses have been proposed over the last couple of years. The problem, however, remains largely unsolved and poorly understood. Here, I argue that the current formulation of the problem serves short term goals, and needs to be revised for us to achieve bigger gains. Specifically, the bound on perturbation has created a somewhat contrived setting and needs to be relaxed. This has misled us to focus on model classes that are not expressive enough to begin with. Instead, inspired by human vision and the fact that we rely more on robust features such as shape, vertices, and foreground objects than non-robust features such as texture, efforts should be steered towards looking for significantly different classes of models. Maybe instead of narrowing down on imperceptible adversarial perturbations, we should attack a more general problem which is finding architectures that are simultaneously robust to perceptible perturbations, geometric transformations (e.g. rotation, scaling), image distortions (lighting, blur), and more (e.g. occlusion, shadow). Only then we may be able to solve the problem of adversarial vulnerability.
Implicit bulk-surface filtering method for node-based shape optimization and comparison of explicit and implicit filtering techniques
Authors: Reza Najian Asl, Kai-Uwe Bletzinger
Subjects: Numerical Analysis (math.NA); Optimization and Control (math.OC)
Abstract
This work studies shape filtering techniques, namely the convolution-based (explicit) and the PDE-based (implicit), and introduces an implicit bulk-surface filtering method to control the boundary smoothness and preserve the internal mesh quality simultaneously in the course of bulk (solid) shape optimization. To that end, volumetric mesh is filtered by the solution of pseudo-solid governing equations which are stiffened by the mesh-Jacobian and endowed with the Robin boundary condition which involves the Laplace-Beltrami operator on the mesh boundaries. Its superior performance from the non-simultaneous (sequential) treatment of boundary and internal meshes is demonstrated for the shape optimization of a complex solid structure. Well-established explicit filters, namely Gaussian and linear, and the Helmholtz/Sobolev-based (implicit) filter are critically examined in terms of consistency (rigid-body-movement production), geometric characteristics and the computational cost. It is shown that the implicit filtering is numerically more efficient and unconditionally consistent, compared to the explicit one. Supported by numerical experiments, a regularized Green's function is introduced as an equivalent explicit form of the Helmholtz/Sobolev filter. Furthermore, we give special attention to derive mesh-independent filtered sensitivities for node-based shape optimization with non-uniform meshes. It is shown that the mesh independent filtering can be achieved by scaling discrete sensitivities with the inverse of the mesh mass matrix.
Keyword: calibration
Geometric deep learning for computational mechanics Part II: Graph embedding for interpretable multiscale plasticity
Authors: Nikolaos N. Vlassis, WaiChing Sun
Subjects: Machine Learning (cs.LG); Disordered Systems and Neural Networks (cond-mat.dis-nn)
Abstract
The history-dependent behaviors of classical plasticity models are often driven by internal variables evolved according to phenomenological laws. The difficulty to interpret how these internal variables represent a history of deformation, the lack of direct measurement of these internal variables for calibration and validation, and the weak physical underpinning of those phenomenological laws have long been criticized as barriers to creating realistic models. In this work, geometric machine learning on graph data (e.g. finite element solutions) is used as a means to establish a connection between nonlinear dimensional reduction techniques and plasticity models. Geometric learning-based encoding on graphs allows the embedding of rich time-history data onto a low-dimensional Euclidean space such that the evolution of plastic deformation can be predicted in the embedded feature space. A corresponding decoder can then convert these low-dimensional internal variables back into a weighted graph such that the dominating topological features of plastic deformation can be observed and analyzed.
Adaptive Temperature Scaling for Robust Calibration of Deep Neural Networks
Authors: Sergio A. Balanya, Juan Maroñas, Daniel Ramos
Abstract
In this paper, we study the post-hoc calibration of modern neural networks, a problem that has drawn a lot of attention in recent years. Many calibration methods of varying complexity have been proposed for the task, but there is no consensus about how expressive these should be. We focus on the task of confidence scaling, specifically on post-hoc methods that generalize Temperature Scaling, we call these the Adaptive Temperature Scaling family. We analyse expressive functions that improve calibration and propose interpretable methods. We show that when there is plenty of data complex models like neural networks yield better performance, but are prone to fail when the amount of data is limited, a common situation in certain post-hoc calibration applications like medical diagnosis. We study the functions that expressive methods learn under ideal conditions and design simpler methods but with a strong inductive bias towards these well-performing functions. Concretely, we propose Entropy-based Temperature Scaling, a simple method that scales the confidence of a prediction according to its entropy. Results show that our method obtains state-of-the-art performance when compared to others and, unlike complex models, it is robust against data scarcity. Moreover, our proposed model enables a deeper interpretation of the calibration process.
A New Calibration Method for Industrial Robot Based on Step-Size Levenberg-Marquardt Algorithm
Abstract
Industrial robots play a vital role in automatic production, which have been widely utilized in industrial production activities, like handling and welding. However, due to an uncalibrated robot with machining tolerance and assembly tolerance, it suffers from low absolute positioning accuracy, which cannot satisfy the requirements of high-precision manufacture. To address this hot issue, we propose a novel calibration method based on an unscented Kalman filter and variable step-size Levenberg-Marquardt algorithm. This work has three ideas: a) proposing a novel variable step-size Levenberg-Marquardt algorithm to addresses the issue of local optimum in a Levenberg-Marquardt algorithm; b) employing an unscented Kalman filter to reduce the influence of the measurement noises; and c) developing a novel calibration method incorporating an unscented Kalman filter with a variable step-size Levenberg-Marquardt algorithm. Furthermore, we conduct enough experiments on an ABB IRB 120 industrial robot. From the experimental results, the proposed method achieves much higher calibration accuracy than some state-of-the-art calibration methods. Hence, this work is an important milestone in the field of robot calibration.
Abstract
We consider the problem of navigating a mobile robot towards a target in an unknown environment that is endowed with visual sensors, where neither the robot nor the sensors have access to global positioning information and only use first-person-view images. While prior work in sensor network based navigation uses explicit mapping and planning techniques, and are often aided by external positioning systems, we propose a vision-only based learning approach that leverages a Graph Neural Network (GNN) to encode and communicate relevant viewpoint information to the mobile robot. During navigation, the robot is guided by a model that we train through imitation learning to approximate optimal motion primitives, thereby predicting the effective cost-to-go (to the target). In our experiments, we first demonstrate generalizability to previously unseen environments with various sensor layouts. Simulation results show that by utilizing communication among the sensors and robot, we can achieve a $18.1\%$ improvement in success rate while decreasing path detour mean by $29.3\%$ and variability by $48.4\%$. This is done without requiring a global map, positioning data, nor pre-calibration of the sensor network. Second, we perform a zero-shot transfer of our model from simulation to the real world. To this end, we train a `translator' model that translates between {latent encodings of} real and simulated images so that the navigation policy (which is trained entirely in simulation) can be used directly on the real robot, without additional fine-tuning. Physical experiments demonstrate our effectiveness in various cluttered environments.
Keyword: out of distribution detection
There is no result
Keyword: out-of-distribution detection
FrOoDo: Framework for Out-of-Distribution Detection
Keyword: expected calibration error
There is no result
Keyword: overconfident
There is no result
Keyword: overconfidence
There is no result
Keyword: confidence
Personalised recommendations of sleep behaviour with neural networks using sleep diaries captured in Sleepio
On Interactive Explanations as Non-Monotonic Reasoning
Taming Multi-Output Recommenders for Software Engineering
Adaptive Temperature Scaling for Robust Calibration of Deep Neural Networks
The Unnecessity of Assuming Statistically Independent Tests in Bayesian Software Reliability Assessments
DNNShield: Dynamic Randomized Model Sparsification, A Defense Against Adversarial Machine Learning
DSLA: Dynamic smooth label assignment for efficient anchor-free object detection
Guidance on the Safety Assurance of Autonomous Systems in Complex Environments (SACE)
What do Deep Neural Networks Learn in Medical Images?
Keyword: scaling
DAS: Densely-Anchored Sampling for Deep Metric Learning
enpheeph: A Fault Injection Framework for Spiking and Compressed Deep Neural Networks
Adaptive Temperature Scaling for Robust Calibration of Deep Neural Networks
Is current research on adversarial robustness addressing the right problem?
Implicit bulk-surface filtering method for node-based shape optimization and comparison of explicit and implicit filtering techniques
Keyword: calibration
Geometric deep learning for computational mechanics Part II: Graph embedding for interpretable multiscale plasticity
Adaptive Temperature Scaling for Robust Calibration of Deep Neural Networks
A New Calibration Method for Industrial Robot Based on Step-Size Levenberg-Marquardt Algorithm
Learning to Navigate using Visual Sensor Networks