New submissions for Fri, 29 Jul 22

[ericbeyer]

Keyword: out of distribution detection

There is no result

Keyword: out-of-distribution detection

There is no result

Keyword: expected calibration error

There is no result

Keyword: overconfident

There is no result

Keyword: overconfidence

There is no result

Keyword: confidence

A Novel Data Augmentation Technique for Out-of-Distribution Sample Detection using Compounded Corruptions

• Authors: Ramya S. Hebbalaguppe, Soumya Suvra Goshal, Jatin Prakash, Harshad Khadilkar, Chetan Arora
• Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
• Arxiv link: https://arxiv.org/abs/2207.13916
• Pdf link: https://arxiv.org/pdf/2207.13916
• Abstract Modern deep neural network models are known to erroneously classify out-of-distribution (OOD) test data into one of the in-distribution (ID) training classes with high confidence. This can have disastrous consequences for safety-critical applications. A popular mitigation strategy is to train a separate classifier that can detect such OOD samples at the test time. In most practical settings OOD examples are not known at the train time, and hence a key question is: how to augment the ID data with synthetic OOD samples for training such an OOD detector? In this paper, we propose a novel Compounded Corruption technique for the OOD data augmentation termed CnC. One of the major advantages of CnC is that it does not require any hold-out data apart from the training set. Further, unlike current state-of-the-art (SOTA) techniques, CnC does not require backpropagation or ensembling at the test time, making our method much faster at inference. Our extensive comparison with 20 methods from the major conferences in last 4 years show that a model trained using CnC based data augmentation, significantly outperforms SOTA, both in terms of OOD detection accuracy as well as inference time. We include a detailed post-hoc analysis to investigate the reasons for the success of our method and identify higher relative entropy and diversity of CnC samples as probable causes. We also provide theoretical insights via a piece-wise decomposition analysis on a two-dimensional dataset to reveal (visually and quantitatively) that our approach leads to a tighter boundary around ID classes, leading to better detection of OOD samples. Source code link: https://github.com/cnc-ood

  Keyword: scaling

  HDSDP: Software for Semidefinite Programming

• Authors: Wenzhi Gao, Dongdong Ge, Yinyu Ye
• Subjects: Mathematical Software (cs.MS); Optimization and Control (math.OC)
• Arxiv link: https://arxiv.org/abs/2207.13862
• Pdf link: https://arxiv.org/pdf/2207.13862
• Abstract HDSDP is a numerical software solving the semidefinite programming problems. The main framework of HDSDP resembles the dual-scaling interior point solver DSDP[2] and several new features, especially a dual method based on the simplified homogeneous self-dual embedding, have been implemented. The embedding enhances stability of dual method and several new heuristics and computational techniques are designed to accelerate its convergence. HDSDP aims to show how dual-scaling algorithms benefit from the self-dual embedding and it is developed in parallel to DSDP5.8. Numerical experiments over several classical benchmark datasets exhibit its robustness and efficiency, and particularly its advantages on SDP instances featuring low-rank structure and sparsity. The pre-built binary of HDSDP is currently freely available at https://github.com/COPT-Public/HDSDP.

  ReFRS: Resource-efficient Federated Recommender System for Dynamic and Diversified User Preferences

• Authors: Mubashir Imran, Hongzhi Yin, Tong Chen, Nguyen Quoc Viet Hung, Alexander Zhou, Kai Zheng
• Subjects: Information Retrieval (cs.IR); Distributed, Parallel, and Cluster Computing (cs.DC)
• Arxiv link: https://arxiv.org/abs/2207.13897
• Pdf link: https://arxiv.org/pdf/2207.13897
• Abstract Owing to its nature of scalability and privacy by design, federated learning (FL) has received increasing interest in decentralized deep learning. FL has also facilitated recent research on upscaling and privatizing personalized recommendation services, using on-device data to learn recommender models locally. These models are then aggregated globally to obtain a more performant model, while maintaining data privacy. Typically, federated recommender systems (FRSs) do not consider the lack of resources and data availability at the end-devices. In addition, they assume that the interaction data between users and items is i.i.d. and stationary across end-devices, and that all local recommender models can be directly averaged without considering the user's behavioral diversity. However, in real scenarios, recommendations have to be made on end-devices with sparse interaction data and limited resources. Furthermore, users' preferences are heterogeneous and they frequently visit new items. This makes their personal preferences highly skewed, and the straightforwardly aggregated model is thus ill-posed for such non-i.i.d. data. In this paper, we propose Resource Efficient Federated Recommender System (ReFRS) to enable decentralized recommendation with dynamic and diversified user preferences. On the device side, ReFRS consists of a lightweight self-supervised local model built upon the variational autoencoder for learning a user's temporal preference from a sequence of interacted items. On the server side, ReFRS utilizes a semantic sampler to adaptively perform model aggregation within each identified user cluster. The clustering module operates in an asynchronous and dynamic manner to support efficient global model update and cope with shifting user interests. As a result, ReFRS achieves superior performance in terms of both accuracy and scalability, as demonstrated by comparative experiments.

  Local Embedded Discrete Fracture Model (LEDFM)

• Authors: Davide Losapio (1), Anna Scotti (1) ((1) Politecnico di Milano)
• Subjects: Numerical Analysis (math.NA)
• Arxiv link: https://arxiv.org/abs/2207.14009
• Pdf link: https://arxiv.org/pdf/2207.14009
• Abstract The study of flow in fractured porous media is a key ingredient for many geoscience applications, such as reservoir management and geothermal energy production. Modelling and simulation of these highly heterogeneous and geometrically complex systems require the adoption of non-standard numerical schemes. The Embedded Discrete Fracture Model (EDFM) is a simple and effective way to account for fractures with coarse and regular grids, but it suffers from some limitations: it assumes a linear pressure distribution around fractures, which holds true only far from the tips and fracture intersections, and it can be employed for highly permeable fractures only. In this paper we propose an improvement of EDFM which aims at overcoming these limitations computing an improved coupling between fractures and the surrounding porous medium by a) relaxing the linear pressure distribution assumption, b) accounting for impermeable fractures modifying near-fracture transmissibilities. These results are achieved by solving different types of local problems with a fine conforming grid, and computing new transmissibilities (for connections between fractures and the surrounding porous medium and those through the porous medium itself near to the fractures). Such local problems are inspired from numerical upscaling techniques present in the literature. The new method is called Local Embedded Discrete Fracture Model (LEDFM) and the results obtained from several numerical tests confirm the aforementioned improvements.

  Short Synchronizing Words for Random Automata

• Authors: Guillaume Chapuy, Guillem Perarnau
• Subjects: Formal Languages and Automata Theory (cs.FL); Discrete Mathematics (cs.DM); Combinatorics (math.CO); Probability (math.PR)
• Arxiv link: https://arxiv.org/abs/2207.14108
• Pdf link: https://arxiv.org/pdf/2207.14108
• Abstract We prove that a uniformly random automaton with $n$ states on a 2-letter alphabet has a synchronizing word of length $O(n^{1/2}\log n)$ with high probability (w.h.p.). That is to say, w.h.p. there exists a word $\omega$ of such length, and a state $v_0$, such that $\omega$ sends all states to $v_0$. Prior to this work, the best upper bound was the quasilinear bound $O(n\log^3n)$ due to Nicaud (2016). The correct scaling exponent had been subject to various estimates by other authors between $0.5$ and $0.56$ based on numerical simulations, and our result confirms that the smallest one indeed gives a valid upper bound (with a log factor). Our proof introduces the concept of $w$-trees, for a word $w$, that is, automata in which the $w$-transitions induce a (loop-rooted) tree. We prove a strong structure result that says that, w.h.p., a random automaton on $n$ states is a $w$-tree for some word $w$ of length at most $(1+\epsilon)\log_2(n)$, for any $\epsilon>0$. The existence of the (random) word $w$ is proved by the probabilistic method. This structure result is key to proving that a short synchronizing word exists.

  Keyword: calibration

  Calibrate: Interactive Analysis of Probabilistic Model Output

• Authors: Peter Xenopoulos, Joao Rulff, Luis Gustavo Nonato, Brian Barr, Claudio Silva
• Subjects: Human-Computer Interaction (cs.HC); Machine Learning (cs.LG)
• Arxiv link: https://arxiv.org/abs/2207.13770
• Pdf link: https://arxiv.org/pdf/2207.13770
• Abstract Analyzing classification model performance is a crucial task for machine learning practitioners. While practitioners often use count-based metrics derived from confusion matrices, like accuracy, many applications, such as weather prediction, sports betting, or patient risk prediction, rely on a classifier's predicted probabilities rather than predicted labels. In these instances, practitioners are concerned with producing a calibrated model, that is, one which outputs probabilities that reflect those of the true distribution. Model calibration is often analyzed visually, through static reliability diagrams, however, the traditional calibration visualization may suffer from a variety of drawbacks due to the strong aggregations it necessitates. Furthermore, count-based approaches are unable to sufficiently analyze model calibration. We present Calibrate, an interactive reliability diagram that addresses the aforementioned issues. Calibrate constructs a reliability diagram that is resistant to drawbacks in traditional approaches, and allows for interactive subgroup analysis and instance-level inspection. We demonstrate the utility of Calibrate through use cases on both real-world and synthetic data. We further validate Calibrate by presenting the results of a think-aloud experiment with data scientists who routinely analyze model calibration.

  Gender In Gender Out: A Closer Look at User Attributes in Context-Aware Recommendation

• Authors: Manel Slokom, Özlem Özgöbek, Martha Larson
• Subjects: Machine Learning (cs.LG)
• Arxiv link: https://arxiv.org/abs/2207.14218
• Pdf link: https://arxiv.org/pdf/2207.14218
• Abstract This paper studies user attributes in light of current concerns in the recommender system community: diversity, coverage, calibration, and data minimization. In experiments with a conventional context-aware recommender system that leverages side information, we show that user attributes do not always improve recommendation. Then, we demonstrate that user attributes can negatively impact diversity and coverage. Finally, we investigate the amount of information about users that ``survives'' from the training data into the recommendation lists produced by the recommender. This information is a weak signal that could in the future be exploited for calibration or studied further as a privacy leak.