Abstract
Advances in computer vision and machine learning enable robots to perceive their surroundings in powerful new ways, but these perception modules have well-known fragilities. We consider the problem of synthesizing a safe controller that is robust despite perception errors. The proposed method constructs a state estimator based on Gaussian processes with input-dependent noises. This estimator computes a high-confidence set for the actual state given a perceived state. Then, a robust neural network controller is synthesized that can provably handle the state uncertainty. Furthermore, an adaptive sampling algorithm is proposed to jointly improve the estimator and controller. Simulation experiments, including a realistic vision-based lane-keeping example in CARLA, illustrate the promise of the proposed approach in synthesizing robust controllers with deep-learning-based perception.
Analysis of Fault Tolerant Multi-stage Switch Architecture for TSN
Authors: Adnan Ghaderi, Rahul Nandkumar Gore
Subjects: Networking and Internet Architecture (cs.NI)
Abstract
We conducted the feasibility analysis of utilizing a highly available multi-stage architecture for TSN switches used for sending high priority, mission-critical traffic within a bounded latency instead of traditional single-stage architectures. To verify the TSN functionality, we implemented the 'strict priority' feature. We evaluated the performance of both architectures on multiple parameters such as fault tolerance, packet latency, throughput, reliability, path length effectiveness, and cost per unit. The fault tolerance analysis demonstrated that the multi-stage architecture fairs better than the single-stage counterpart. The average latency and throughput performance of multi-stage architectures, although low, can be considered comparable with single-stage counterparts. However, the multi-stage architecture fails to meet the performance of single-stage architectures on parameters such as reliability, path length effectiveness, and cost-effectiveness. The improved fault tolerance comes at the cost of increased hardware resources, cost, and complexity. However, with the advent of cost-effective technologies in hardware design and efficient architecture designs, the multi-stage switching architecture-based TSN switches can be made reasonably comparable to single-stage switching TSN switches. This work gives initial confidence that the multi-stage architecture can be pursued further for safety-critical systems that require determinism and reliability in the communication of critical messages.
Neural Clamping: Joint Input Perturbation and Temperature Scaling for Neural Network Calibration
Abstract
Neural network calibration is an essential task in deep learning to ensure consistency between the confidence of model prediction and the true correctness likelihood. In this paper, we propose a new post-processing calibration method called Neural Clamping, which employs a simple joint input-output transformation on a pre-trained classifier via a learnable universal input perturbation and an output temperature scaling parameter. Moreover, we provide theoretical explanations on why Neural Clamping is provably better than temperature scaling. Evaluated on CIFAR-100 and ImageNet image recognition datasets and a variety of deep neural network models, our empirical results show that Neural Clamping significantly outperforms state-of-the-art post-processing calibration methods.
Keyword: scaling
Achievable Diversity Order of HARQ-Aided Downlink NOMA Systems
Authors: Zheng Shi, Chenmeng Zhang, Yaru Fu, Hong Wang, Guanghua Yang, Shaodan Ma
Abstract
The combination between non-orthogonal multiple access (NOMA) and hybrid automatic repeat request (HARQ) is capable of realizing ultra-reliability, high throughput and many concurrent connections particularly for emerging communication systems. This paper focuses on characterizing the asymptotic scaling law of the outage probability of HARQ-aided NOMA systems with respect to the transmit power, i.e., diversity order. The analysis of diversity order is carried out for three basic types of HARQ-aided downlink NOMA systems, including Type I HARQ, HARQ with chase combining (HARQ-CC) and HARQ with incremental redundancy (HARQ-IR). The diversity orders of three HARQ-aided downlink NOMA systems are derived in closed-form, where an integration domain partition trick is developed to obtain the bounds of the outage probability specially for HARQ-CC and HARQ-IR-aided NOMA systems. The analytical results show that the diversity order is a decreasing step function of transmission rate, and full time diversity can only be achieved under a sufficiently low transmission rate. It is also revealed that HARQ-IR-aided NOMA systems have the largest diversity order, followed by HARQ-CC-aided and then Type I HARQ-aided NOMA systems. Additionally, the users' diversity orders follow a descending order according to their respective average channel gains. Furthermore, we expand discussions on the cases of power-efficient transmissions and imperfect channel state information (CSI). Monte Carlo simulations finally confirm our analysis.
Relation Embedding based Graph Neural Networks for Handling Heterogeneous Graph
Authors: Junfu Wang, Yuanfang Guo, Liang Yang, Yunhong Wang
Abstract
Heterogeneous graph learning has drawn significant attentions in recent years, due to the success of graph neural networks (GNNs) and the broad applications of heterogeneous information networks. Various heterogeneous graph neural networks have been proposed to generalize GNNs for processing the heterogeneous graphs. Unfortunately, these approaches model the heterogeneity via various complicated modules. This paper aims to propose a simple yet efficient framework to make the homogeneous GNNs have adequate ability to handle heterogeneous graphs. Specifically, we propose Relation Embedding based Graph Neural Networks (RE-GNNs), which employ only one parameter per relation to embed the importance of edge type relations and self-loop connections. To optimize these relation embeddings and the other parameters simultaneously, a gradient scaling factor is proposed to constrain the embeddings to converge to suitable values. Besides, we theoretically demonstrate that our RE-GNNs have more expressive power than the meta-path based heterogeneous GNNs. Extensive experiments on the node classification tasks validate the effectiveness of our proposed method.
Neural Clamping: Joint Input Perturbation and Temperature Scaling for Neural Network Calibration
Abstract
Neural network calibration is an essential task in deep learning to ensure consistency between the confidence of model prediction and the true correctness likelihood. In this paper, we propose a new post-processing calibration method called Neural Clamping, which employs a simple joint input-output transformation on a pre-trained classifier via a learnable universal input perturbation and an output temperature scaling parameter. Moreover, we provide theoretical explanations on why Neural Clamping is provably better than temperature scaling. Evaluated on CIFAR-100 and ImageNet image recognition datasets and a variety of deep neural network models, our empirical results show that Neural Clamping significantly outperforms state-of-the-art post-processing calibration methods.
Keyword: calibration
The Microsoft System for VoxCeleb Speaker Recognition Challenge 2022
Authors: Gang Liu, Tianyan Zhou, Yong Zhao, Yu Wu, Zhuo Chen, Yao Qian, Jian Wu
Abstract
In this report, we describe our submitted system for track 2 of the VoxCeleb Speaker Recognition Challenge 2022 (VoxSRC-22). We fuse a variety of good-performing models ranging from supervised models to self-supervised learning(SSL) pre-trained models. The supervised models, trained using VoxCeleb-2 dev data, consist of ECAPA-TDNN and Res2Net in a very deep structure. The SSL pre-trained models, wav2vec and wavLM, are trained using large scale unlabeled speech data up to million hours. These models are cascaded with ECAPA-TDNN and further fine-tuned in a supervised fashion to extract the speaker representations. All 13 models are applied with score normalization and calibration and then fused into the the submitted system. We also explore the audio quality measures in the calibration stage such as duration, SNR, T60, and MOS. The best submitted system achieves 0.073 in minDCF and 1.436% in EER on the VoxSRC-22 evaluation set.
WS-3D-Lane: Weakly Supervised 3D Lane Detection With 2D Lane Labels
Abstract
Compared to 2D lanes, real 3D lane data is difficult to collect accurately. In this paper, we propose a novel method for training 3D lanes with only 2D lane labels, called weakly supervised 3D lane detection WS-3D-Lane. By assumptions of constant lane width and equal height on adjacent lanes, we indirectly supervise 3D lane heights in the training. To overcome the problem of the dynamic change of the camera pitch during data collection, a camera pitch self-calibration method is proposed. In anchor representation, we propose a double-layer anchor with a improved non-maximum suppression (NMS) method, which enables the anchor-based method to predict two lane lines that are close. Experiments are conducted on the base of 3D-LaneNet under two supervision methods. Under weakly supervised setting, our WS-3D-Lane outperforms previous 3D-LaneNet: F-score rises to 92.3% on Apollo 3D synthetic dataset, and F1 rises to 74.5% on ONCE-3DLanes. Meanwhile, WS-3D-Lane in purely supervised setting makes more increments and outperforms state-of-the-art. To the best of our knowledge, WS-3D-Lane is the first try of 3D lane detection under weakly supervised setting.
Neural Clamping: Joint Input Perturbation and Temperature Scaling for Neural Network Calibration
Abstract
Neural network calibration is an essential task in deep learning to ensure consistency between the confidence of model prediction and the true correctness likelihood. In this paper, we propose a new post-processing calibration method called Neural Clamping, which employs a simple joint input-output transformation on a pre-trained classifier via a learnable universal input perturbation and an output temperature scaling parameter. Moreover, we provide theoretical explanations on why Neural Clamping is provably better than temperature scaling. Evaluated on CIFAR-100 and ImageNet image recognition datasets and a variety of deep neural network models, our empirical results show that Neural Clamping significantly outperforms state-of-the-art post-processing calibration methods.
Keyword: out of distribution detection
There is no result
Keyword: out-of-distribution detection
There is no result
Keyword: expected calibration error
There is no result
Keyword: overconfident
There is no result
Keyword: overconfidence
There is no result
Keyword: confidence
Learning Certifiably Robust Controllers Using Fragile Perception
Analysis of Fault Tolerant Multi-stage Switch Architecture for TSN
Neural Clamping: Joint Input Perturbation and Temperature Scaling for Neural Network Calibration
Keyword: scaling
Achievable Diversity Order of HARQ-Aided Downlink NOMA Systems
Relation Embedding based Graph Neural Networks for Handling Heterogeneous Graph
Neural Clamping: Joint Input Perturbation and Temperature Scaling for Neural Network Calibration
Keyword: calibration
The Microsoft System for VoxCeleb Speaker Recognition Challenge 2022
WS-3D-Lane: Weakly Supervised 3D Lane Detection With 2D Lane Labels
Neural Clamping: Joint Input Perturbation and Temperature Scaling for Neural Network Calibration