Abstract
In this paper, we focus on differentially private point and interval estimators for simple linear regression. Motivated by recent work that highlights the strong empirical performance of a robust algorithm called $\texttt{DPTheilSen}$, we provide a theoretical analysis of its privacy and accuracy guarantees, offer guidance on setting hyperparameters, and show how to produce differentially private confidence intervals for the slope.
GPS-GLASS: Learning Nighttime Semantic Segmentation Using Daytime Video and GPS data
Abstract
Semantic segmentation for autonomous driving should be robust against various in-the-wild environments. Nighttime semantic segmentation is especially challenging due to a lack of annotated nighttime images and a large domain gap from daytime images with sufficient annotation. In this paper, we propose a novel GPS-based training framework for nighttime semantic segmentation. Given GPS-aligned pairs of daytime and nighttime images, we perform cross-domain correspondence matching to obtain pixel-level pseudo supervision. Moreover, we conduct flow estimation between daytime video frames and apply GPS-based scaling to acquire another pixel-level pseudo supervision. Using these pseudo supervisions with a confidence map, we train a nighttime semantic segmentation network without any annotation from nighttime images. Experimental results demonstrate the effectiveness of the proposed method on several nighttime semantic segmentation datasets. Our source code is available at https://github.com/jimmy9704/GPS-GLASS.
Visualizing Confidence Intervals for Critical Point Probabilities in 2D Scalar Field Ensembles
Authors: Dominik Vietinghoff, Michael Böttinger, Gerik Scheuermann, Christian Heine
Abstract
An important task in visualization is the extraction and highlighting of dominant features in data to support users in their analysis process. Topological methods are a well-known means of identifying such features in deterministic fields. However, many real-world phenomena studied today are the result of a chaotic system that cannot be fully described by a single simulation. Instead, the variability of such systems is usually captured with ensemble simulations that produce a variety of possible outcomes of the simulated process. The topological analysis of such ensemble data sets and uncertain data, in general, is less well studied. In this work, we present an approach for the computation and visual representation of confidence intervals for the occurrence probabilities of critical points in ensemble data sets. We demonstrate the added value of our approach over existing methods for critical point prediction in uncertain data on a synthetic data set and show its applicability to a data set from climate research.
Keyword: scaling
Dalorex: A Data-Local Program Execution and Architecture for Memory-bound Applications
Authors: Marcelo Orenes Vera, Esin Tureci, David Wentzlaff, Margaret Martonosi
Subjects: Hardware Architecture (cs.AR); Distributed, Parallel, and Cluster Computing (cs.DC)
Abstract
Applications with low data reuse and frequent irregular memory accesses, such as graph or sparse linear algebra workloads, fail to scale well due to memory bottlenecks and poor core utilization. While prior work that utilizes prefetching, decoupling, or pipelining can mitigate memory latency and improve core utilization, memory bandwidth bottlenecks persist due to limited off-chip bandwidth. Approaches using in-memory processing (PIM) with Hybrid Memory Cube (HMC) surpass DRAM bandwidth limitations but fail to achieve high core utilization due to poor task scheduling and synchronization overheads. Moreover, the granularity of the memory available to each processing core with HMC limits the level of parallelism. This work proposes Dalorex, a hardware-software co-design that achieves high parallelism and energy efficiency, demonstrating strong scaling with 16,000 cores when processing graph and sparse linear algebra workloads. Over the prior work in PIM, using both 256 cores, Dalorex improves performance and energy consumption by two orders of magnitude through (1) a tile-based distributed-memory architecture where each processing tile holds an equal amount of data, and all memory operations are local; (2) a task-based parallel programming model where tasks are executed by the processing unit that is co-located with the target data; (3) a network design optimized for irregular traffic, where all communication is one-way, and messages do not contain routing metadata; and (4) a novel traffic-aware task scheduling hardware that maintains high core utilization; and (5) a data placement strategy improving work balance. This work proposes architectural and software innovations to provide, to our knowledge, the fastest design for running graph algorithms, while still being programmable for other domains.
GPS-GLASS: Learning Nighttime Semantic Segmentation Using Daytime Video and GPS data
Abstract
Semantic segmentation for autonomous driving should be robust against various in-the-wild environments. Nighttime semantic segmentation is especially challenging due to a lack of annotated nighttime images and a large domain gap from daytime images with sufficient annotation. In this paper, we propose a novel GPS-based training framework for nighttime semantic segmentation. Given GPS-aligned pairs of daytime and nighttime images, we perform cross-domain correspondence matching to obtain pixel-level pseudo supervision. Moreover, we conduct flow estimation between daytime video frames and apply GPS-based scaling to acquire another pixel-level pseudo supervision. Using these pseudo supervisions with a confidence map, we train a nighttime semantic segmentation network without any annotation from nighttime images. Experimental results demonstrate the effectiveness of the proposed method on several nighttime semantic segmentation datasets. Our source code is available at https://github.com/jimmy9704/GPS-GLASS.
Bridging Traditional and Machine Learning-based Algorithms for Solving PDEs: The Random Feature Method
Authors: Jingrun Chen, Xurong Chi, Weinan E, Zhouwang Yang
Abstract
One of the oldest and most studied subject in scientific computing is algorithms for solving partial differential equations (PDEs). A long list of numerical methods have been proposed and successfully used for various applications. In recent years, deep learning methods have shown their superiority for high-dimensional PDEs where traditional methods fail. However, for low dimensional problems, it remains unclear whether these methods have a real advantage over traditional algorithms as a direct solver. In this work, we propose the random feature method (RFM) for solving PDEs, a natural bridge between traditional and machine learning-based algorithms. RFM is based on a combination of well-known ideas: 1. representation of the approximate solution using random feature functions; 2. collocation method to take care of the PDE; 3. the penalty method to treat the boundary conditions, which allows us to treat the boundary condition and the PDE in the same footing. We find it crucial to add several additional components including multi-scale representation and rescaling the weights in the loss function. We demonstrate that the method exhibits spectral accuracy and can compete with traditional solvers in terms of both accuracy and efficiency. In addition, we find that RFM is particularly suited for complex problems with complex geometry, where both traditional and machine learning-based algorithms encounter difficulties.
Rethinking Efficacy of Softmax for Lightweight Non-Local Neural Networks
Authors: Yooshin Cho, Youngsoo Kim, Hanbyel Cho, Jaesung Ahn, Hyeong Gwon Hong, Junmo Kim
Abstract
Non-local (NL) block is a popular module that demonstrates the capability to model global contexts. However, NL block generally has heavy computation and memory costs, so it is impractical to apply the block to high-resolution feature maps. In this paper, to investigate the efficacy of NL block, we empirically analyze if the magnitude and direction of input feature vectors properly affect the attention between vectors. The results show the inefficacy of softmax operation which is generally used to normalize the attention map of the NL block. Attention maps normalized with softmax operation highly rely upon magnitude of key vectors, and performance is degenerated if the magnitude information is removed. By replacing softmax operation with the scaling factor, we demonstrate improved performance on CIFAR-10, CIFAR-100, and Tiny-ImageNet. In Addition, our method shows robustness to embedding channel reduction and embedding weight initialization. Notably, our method makes multi-head attention employable without additional computational cost.
Keyword: calibration
RobotIO: A Python Library for Robot Manipulation Experiments
Authors: Lukas Hermann, Max Argus, Adrian Roefer, Abhinav Valada, Thomas Brox, Wolfram Burgard
Abstract
Setting up robot environments to quickly test newly developed algorithms is still a difficult and time consuming process. This presents a significant hurdle to researchers interested in performing real-world robotic experiments. RobotIO is a python library designed to solve this problem. It focuses on providing common, simple, and well structured python interfaces for robots, grippers, and cameras, etc. These are provided with implementations of these interfaces for common hardware. This enables code using RobotIO to be portable across different robot setups. In terms of architecture, RobotIO is designed to be compatible with OpenAI gym environments, as well as ROS; examples of both of these are provided. The library comes together with a number of helpful tools, such as camera calibration scripts and episode recording functionality that further support algorithm development.
Keyword: out of distribution detection
There is no result
Keyword: out-of-distribution detection
There is no result
Keyword: expected calibration error
There is no result
Keyword: overconfident
There is no result
Keyword: overconfidence
There is no result
Keyword: confidence
Analyzing the Differentially Private Theil-Sen Estimator for Simple Linear Regression
GPS-GLASS: Learning Nighttime Semantic Segmentation Using Daytime Video and GPS data
Visualizing Confidence Intervals for Critical Point Probabilities in 2D Scalar Field Ensembles
Keyword: scaling
Dalorex: A Data-Local Program Execution and Architecture for Memory-bound Applications
GPS-GLASS: Learning Nighttime Semantic Segmentation Using Daytime Video and GPS data
Bridging Traditional and Machine Learning-based Algorithms for Solving PDEs: The Random Feature Method
Rethinking Efficacy of Softmax for Lightweight Non-Local Neural Networks
Keyword: calibration
RobotIO: A Python Library for Robot Manipulation Experiments