Abstract
Due to the necessary security for the airport and flight, passengers are required to have strict security check before getting aboard. However, there are frequent complaints of wasting huge amount of time while waiting for the security check. This paper presents a potential solution aimed at optimizing gate setup procedures specifically tailored for Chicago OHare International Airport. By referring to queueing theory and performing Monte Carlo simulations, we propose an approach to significantly diminish the average waiting time to a more manageable level. Additionally, our study meticulously examines and identifies the influential factors contributing to this optimization, providing a comprehensive understanding of their impact.
0.1% Data Makes Segment Anything Slim
Authors: Authors: Zigeng Chen, Gongfan Fang, Xinyin Ma, Xinchao Wang
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Abstract
The formidable model size and demanding computational requirements of Segment Anything Model (SAM) have rendered it cumbersome for deployment on resource-constrained devices. Existing approaches for SAM compression typically involve training a new network from scratch, posing a challenging trade-off between compression costs and model performance. To address this issue, this paper introduces SlimSAM, a novel SAM compression method that achieves superior performance with remarkably low training costs. This is achieved by the efficient reuse of pre-trained SAMs through a unified pruning-distillation framework. To enhance knowledge inheritance from the original SAM, we employ an innovative alternate slimming strategy that partitions the compression process into a progressive procedure. Diverging from prior pruning techniques, we meticulously prune and distill decoupled model structures in an alternating fashion. Furthermore, a novel label-free pruning criterion is also proposed to align the pruning objective with the optimization target, thereby boosting the post-distillation after pruning. SlimSAM yields significant performance improvements while demanding over 10 times less training costs than any other existing methods. Even when compared to the original SAM-H, SlimSAM achieves approaching performance while reducing parameter counts to merely 0.9% (5.7M), MACs to 0.8% (21G), and requiring only 0.1% (10k) of the SAM training data.
A Data-Driven Framework for Improving Public EV Charging Infrastructure: Modeling and Forecasting
Authors: Authors: Nassr Al-Dahabreh, Mohammad Ali Sayed, Khaled Sarieddine, Mohamed Elhattab, Maurice Khabbaz, Ribal Atallah, Chadi Assi
Abstract
This work presents an investigation and assessment framework, which, supported by realistic data, aims at provisioning operators with in-depth insights into the consumer-perceived Quality-of-Experience (QoE) at public Electric Vehicle (EV) charging infrastructures. Motivated by the unprecedented EV market growth, it is suspected that the existing charging infrastructure will soon be no longer capable of sustaining the rapidly growing charging demands; let alone that the currently adopted ad hoc infrastructure expansion strategies seem to be far from contributing any quality service sustainability solutions that tangibly reduce (ultimately mitigate) the severity of this problem. Without suitable QoE metrics, operators, today, face remarkable difficulty in assessing the performance of EV Charging Stations (EVCSs) in this regard. This paper aims at filling this gap through the formulation of novel and original critical QoE performance metrics that provide operators with visibility into the per-EVCS operational dynamics and allow for the optimization of these stations' respective utilization. Such metrics shall then be used as inputs to a Machine Learning model finely tailored and trained using recent real-world data sets for the purpose of forecasting future long-term EVCS loads. This will, in turn, allow for making informed optimal EV charging infrastructure expansions that will be capable of reliably coping with the rising EV charging demands and maintaining acceptable QoE levels. The model's accuracy has been tested and extensive simulations are conducted to evaluate the achieved performance in terms of the above listed metrics and show the suitability of the recommended infrastructure expansions.
Guaranteed Trust Region Optimization via Two-Phase KL Penalization
Abstract
On-policy reinforcement learning (RL) has become a popular framework for solving sequential decision problems due to its computational efficiency and theoretical simplicity. Some on-policy methods guarantee every policy update is constrained to a trust region relative to the prior policy to ensure training stability. These methods often require computationally intensive non-linear optimization or require a particular form of action distribution. In this work, we show that applying KL penalization alone is nearly sufficient to enforce such trust regions. Then, we show that introducing a "fixup" phase is sufficient to guarantee a trust region is enforced on every policy update while adding fewer than 5% additional gradient steps in practice. The resulting algorithm, which we call FixPO, is able to train a variety of policy architectures and action spaces, is easy to implement, and produces results competitive with other trust region methods.
Real-time Scheduling of Building Battery Systems Under Electrical Load Uncertainty Using Dynamic Markov Decision Process
Authors: Authors: Hussein Sharadga, Golbon Zakeri, Abdullah Hayajneh, Geoff Pritchard
Abstract
In response to the increasing deployment of battery storage systems for cost reduction and grid stress mitigation, this study presents the development of a new real-time Markov decision process model to efficiently schedule battery systems in buildings under electrical load uncertainty. The proposed model incorporates quantile Fourier regression for load fitting, leading to a large-scale optimization problem with approximately a million decision variables and constraints. To address this complexity, the problem is formulated as a linear program and solved using a commercial solver, ensuring effective navigation and identification of optimal solutions. The model's performance is evaluated by considering different pricing policies and scenarios, including demand peak shaving. Validation of the Markov model is conducted using one year of historical demand data from a school. Findings indicate that MDP performance in adapting to uncertain loads can range from 30 to 99% depending on the pricing policy.
On the calibration of compartmental epidemiological models
Authors: Authors: Nikunj Gupta, Anh Mai, Azza Abouzied, Dennis Shasha
Subjects: Machine Learning (cs.LG); Physics and Society (physics.soc-ph); Populations and Evolution (q-bio.PE)
Abstract
Epidemiological compartmental models are useful for understanding infectious disease propagation and directing public health policy decisions. Calibration of these models is an important step in offering accurate forecasts of disease dynamics and the effectiveness of interventions. In this study, we present an overview of calibrating strategies that can be employed, including several optimization methods and reinforcement learning (RL). We discuss the benefits and drawbacks of these methods and highlight relevant practical conclusions from our experiments. Optimization methods iteratively adjust the parameters of the model until the model output matches the available data, whereas RL uses trial and error to learn the optimal set of parameters by maximizing a reward signal. Finally, we discuss how the calibration of parameters of epidemiological compartmental models is an emerging field that has the potential to improve the accuracy of disease modeling and public health decision-making. Further research is needed to validate the effectiveness and scalability of these approaches in different epidemiological contexts. All codes and resources are available on \url{https://github.com/Nikunj-Gupta/On-the-Calibration-of-Compartmental-Epidemiological-Models}. We hope this work can facilitate related research.
Poisoning $\times$ Evasion: Symbiotic Adversarial Robustness for Graph Neural Networks
Authors: Authors: Ege Erdogan, Simon Geisler, Stephan Günnemann
Abstract
It is well-known that deep learning models are vulnerable to small input perturbations. Such perturbed instances are called adversarial examples. Adversarial examples are commonly crafted to fool a model either at training time (poisoning) or test time (evasion). In this work, we study the symbiosis of poisoning and evasion. We show that combining both threat models can substantially improve the devastating efficacy of adversarial attacks. Specifically, we study the robustness of Graph Neural Networks (GNNs) under structure perturbations and devise a memory-efficient adaptive end-to-end attack for the novel threat model using first-order optimization.
Flexible Base Station Sleeping and Resource Cooperation Enabled Green Fully-Decoupled RAN
Abstract
Base station (BS) sleeping, a promising technique to address the growing energy consumption in wireless communication networks, encounters challenges such as coverage holes and coupled uplink and downlink transmissions. As an innovative architecture designed for future-generation mobile communication networks, the fully-decoupled radio access network (FD-RAN) is anticipated to overcome these challenges by fully decoupled control-data planes and uplink-downlink transmissions. In this paper, we investigate energy-efficient uplink FD-RAN leveraging flexible BS sleeping and resource cooperation. First, we introduce a holistic energy consumption model and formulate a bi-level energy efficiency maximizing problem for FD-RAN, involved with the joint optimization of user equipment (UE) association, BS sleeping, and power control. Subsequently, through employing the Tammer decomposition method, the formulated bi-level problem is converted into two equivalent upper-level and lower-level problems. The lower-level problem encompassed with UE power control is addressed by introducing a successive lower-bound maximization-based Dinkelbach's algorithm, and the upper-level problem for UE association and BS sleeping is solved through a modified low-complexity many-to-many swap matching algorithm, respectively. Extensive simulation results not only demonstrate the superior effectiveness of FD-RAN and our proposed algorithms but also reveal the sources of energy efficiency gains within FD-RAN.
Federated Causality Learning with Explainable Adaptive Optimization
Authors: Authors: Dezhi Yang, Xintong He, Jun Wang, Guoxian Yu, Carlotta Domeniconi, Jinglin Zhang
Abstract
Discovering the causality from observational data is a crucial task in various scientific domains. With increasing awareness of privacy, data are not allowed to be exposed, and it is very hard to learn causal graphs from dispersed data, since these data may have different distributions. In this paper, we propose a federated causal discovery strategy (FedCausal) to learn the unified global causal graph from decentralized heterogeneous data. We design a global optimization formula to naturally aggregate the causal graphs from client data and constrain the acyclicity of the global graph without exposing local data. Unlike other federated causal learning algorithms, FedCausal unifies the local and global optimizations into a complete directed acyclic graph (DAG) learning process with a flexible optimization objective. We prove that this optimization objective has a high interpretability and can adaptively handle homogeneous and heterogeneous data. Experimental results on synthetic and real datasets show that FedCausal can effectively deal with non-independently and identically distributed (non-iid) data and has a superior performance.
DPoser: Diffusion Model as Robust 3D Human Pose Prior
Abstract
Modeling human pose is a cornerstone in applications from human-robot interaction to augmented reality, yet crafting a robust human pose prior remains a challenge due to biomechanical constraints and diverse human movements. Traditional priors like VAEs and NDFs often fall short in realism and generalization, especially in extreme conditions such as unseen noisy poses. To address these issues, we introduce DPoser, a robust and versatile human pose prior built upon diffusion models. Designed with optimization frameworks, DPoser seamlessly integrates into various pose-centric applications, including human mesh recovery, pose completion, and motion denoising. Specifically, by formulating these tasks as inverse problems, we employ variational diffusion sampling for efficient solving. Furthermore, acknowledging the disparity between the articulated poses we focus on and structured images in previous research, we propose a truncated timestep scheduling to boost performance on downstream tasks. Our exhaustive experiments demonstrate DPoser's superiority over existing state-of-the-art pose priors across multiple tasks.
Multi-dimensional Fair Federated Learning
Authors: Authors: Cong Su, Guoxian Yu, Jun Wang, Hui Li, Qingzhong Li, Han Yu
Abstract
Federated learning (FL) has emerged as a promising collaborative and secure paradigm for training a model from decentralized data without compromising privacy. Group fairness and client fairness are two dimensions of fairness that are important for FL. Standard FL can result in disproportionate disadvantages for certain clients, and it still faces the challenge of treating different groups equitably in a population. The problem of privately training fair FL models without compromising the generalization capability of disadvantaged clients remains open. In this paper, we propose a method, called mFairFL, to address this problem and achieve group fairness and client fairness simultaneously. mFairFL leverages differential multipliers to construct an optimization objective for empirical risk minimization with fairness constraints. Before aggregating locally trained models, it first detects conflicts among their gradients, and then iteratively curates the direction and magnitude of gradients to mitigate these conflicts. Theoretical analysis proves mFairFL facilitates the fairness in model development. The experimental evaluations based on three benchmark datasets show significant advantages of mFairFL compared to seven state-of-the-art baselines.
Long-Term Rate-Fairness-Aware Beamforming Based Massive MIMO Systems
Authors: Authors: W. Zhu, H. D. Tuan, E. Dutkiewicz, Y. Fang, H. V. Poor, L. Hanzo
Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Abstract
This is the first treatise on multi-user (MU) beamforming designed for achieving long-term rate-fairness in fulldimensional MU massive multi-input multi-output (m-MIMO) systems. Explicitly, based on the channel covariances, which can be assumed to be known beforehand, we address this problem by optimizing the following objective functions: the users' signal-toleakage-noise ratios (SLNRs) using SLNR max-min optimization, geometric mean of SLNRs (GM-SLNR) based optimization, and SLNR soft max-min optimization. We develop a convex-solver based algorithm, which invokes a convex subproblem of cubic time-complexity at each iteration for solving the SLNR maxmin problem. We then develop closed-form expression based algorithms of scalable complexity for the solution of the GMSLNR and of the SLNR soft max-min problem. The simulations provided confirm the users' improved-fairness ergodic rate distributions.
Abstract
The theory of Bayesian learning incorporates the use of Student-t Processes to model heavy-tailed distributions and datasets with outliers. However, despite Student-t Processes having a similar computational complexity as Gaussian Processes, there has been limited emphasis on the sparse representation of this model. This is mainly due to the increased difficulty in modeling and computation compared to previous sparse Gaussian Processes. Our motivation is to address the need for a sparse representation framework that reduces computational complexity, allowing Student-t Processes to be more flexible for real-world datasets. To achieve this, we leverage the conditional distribution of Student-t Processes to introduce sparse inducing points. Bayesian methods and variational inference are then utilized to derive a well-defined lower bound, facilitating more efficient optimization of our model through stochastic gradient descent. We propose two methods for computing the variational lower bound, one utilizing Monte Carlo sampling and the other employing Jensen's inequality to compute the KL regularization term in the loss function. We propose adopting these approaches as viable alternatives to Gaussian processes when the data might contain outliers or exhibit heavy-tailed behavior, and we provide specific recommendations for their applicability. We evaluate the two proposed approaches on various synthetic and real-world datasets from UCI and Kaggle, demonstrating their effectiveness compared to baseline methods in terms of computational complexity and accuracy, as well as their robustness to outliers.
Dynamic Adjustment of Matching Radii under the Broadcasting Mode: A Novel Multitask Learning Strategy and Temporal Modeling Approach
Abstract
As ride-hailing services have experienced significant growth, the majority of research has concentrated on the dispatching mode, where drivers must adhere to the platform's assigned routes. However, the broadcasting mode, in which drivers can freely choose their preferred orders from those broadcast by the platform, has received less attention. One important but challenging task in such a system is the determination of the optimal matching radius, which usually varies across space, time, and real-time supply/demand characteristics. This study develops a Transformer-Encoder-Based (TEB) model that predicts key system performance metrics for a range of matching radii, which enables the ride-hailing platform to select an optimal matching radius that maximizes overall system performance according to real-time supply and demand information. To simultaneously maximize multiple system performance metrics for matching radius determination, we devise a novel multi-task learning algorithm that enhances convergence speed of each task (corresponding to the optimization of one metric) and delivers more accurate overall predictions. We evaluate our methods in a simulation environment specifically designed for broadcasting-mode-based ride-hailing service. Our findings reveal that dynamically adjusting matching radii based on our proposed predict-then-optimize approach significantly improves system performance, e.g., increasing platform revenue by 7.55% and enhancing order fulfillment rate by 13% compared to benchmark algorithms.
Not All Data Matters: An End-to-End Adaptive Dataset Pruning Framework for Enhancing Model Performance and Efficiency
Abstract
While deep neural networks have demonstrated remarkable performance across various tasks, they typically require massive training data. Due to the presence of redundancies and biases in real-world datasets, not all data in the training dataset contributes to the model performance. To address this issue, dataset pruning techniques have been introduced to enhance model performance and efficiency by eliminating redundant training samples and reducing computational and memory overhead. However, previous works most rely on manually crafted scalar scores, limiting their practical performance and scalability across diverse deep networks and datasets. In this paper, we propose AdaPruner, an end-to-end Adaptive DAtaset PRUNing framEwoRk. AdaPruner can perform effective dataset pruning without the need for explicitly defined metrics. Our framework jointly prunes training data and fine-tunes models with task-specific optimization objectives. AdaPruner leverages (1) An adaptive dataset pruning (ADP) module, which iteratively prunes redundant samples to an expected pruning ratio; and (2) A pruning performance controller (PPC) module, which optimizes the model performance for accurate pruning. Therefore, AdaPruner exhibits high scalability and compatibility across various datasets and deep networks, yielding improved dataset distribution and enhanced model performance. AdaPruner can still significantly enhance model performance even after pruning up to 10-30\% of the training data. Notably, these improvements are accompanied by substantial savings in memory and computation costs. Qualitative and quantitative experiments suggest that AdaPruner outperforms other state-of-the-art dataset pruning methods by a large margin.
Abstract
Neural integral equations are deep learning models based on the theory of integral equations, where the model consists of an integral operator and the corresponding equation (of the second kind) which is learned through an optimization procedure. This approach allows to leverage the nonlocal properties of integral operators in machine learning, but it is computationally expensive. In this article, we introduce a framework for neural integral equations based on spectral methods that allows us to learn an operator in the spectral domain, resulting in a cheaper computational cost, as well as in high interpolation accuracy. We study the properties of our methods and show various theoretical guarantees regarding the approximation capabilities of the model, and convergence to solutions of the numerical methods. We provide numerical experiments to demonstrate the practical effectiveness of the resulting model.
Leveraging Reinforcement Learning and Large Language Models for Code Optimization
Authors: Authors: Shukai Duan, Nikos Kanakaris, Xiongye Xiao, Heng Ping, Chenyu Zhou, Nesreen K. Ahmed, Guixiang Ma, Mihai Capota, Theodore L. Willke, Shahin Nazarian, Paul Bogdan
Abstract
Code optimization is a daunting task that requires a significant level of expertise from experienced programmers. This level of expertise is not sufficient when compared to the rapid development of new hardware architectures. Towards advancing the whole code optimization process, recent approaches rely on machine learning and artificial intelligence techniques. This paper introduces a new framework to decrease the complexity of code optimization. The proposed framework builds on large language models (LLMs) and reinforcement learning (RL) and enables LLMs to receive feedback from their environment (i.e., unit tests) during the fine-tuning process. We compare our framework with existing state-of-the-art models and show that it is more efficient with respect to speed and computational usage, as a result of the decrement in training steps and its applicability to models with fewer parameters. Additionally, our framework reduces the possibility of logical and syntactical errors. Toward evaluating our approach, we run several experiments on the PIE dataset using a CodeT5 language model and RRHF, a new reinforcement learning algorithm. We adopt a variety of evaluation metrics with regards to optimization quality, and speedup. The evaluation results demonstrate that the proposed framework has similar results in comparison with existing models using shorter training times and smaller pre-trained models. In particular, we accomplish an increase of 5.6% and 2.2 over the baseline models concerning the %OP T and SP metrics.
Minimum-Time Trajectory Optimization With Data-Based Models: A Linear Programming Approach
Authors: Authors: Nan Li, Ehsan Taheri, Ilya Kolmanovsky, Dimitar Filev
Subjects: Systems and Control (eess.SY); Optimization and Control (math.OC)
Abstract
In this paper, we develop a computationally-efficient approach to minimum-time trajectory optimization using input-output data-based models, to produce an end-to-end data-to-control solution to time-optimal planning/control of dynamic systems and hence facilitate their autonomous operation. The approach integrates a non-parametric data-based model for trajectory prediction and a continuous optimization formulation based on an exponential weighting scheme for minimum-time trajectory planning. The optimization problem in its final form is a linear program and is easy to solve. We validate the approach and illustrate its application with a spacecraft relative motion planning problem.
On Convergence Rates of Quadratic Transform and WMMSE Methods
Authors: Authors: Kaiming Shen, Ziping Zhao, Yannan Chen, Zepeng Zhang, Hei Victor Cheng
Abstract
Fractional programming (FP) plays an important role in information science because of the Cramer-Rao bound,the Fisher information, and the signal-to-interference-plus-noise ratio (SINR). A state-of-the-art method called the quadratic transform has been extensively used to address the FP problems. This work aims to accelerate the quadratic transform-based iterative optimization via gradient projection and extrapolation. The main contributions of this work are three-fold. First, we relate the quadratic transform to the gradient projection, thereby eliminating the matrix inverse operation from the iterative optimization; our result generalizes the weighted sum-of-rates (WSR) maximization algorithm in [1] to a wide range of FP problems. Second, based on this connection to gradient projection, we incorporate Nesterov's extrapolation strategy [2] into the quadratic transform so as to accelerate the convergence of the iterative optimization. Third, from a minorization-maximization (MM) point of view, we examine the convergence rates of the conventional quadratic transform methods--which include the weighted minimum mean square error (WMMSE) algorithm as a special case--and the proposed accelerated ones. Moreover, we illustrate the practical use of the accelerated quadratic transform in two popular application cases of future wireless networks: (i) integrated sensing and communication (ISAC) and (ii) massive multiple-input multiple-output (MIMO).
A Scalable MARL Solution for Scheduling in Conflict Graphs
Authors: Authors: Yiming Zhang, Dongning Guo
Subjects: Multiagent Systems (cs.MA); Systems and Control (eess.SY)
Abstract
This paper proposes a fully scalable multi-agent reinforcement learning (MARL) approach for packet scheduling in conflict graphs, aiming to minimizing average packet delays. Each agent autonomously manages the schedule of a single link over one or multiple sub-bands, considering its own state and states of conflicting links. The problem can be conceptualized as a decentralized partially observable Markov decision process (Dec-POMDP). The proposed solution leverages an on-policy reinforcement learning algorithms multi-agent proximal policy optimization (MAPPO) within a multi-agent networked system, incorporating advanced recurrent structures in the neural network. The MARL design allows for fully decentralized training and execution, seamlessly scaling to very large networks. Extensive simulations across a diverse range of conflict graphs demonstrate that the proposed solution compares favorably to well-established schedulers in terms of both throughput and delay under various traffic conditions.
IL-NeRF: Incremental Learning for Neural Radiance Fields with Camera Pose Alignment
Authors: Authors: Letian Zhang, Ming Li, Chen Chen, Jie Xu
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Abstract
Neural radiance fields (NeRF) is a promising approach for generating photorealistic images and representing complex scenes. However, when processing data sequentially, it can suffer from catastrophic forgetting, where previous data is easily forgotten after training with new data. Existing incremental learning methods using knowledge distillation assume that continuous data chunks contain both 2D images and corresponding camera pose parameters, pre-estimated from the complete dataset. This poses a paradox as the necessary camera pose must be estimated from the entire dataset, even though the data arrives sequentially and future chunks are inaccessible. In contrast, we focus on a practical scenario where camera poses are unknown. We propose IL-NeRF, a novel framework for incremental NeRF training, to address this challenge. IL-NeRF's key idea lies in selecting a set of past camera poses as references to initialize and align the camera poses of incoming image data. This is followed by a joint optimization of camera poses and replay-based NeRF distillation. Our experiments on real-world indoor and outdoor scenes show that IL-NeRF handles incremental NeRF training and outperforms the baselines by up to $54.04\%$ in rendering quality.
Movable-Antenna Array Enabled Multiuser Uplink: A Low-Complexity Gradient Descent for Total Transmit Power Minimization
Abstract
We investigate multiuser uplink communication from multiple single-antenna users to a base station (BS), which is equipped with a movable-antenna (MA) array and adopts zero-forcing receivers to decode multiple signals. We aim to optimize the MAs' positions at the BS, to minimize the total transmit power of all users subject to the minimum rate requirement. After applying transformations, we show that the problem is equivalent to minimizing the sum of each eigenvalue's reciprocal of a matrix, which is a function of all MAs' positions. Subsequently, the projected gradient descent (PGD) method is utilized to find a locally optimal solution. In particular, different from the latest related work, we exploit the eigenvalue decomposition to successfully derive a closed-form gradient for the PGD, which facilitates the practical implementation greatly. We demonstrate by simulations that via careful optimization for all MAs' positions in our proposed design, the total transmit power of all users can be decreased significantly as compared to competitive benchmarks.
Hacking Task Confounder in Meta-Learning
Authors: Authors: Jingyao Wang, Wenwen Qiang, Yi Ren, Zeen Song, Xingzhe Su, Changwen Zheng
Abstract
Meta-learning enables rapid generalization to new tasks by learning meta-knowledge from a variety of tasks. It is intuitively assumed that the more tasks a model learns in one training batch, the richer knowledge it acquires, leading to better generalization performance. However, contrary to this intuition, our experiments reveal an unexpected result: adding more tasks within a single batch actually degrades the generalization performance. To explain this unexpected phenomenon, we conduct a Structural Causal Model (SCM) for causal analysis. Our investigation uncovers the presence of spurious correlations between task-specific causal factors and labels in meta-learning. Furthermore, the confounding factors differ across different batches. We refer to these confounding factors as ``Task Confounders". Based on this insight, we propose a plug-and-play Meta-learning Causal Representation Learner (MetaCRL) to eliminate task confounders. It encodes decoupled causal factors from multiple tasks and utilizes an invariant-based bi-level optimization mechanism to ensure their causality for meta-learning. Extensive experiments on various benchmark datasets demonstrate that our work achieves state-of-the-art (SOTA) performance.
Autotuning by Changing Directives and Number of Threads in OpenMP using ppOpen-AT
Abstract
Recently, computers have diversified architectures. To achieve high numerical calculation software performance, it is necessary to tune the software according to the target computer architecture. However, code optimization for each environment is difficult unless it is performed by a specialist who knows computer architectures well. By applying autotuning (AT), the tuning effort can be reduced. Optimized implementation by AT that enhances computer performance can be used even by non-experts. In this research, we propose a technique for AT for programs using open multi-processing (OpenMP). We propose an AT method using an AT language that changes the OpenMP optimized loop and dynamically changes the number of threads in OpenMP according to computational kernels. Performance evaluation was performed using the Fujitsu PRIMEHPC FX100, which is a K-computer type supercomputer installed at the Information Technology Center, Nagoya University. As a result, we found there was a performance increase of 1.801 times that of the original code in a plasma turbulence analysis.
Graph-based Prediction and Planning Policy Network (GP3Net) for scalable self-driving in dynamic environments using Deep Reinforcement Learning
Authors: Authors: Jayabrata Chowdhury, Venkataramanan Shivaraman, Suresh Sundaram, P B Sujit
Abstract
Recent advancements in motion planning for Autonomous Vehicles (AVs) show great promise in using expert driver behaviors in non-stationary driving environments. However, learning only through expert drivers needs more generalizability to recover from domain shifts and near-failure scenarios due to the dynamic behavior of traffic participants and weather conditions. A deep Graph-based Prediction and Planning Policy Network (GP3Net) framework is proposed for non-stationary environments that encodes the interactions between traffic participants with contextual information and provides a decision for safe maneuver for AV. A spatio-temporal graph models the interactions between traffic participants for predicting the future trajectories of those participants. The predicted trajectories are utilized to generate a future occupancy map around the AV with uncertainties embedded to anticipate the evolving non-stationary driving environments. Then the contextual information and future occupancy maps are input to the policy network of the GP3Net framework and trained using Proximal Policy Optimization (PPO) algorithm. The proposed GP3Net performance is evaluated on standard CARLA benchmarking scenarios with domain shifts of traffic patterns (urban, highway, and mixed). The results show that the GP3Net outperforms previous state-of-the-art imitation learning-based planning models for different towns. Further, in unseen new weather conditions, GP3Net completes the desired route with fewer traffic infractions. Finally, the results emphasize the advantage of including the prediction module to enhance safety measures in non-stationary environments.
Physics-Aware Multifidelity Bayesian Optimization: a Generalized Formulation
Authors: Authors: Francesco Di Fiore, Laura Mainini
Subjects: Machine Learning (cs.LG); Optimization and Control (math.OC)
Abstract
The adoption of high-fidelity models for many-query optimization problems is majorly limited by the significant computational cost required for their evaluation at every query. Multifidelity Bayesian methods (MFBO) allow to include costly high-fidelity responses for a sub-selection of queries only, and use fast lower-fidelity models to accelerate the optimization process. State-of-the-art methods rely on a purely data-driven search and do not include explicit information about the physical context. This paper acknowledges that prior knowledge about the physical domains of engineering problems can be leveraged to accelerate these data-driven searches, and proposes a generalized formulation for MFBO to embed a form of domain awareness during the optimization procedure. In particular, we formalize a bias as a multifidelity acquisition function that captures the physical structure of the domain. This permits to partially alleviate the data-driven search from learning the domain properties on-the-fly, and sensitively enhances the management of multiple sources of information. The method allows to efficiently include high-fidelity simulations to guide the optimization search while containing the overall computational expense. Our physics-aware multifidelity Bayesian optimization is presented and illustrated for two classes of optimization problems frequently met in science and engineering, namely design optimization and health monitoring problems.
Data-Driven Robust Covariance Control for Uncertain Linear Systems
Abstract
The theory of covariance control and covariance steering (CS) deals with controlling the dispersion of trajectories of a dynamical system, under the implicit assumption that accurate prior knowledge of the system being controlled is available. In this work, we consider the problem of steering the distribution of a discrete-time, linear system subject to exogenous disturbances under an unknown dynamics model. Leveraging concepts from behavioral systems theory, the trajectories of this unknown, noisy system may be (approximately) represented using system data collected through experimentation. Using this fact, we formulate a direct data-driven covariance control problem using input-state data. We then propose a maximum likelihood uncertainty quantification method to estimate and bound the noise realizations in the data collection process. Lastly, we utilize robust convex optimization techniques to solve the resulting norm-bounded uncertain convex program. We illustrate the proposed end-to-end data-driven CS algorithm on a double integrator example and showcase the efficacy and accuracy of the proposed method compared to that of model-based methods
Composite Survival Analysis: Learning with Auxiliary Aggregated Baselines and Survival Scores
Abstract
Survival Analysis (SA) constitutes the default method for time-to-event modeling due to its ability to estimate event probabilities of sparsely occurring events over time. In this work, we show how to improve the training and inference of SA models by decoupling their full expression into (1) an aggregated baseline hazard, which captures the overall behavior of a given population, and (2) independently distributed survival scores, which model idiosyncratic probabilistic dynamics of its given members, in a fully parametric setting. The proposed inference method is shown to dynamically handle right-censored observation horizons, and to achieve competitive performance when compared to other state-of-the-art methods in a variety of real-world datasets, including computationally inefficient Deep Learning-based SA methods and models that require MCMC for inference. Nevertheless, our method achieves robust results from the outset, while not being subjected to fine-tuning or hyperparameter optimization.
Optimization for the Metaverse over Mobile Edge Computing with Play to Earn
Authors: Authors: Chang Liu, Terence Jie Chua, Jun Zhao
Subjects: Distributed, Parallel, and Cluster Computing (cs.DC); Systems and Control (eess.SY); Optimization and Control (math.OC)
Abstract
The concept of the Metaverse has garnered growing interest from both academic and industry circles. The decentralization of both the integrity and security of digital items has spurred the popularity of play-to-earn (P2E) games, where players are entitled to earn and own digital assets which they may trade for physical-world currencies. However, these computationally-intensive games are hardly playable on resource-limited mobile devices and the computational tasks have to be offloaded to an edge server. Through mobile edge computing (MEC), users can upload data to the Metaverse Service Provider (MSP) edge servers for computing. Nevertheless, there is a trade-off between user-perceived in-game latency and user visual experience. The downlink transmission of lower-resolution videos lowers user-perceived latency while lowering the visual fidelity and consequently, earnings of users. In this paper, we design a method to enhance the Metaverse-based mobile augmented reality (MAR) in-game user experience. Specifically, we formulate and solve a multi-objective optimization problem. Given the inherent NP-hardness of the problem, we present a low-complexity algorithm to address it, mitigating the trade-off between delay and earnings. The experiment results show that our method can effectively balance the user-perceived latency and profitability, thus improving the performance of Metaverse-based MAR systems.
Learning for CasADi: Data-driven Models in Numerical Optimization
Authors: Authors: Tim Salzmann, Jon Arrizabalaga, Joel Andersson, Marco Pavone, Markus Ryll
Subjects: Systems and Control (eess.SY); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Robotics (cs.RO); Symbolic Computation (cs.SC)
Abstract
While real-world problems are often challenging to analyze analytically, deep learning excels in modeling complex processes from data. Existing optimization frameworks like CasADi facilitate seamless usage of solvers but face challenges when integrating learned process models into numerical optimizations. To address this gap, we present the Learning for CasADi (L4CasADi) framework, enabling the seamless integration of PyTorch-learned models with CasADi for efficient and potentially hardware-accelerated numerical optimization. The applicability of L4CasADi is demonstrated with two tutorial examples: First, we optimize a fish's trajectory in a turbulent river for energy efficiency where the turbulent flow is represented by a PyTorch model. Second, we demonstrate how an implicit Neural Radiance Field environment representation can be easily leveraged for optimal control with L4CasADi. L4CasADi, along with examples and documentation, is available under MIT license at https://github.com/Tim-Salzmann/l4casadi
Maximum capacity path problem with loss factors
Authors: Authors: Adrian Marius Deaconu, Javad Tayyebi, Mihai-Lucian Rîtan
Abstract
The maximum capacity path problem is to find a path from a source to a sink which has the maximum capacity among all paths. This paper addresses an extension of this problem which considers loss factors. It is called the generalized maximum capacity path problem. The problem is a network flow optimization problem whose network contains capacities as well as loss factors for arcs. The aim of the problem is to find a path from an origin to a destination so as to send a maximum flow along the path considering loss factors and respecting capacity constraints. The paper presents a zero-one formulation of the problem and moreover, it presents two efficient algorithms which solve the problem in polynomial time.
The survival of scientific stylization
Authors: Authors: Yuanyuan Shu, Tianxing Pan
Subjects: Digital Libraries (cs.DL); Social and Information Networks (cs.SI); Physics and Society (physics.soc-ph)
Abstract
This study elaborates a text-based metric to quantify the unique position of stylized scientific research, characterized by its innovative integration of diverse knowledge components and potential to pivot established scientific paradigms. Our analysis reveals a concerning decline in stylized research, highlighted by its comparative undervaluation in terms of citation counts and protracted peer-review duration. Despite facing these challenges, the disruptive potential of stylized research remains robust, consistently introducing groundbreaking questions and theories. This paper posits that substantive reforms are necessary to incentivize and recognize the value of stylized research, including optimizations to the peer-review process and the criteria for evaluating scientific impact. Embracing these changes may be imperative to halt the downturn in stylized research and ensure enduring scholarly exploration in endless frontiers.
Uncertainty Propagation through Trained Deep Neural Networks Using Factor Graphs
Authors: Authors: Angel Daruna, Yunye Gong, Abhinav Rajvanshi, Han-Pang Chiu, Yi Yao
Abstract
Predictive uncertainty estimation remains a challenging problem precluding the use of deep neural networks as subsystems within safety-critical applications. Aleatoric uncertainty is a component of predictive uncertainty that cannot be reduced through model improvements. Uncertainty propagation seeks to estimate aleatoric uncertainty by propagating input uncertainties to network predictions. Existing uncertainty propagation techniques use one-way information flows, propagating uncertainties layer-by-layer or across the entire neural network while relying either on sampling or analytical techniques for propagation. Motivated by the complex information flows within deep neural networks (e.g. skip connections), we developed and evaluated a novel approach by posing uncertainty propagation as a non-linear optimization problem using factor graphs. We observed statistically significant improvements in performance over prior work when using factor graphs across most of our experiments that included three datasets and two neural network architectures. Our implementation balances the benefits of sampling and analytical propagation techniques, which we believe, is a key factor in achieving performance improvements.
Neural Architecture Codesign for Fast Bragg Peak Analysis
Authors: Authors: Luke McDermott, Jason Weitz, Dmitri Demler, Daniel Cummings, Nhan Tran, Javier Duarte
Abstract
We develop an automated pipeline to streamline neural architecture codesign for fast, real-time Bragg peak analysis in high-energy diffraction microscopy. Traditional approaches, notably pseudo-Voigt fitting, demand significant computational resources, prompting interest in deep learning models for more efficient solutions. Our method employs neural architecture search and AutoML to enhance these models, including hardware costs, leading to the discovery of more hardware-efficient neural architectures. Our results match the performance, while achieving a 13$\times$ reduction in bit operations compared to the previous state-of-the-art. We show further speedup through model compression techniques such as quantization-aware-training and neural network pruning. Additionally, our hierarchical search space provides greater flexibility in optimization, which can easily extend to other tasks and domains.
Constructing Vec-tionaries to Extract Latent Message Features from Texts: A Case Study of Moral Appeals
Authors: Authors: Zening Duan, Anqi Shao, Yicheng Hu, Heysung Lee, Xining Liao, Yoo Ji Suh, Jisoo Kim, Kai-Cheng Yang, Kaiping Chen, Sijia Yang
Abstract
While communication research frequently studies latent message features like moral appeals, their quantification remains a challenge. Conventional human coding struggles with scalability and intercoder reliability. While dictionary-based methods are cost-effective and computationally efficient, they often lack contextual sensitivity and are limited by the vocabularies developed for the original applications. In this paper, we present a novel approach to construct vec-tionary measurement tools that boost validated dictionaries with word embeddings through nonlinear optimization. By harnessing semantic relationships encoded by embeddings, vec-tionaries improve the measurement of latent message features by expanding the applicability of original vocabularies to other contexts. Vec-tionaries can also help extract semantic information from texts, especially those in short format, beyond the original vocabulary of a dictionary. Importantly, a vec-tionary can produce additional metrics to capture the valence and ambivalence of a latent feature beyond its strength in texts. Using moral appeals in COVID-19-related tweets as a case study, we illustrate the steps to construct the moral foundations vec-tionary, showcasing its ability to process posts missed by dictionary methods and to produce measurements better aligned with crowdsourced human assessments. Furthermore, additional metrics from the moral foundations vec-tionary unveiled unique insights that facilitated predicting outcomes such as message retransmission.
From Correspondences to Pose: Non-minimal Certifiably Optimal Relative Pose without Disambiguation
Authors: Authors: Javier Tirado-Garín, Javier Civera
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Abstract
Estimating the relative camera pose from $n \geq 5$ correspondences between two calibrated views is a fundamental task in computer vision. This process typically involves two stages: 1) estimating the essential matrix between the views, and 2) disambiguating among the four candidate relative poses that satisfy the epipolar geometry. In this paper, we demonstrate a novel approach that, for the first time, bypasses the second stage. Specifically, we show that it is possible to directly estimate the correct relative camera pose from correspondences without needing a post-processing step to enforce the cheirality constraint on the correspondences. Building on recent advances in certifiable non-minimal optimization, we frame the relative pose estimation as a Quadratically Constrained Quadratic Program (QCQP). By applying the appropriate constraints, we ensure the estimation of a camera pose that corresponds to a valid 3D geometry and that is globally optimal when certified. We validate our method through exhaustive synthetic and real-world experiments, confirming the efficacy, efficiency and accuracy of the proposed approach. Code is available at https://github.com/javrtg/C2P.
Multiplier Optimization via E-Graph Rewriting
Authors: Authors: Andy Wanna (1), Samuel Coward (1 and 2), Theo Drane (2), George A. Constantinides (1), Miloš D. Ercegovac (3) ((1) Imperial College London, (2) Intel Corporation (3) University of California Los Angeles)
Abstract
Multiplier circuits account for significant resource usage in datapath-dominated circuit designs, and RTL designers continue to build bespoke hand-crafted multiplication arrays for their particular application. The construction of an optimized multiplier presents trade-offs between pre-processing to generate a smaller array and array reduction. A data structure known as an e-graph has recently been applied to datapath optimization, where the e-graph's ability to efficiently explore trade-offs has been shown to be crucial. We propose an e-graph based rewriting framework to construct optimized multiplier circuits. Such a framework can express alternative multiplier representations and generate customized circuit designs. We demonstrate that the proposed tool, which we call OptiMult, can reduce the latency of a squarer by up to 46% and reduce the latency of a standard multiplier by up to 9% when compared against logic synthesis instantiated components.
Software issues report for bug fixing process: An empirical study of machine-learning libraries
Authors: Authors: Adekunle Ajibode, Dong Yunwei, Yang Hongji
Abstract
Issue resolution and bug-fixing processes are essential in the development of machine-learning libraries, similar to software development, to ensure well-optimized functions. Understanding the issue resolution and bug-fixing process of machine-learning libraries can help developers identify areas for improvement and optimize their strategies for issue resolution and bug-fixing. However, detailed studies on this topic are lacking. Therefore, we investigated the effectiveness of issue resolution for bug-fixing processes in six machine-learning libraries: Tensorflow, Keras, Theano, Pytorch, Caffe, and Scikit-learn. We addressed seven research questions (RQs) using 16,921 issues extracted from the GitHub repository via the GitHub Rest API. We employed several quantitative methods of data analysis, including correlation, OLS regression, percentage and frequency count, and heatmap to analyze the RQs. We found the following through our empirical investigation: (1) The most common categories of issues that arise in machine-learning libraries are bugs, documentation, optimization, crashes, enhancement, new feature requests, build/CI, support, and performance. (2) Effective strategies for addressing these problems include fixing critical bugs, optimizing performance, and improving documentation. (3) These categorized issues are related to testing and runtime and are common among all six machine-learning libraries. (4) Monitoring the total number of comments on issues can provide insights into the duration of the issues. (5) It is crucial to strike a balance between prioritizing critical issues and addressing other issues in a timely manner. Therefore, this study concludes that efficient issue-tracking processes, effective communication, and collaboration are vital for effective resolution of issues and bug fixing processes in machine-learning libraries.
Complex-valued Neural Networks -- Theory and Analysis
Authors: Authors: Rayyan Abdalla
Subjects: Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE); Systems and Control (eess.SY)
Abstract
Complex-valued neural networks (CVNNs) have recently been successful in various pioneering areas which involve wave-typed information and frequency-domain processing. This work addresses different structures and classification of CVNNs. The theory behind complex activation functions, implications related to complex differentiability and special activations for CVNN output layers are presented. The work also discusses CVNN learning and optimization using gradient and non-gradient based algorithms. Complex Backpropagation utilizing complex chain rule is also explained in terms of Wirtinger calculus. Moreover, special modules for building CVNN models, such as complex batch normalization and complex random initialization are also discussed. The work also highlights libraries and software blocks proposed for CVNN implementations and discusses future directions. The objective of this work is to understand the dynamics and most recent developments of CVNNs.
ROSE: A Recognition-Oriented Speech Enhancement Framework in Air Traffic Control Using Multi-Objective Learning
Authors: Authors: Xincheng Yu, Dongyue Guo, Jianwei Zhang, Yi Lin
Abstract
Radio speech echo is a specific phenomenon in the air traffic control (ATC) domain, which degrades speech quality and further impacts automatic speech recognition (ASR) accuracy. In this work, a recognition-oriented speech enhancement (ROSE) framework is proposed to improve speech intelligibility and also advance ASR accuracy, which serves as a plug-and-play tool in ATC scenarios and does not require additional retraining of the ASR model. Specifically, an encoder-decoder-based U-Net framework is proposed to eliminate the radio speech echo based on the real-world collected corpus. By incorporating the SE-oriented and ASR-oriented loss, ROSE is implemented in a multi-objective manner by learning shared representations across the two optimization objectives. An attention-based skip-fusion (ABSF) mechanism is applied to skip connections to refine the features. A channel and sequence attention (CSAtt) block is innovatively designed to guide the model to focus on informative representations and suppress disturbing features. The experimental results show that the ROSE significantly outperforms other state-of-the-art methods for both the SE and ASR tasks. In addition, the proposed approach can contribute to the desired performance improvements on public datasets.
Abstract
In software engineering, the meticulous configuration of software tools is crucial in ensuring optimal performance within intricate systems. However, the complexity inherent in selecting optimal configurations is exacerbated by the high-dimensional search spaces presented in modern applications. Conventional trial-and-error or intuition-driven methods are both inefficient and error-prone, impeding scalability and reproducibility. In this study, we embark on an exploration of leveraging Large-Language Models (LLMs) to streamline the software configuration process. We identify that the task of hyperparameter configuration for machine learning components within intelligent applications is particularly challenging due to the extensive search space and performance-critical nature. Existing methods, including Bayesian optimization, have limitations regarding initial setup, computational cost, and convergence efficiency. Our work presents a novel approach that employs LLMs, such as Chat-GPT, to identify starting conditions and narrow down the search space, improving configuration efficiency. We conducted a series of experiments to investigate the variability of LLM-generated responses, uncovering intriguing findings such as potential response caching and consistent behavior based on domain-specific keywords. Furthermore, our results from hyperparameter optimization experiments reveal the potential of LLMs in expediting initialization processes and optimizing configurations. While our initial insights are promising, they also indicate the need for further in-depth investigations and experiments in this domain.
Efficient Estimation of Pairwise Effective Resistance
Authors: Authors: Renchi Yang, Jing Tang
Subjects: Social and Information Networks (cs.SI); Databases (cs.DB)
Abstract
Given an undirected graph G, the effective resistance r(s,t) measures the dissimilarity of node pair s,t in G, which finds numerous applications in real-world problems, such as recommender systems, combinatorial optimization, molecular chemistry, and electric power networks. Existing techniques towards pairwise effective resistance estimation either trade approximation guarantees for practical efficiency, or vice versa. In particular, the state-of-the-art solution is based on a multitude of Monte Carlo random walks, rendering it rather inefficient in practice, especially on large graphs. Motivated by this, this paper first presents an improved Monte Carlo approach, AMC, which reduces both the length and amount of random walks required without degrading the theoretical accuracy guarantee, through careful theoretical analysis and an adaptive sampling scheme. Further, we develop a greedy approach, GEER, which combines AMC with sparse matrix-vector multiplications in an optimized and non-trivial way. GEER offers significantly improved practical efficiency over AMC without compromising its asymptotic performance and accuracy guarantees. Extensive experiments on multiple benchmark datasets reveal that GEER is orders of magnitude faster than the state of the art in terms of computational time when achieving the same accuracy.
Pre-Evolved Model for Complex Multi-objective Optimization Problems
Authors: Authors: Haokai Hong, Min Jiang
Subjects: Neural and Evolutionary Computing (cs.NE)
Abstract
Multi-objective optimization problems (MOPs) necessitate the simultaneous optimization of multiple objectives. Numerous studies have demonstrated that evolutionary computation is a promising paradigm for solving complex MOPs, which involve optimization problems with large-scale decision variables, many objectives, and expensive evaluation functions. However, existing multi-objective evolutionary algorithms (MOEAs) encounter significant challenges in generating high-quality populations when solving diverse complex MOPs. Specifically, the distinct requirements and constraints of the population result in the inefficiency or even incompetence of MOEAs in addressing various complex MOPs. Therefore, this paper proposes the concept of pre-evolving for MOEAs to generate high-quality populations for diverse complex MOPs. Drawing inspiration from the classical transformer architecture, we devise dimension embedding and objective encoding techniques to configure the pre-evolved model (PEM). The PEM is pre-evolved on a substantial number of existing MOPs. Subsequently, when fine-evolving on new complex MOPs, the PEM transforms the population into the next generation to approximate the Pareto-optimal front. Furthermore, it utilizes evaluations on new solutions to iteratively update the PEM for subsequent generations, thereby efficiently solving various complex MOPs. Experimental results demonstrate that the PEM outperforms state-of-the-art MOEAs on a range of complex MOPs.
Order Matters in the Presence of Dataset Imbalance for Multilingual Learning
Authors: Authors: Dami Choi, Derrick Xin, Hamid Dadkhahi, Justin Gilmer, Ankush Garg, Orhan Firat, Chih-Kuan Yeh, Andrew M. Dai, Behrooz Ghorbani
Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG)
Abstract
In this paper, we empirically study the optimization dynamics of multi-task learning, particularly focusing on those that govern a collection of tasks with significant data imbalance. We present a simple yet effective method of pre-training on high-resource tasks, followed by fine-tuning on a mixture of high/low-resource tasks. We provide a thorough empirical study and analysis of this method's benefits showing that it achieves consistent improvements relative to the performance trade-off profile of standard static weighting. We analyze under what data regimes this method is applicable and show its improvements empirically in neural machine translation (NMT) and multi-lingual language modeling.
Long-Term Carbon-Efficient Planning for Geographically Shiftable Resources: A Monte Carlo Tree Search Approach
Authors: Authors: Xuan He, Danny H.K. Tsang, Yize Chen
Abstract
Global climate challenge is demanding urgent actions for decarbonization, while electric power systems take the major roles in clean energy transition. Due to the existence of spatially and temporally dispersed renewable energy resources and the uneven distribution of carbon emission intensity throughout the grid, it is worth investigating future load planning and demand management to offset those generations with higher carbon emission rates. Such techniques include inter-region utilization of geographically shiftable resources and stochastic renewable energy. For instance, data center is considered to be a major carbon emission producer in the future due to increasing information load, while it holds the capability of geographical load balancing. In this paper, we propose a novel planning and operation model minimizing the system-level carbon emissions via sitting and operating geographically shiftable resources. This model decides the optimal locations for shiftable resources expansion along with power dispatch schedule. To accommodate future system operation patterns and a wide range of operating conditions, we incorporate 20-year fine-grained load and renewables scenarios for grid simulations of realistic sizes (e.g., up to 1888 buses). To tackle the computational challenges coming from the combinatorial nature of such large-scale planning problem, we develop a customized Monte Carlo Tree Search (MCTS) method, which can find reasonable solutions satisfying solution time limits. Besides, MCTS enables flexible time window settings and offline solution adjustments. Extensive simulations validate that our planning model can reduce more than 10\% carbon emission across all setups. Compared to off-the-shelf optimization solvers such as Gurobi, our method achieves up to 8.1X acceleration while the solution gaps are less than 1.5\% in large-scale cases.
Motion Planning for Multiple Mobile Manipulator System in Complex Flipping Manipulation
Authors: Authors: Wenhang Liu, Kun Song, Meng Ren, Jiawei Hu, Michael Yu Wang, Zhenhua Xiong
Abstract
Multiple robot systems are favored for object manipulation and transportation, especially for large objects. However, in more complex manipulation such as flipping, these systems encounter a new challenge, configuration disconnectivity of manipulators. Grasping objects by manipulators will impose closed-chain constraints on the system, which in turn limits the feasible motions of manipulators and further compromises the configuration connectivity. Multiple mobile manipulator systems show much more flexibility in object manipulation with the mobility of the mobile platform and have the potential to address the above problem. In this paper, a novel planning framework is proposed for complex flipping manipulation by incorporating platform motions and regrasping. Firstly, two types of trajectories, mobile manipulator planning and regrasping planning, are classified and can be assigned different priorities for different tasks. Secondly, corresponding planning methods are designed for each type of trajectory. Specifically, in mobile manipulator planning, the configuration of the platform is determined through optimization to ensure connectivity when the manipulator approaches configuration boundaries. In regrasping planning, closed-chain constraints are temporarily disregarded and the manipulation capabilities are prioritized to facilitate subsequent planning. Finally, the structure of the overall planning framework is provided. Experimental results demonstrate that the proposed planner efficiently plans the motions of the system to accomplish flipping manipulation. Additionally, a comprehensive experiment emphasizes the significance of our planner in extending the capabilities of multiple mobile manipulator systems in complex tasks.
Concrete Subspace Learning based Interference Elimination for Multi-task Model Fusion
Authors: Authors: Anke Tang, Li Shen, Yong Luo, Liang Ding, Han Hu, Bo Du, Dacheng Tao
Abstract
Merging models fine-tuned from a common, extensively pre-trained large model but specialized for different tasks has been demonstrated as a cheap and scalable strategy to construct a multi-task model that performs well across diverse tasks. Recent research, exemplified by task arithmetic, highlights that this multi-task model can be derived through arithmetic operations on task vectors. Nevertheless, current merging techniques frequently resolve potential conflicts among parameters from task-specific models by evaluating individual attributes, such as the parameters' magnitude or sign, overlooking their collective impact on the overall functionality of the model. In this work, we propose the CONtinuous relaxation of disCRETE (Concrete) subspace learning method to identify a common low-dimensional subspace and utilize its shared information to track the interference problem without sacrificing much performance. Specifically, we model the problem as a bi-level optimization problem and introduce a meta-learning framework to find the Concrete subspace mask through gradient-based techniques. At the upper level, we focus on learning a shared Concrete mask to identify the subspace, while at the inner level, model merging is performed to maximize the performance of the merged model. We conduct extensive experiments on both vision domain and language domain, and the results demonstrate the effectiveness of our method. The code is available at https://github.com/tanganke/subspace_fusion
Optimized View and Geometry Distillation from Multi-view Diffuser
Authors: Authors: Youjia Zhang, Junqing Yu, Zikai Song, Wei Yang
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Abstract
Generating multi-view images from a single input view using image-conditioned diffusion models is a recent advancement and has shown considerable potential. However, issues such as the lack of consistency in synthesized views and over-smoothing in extracted geometry persist. Previous methods integrate multi-view consistency modules or impose additional supervisory to enhance view consistency while compromising on the flexibility of camera positioning and limiting the versatility of view synthesis. In this study, we consider the radiance field optimized during geometry extraction as a more rigid consistency prior, compared to volume and ray aggregation used in previous works. We further identify and rectify a critical bias in the traditional radiance field optimization process through score distillation from a multi-view diffuser. We introduce an Unbiased Score Distillation (USD) that utilizes unconditioned noises from a 2D diffusion model, greatly refining the radiance field fidelity. we leverage the rendered views from the optimized radiance field as the basis and develop a two-step specialization process of a 2D diffusion model, which is adept at conducting object-specific denoising and generating high-quality multi-view images. Finally, we recover faithful geometry and texture directly from the refined multi-view images. Empirical evaluations demonstrate that our optimized geometry and view distillation technique generates comparable results to the state-of-the-art models trained on extensive datasets, all while maintaining freedom in camera positioning.
Towards Transferable Adversarial Attacks with Centralized Perturbation
Abstract
Adversarial transferability enables black-box attacks on unknown victim deep neural networks (DNNs), rendering attacks viable in real-world scenarios. Current transferable attacks create adversarial perturbation over the entire image, resulting in excessive noise that overfit the source model. Concentrating perturbation to dominant image regions that are model-agnostic is crucial to improving adversarial efficacy. However, limiting perturbation to local regions in the spatial domain proves inadequate in augmenting transferability. To this end, we propose a transferable adversarial attack with fine-grained perturbation optimization in the frequency domain, creating centralized perturbation. We devise a systematic pipeline to dynamically constrain perturbation optimization to dominant frequency coefficients. The constraint is optimized in parallel at each iteration, ensuring the directional alignment of perturbation optimization with model prediction. Our approach allows us to centralize perturbation towards sample-specific important frequency features, which are shared by DNNs, effectively mitigating source model overfitting. Experiments demonstrate that by dynamically centralizing perturbation on dominating frequency coefficients, crafted adversarial examples exhibit stronger transferability, and allowing them to bypass various defenses.
Transforms for Multiplicative and Fractional Programming with Broad Applications in Edge Computing and Communication Networks
Authors: Authors: Yitong Wang, Chang Liu, Jun Zhao
Abstract
Multiplicative Programming (MP) pertains to a spectrum of optimization problems that involve product term(s). As computational paradigms of communication systems continue to evolve, particularly concerning the offloading strategies of computationally intensive tasks simultaneously to centralized or decentralized servers, designing or optimizing effective communication systems with MP techniques becomes increasingly indispensable. Similarly, Fractional Programming (FP) is another significant branch in the optimization domain, addressing various essential scenarios in communication. For instance, in minimization optimization problems, transmission power and processing delay of communication systems are considered critical metrics. In a very recent JSAC paper by Zhao et al. [2], an innovative transform (Zhao's Optimization Transform) was proposed for solving the minimization of MP and FP problems. Nevertheless, the resolution of optimization problems in communication systems encounters several limitations when adopting Zhao's optimization transform, especially in MP problems. Primarily, objective functions proposed in these optimization problems typically involve sum-of-products terms and the optimization variables are always discrete leading to NP-hard problems. Furthermore, multiple functions mapping to the non-negative domain in these scenarios can result in auxiliary variables being zero values, while the same situation is avoidable in FP problems due to the presence of these functions in the denominator. In this paper, we introduce an updated transform, building on the foundations of Zhao's original method, designed to effectively overcome these challenges by reformulating the original problem into a series of convex or concave problems. This introduced problem reformulation provides a superior iteration algorithm with demonstrable convergence to a stationary point.
Offloading and Quality Control for AI Generated Content Services in Edge Computing Networks
Authors: Authors: Yitong Wang, Chang Liu, Jun Zhao
Subjects: Artificial Intelligence (cs.AI); Networking and Internet Architecture (cs.NI); Performance (cs.PF)
Abstract
AI-Generated Content (AIGC), as a novel manner of providing Metaverse services in the forthcoming Internet paradigm, can resolve the obstacles of immersion requirements. Concurrently, edge computing, as an evolutionary paradigm of computing in communication systems, effectively augments real-time interactive services. In pursuit of enhancing the accessibility of AIGC services, the deployment of AIGC models (e.g., diffusion models) to edge servers and local devices has become a prevailing trend. Nevertheless, this approach faces constraints imposed by battery life and computational resources when tasks are offloaded to local devices, limiting the capacity to deliver high-quality content to users while adhering to stringent latency requirements. So there will be a tradeoff between the utility of AIGC models and offloading decisions in the edge computing paradigm. This paper proposes a joint optimization algorithm for offloading decisions, computation time, and diffusion steps of the diffusion models in the reverse diffusion stage. Moreover, we take the average error into consideration as the metric for evaluating the quality of the generated results. Experimental results conclusively demonstrate that the proposed algorithm achieves superior joint optimization performance compared to the baselines.
Invariant Representation Learning via Decoupling Style and Spurious Features
Authors: Authors: Ruimeng Li, Yuanhao Pu, Zhaoyi Li, Hong Xie, Defu Lian
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Abstract
This paper considers the out-of-distribution (OOD) generalization problem under the setting that both style distribution shift and spurious features exist and domain labels are missing. This setting frequently arises in real-world applications and is underlooked because previous approaches mainly handle either of these two factors. The critical challenge is decoupling style and spurious features in the absence of domain labels. To address this challenge, we first propose a structural causal model (SCM) for the image generation process, which captures both style distribution shift and spurious features. The proposed SCM enables us to design a new framework called IRSS, which can gradually separate style distribution and spurious features from images by introducing adversarial neural networks and multi-environment optimization, thus achieving OOD generalization. Moreover, it does not require additional supervision (e.g., domain labels) other than the images and their corresponding labels. Experiments on benchmark datasets demonstrate that IRSS outperforms traditional OOD methods and solves the problem of Invariant risk minimization (IRM) degradation, enabling the extraction of invariant features under distribution shift.
No Prior Mask: Eliminate Redundant Action for Deep Reinforcement Learning
Abstract
The large action space is one fundamental obstacle to deploying Reinforcement Learning methods in the real world. The numerous redundant actions will cause the agents to make repeated or invalid attempts, even leading to task failure. Although current algorithms conduct some initial explorations for this issue, they either suffer from rule-based systems or depend on expert demonstrations, which significantly limits their applicability in many real-world settings. In this work, we examine the theoretical analysis of what action can be eliminated in policy optimization and propose a novel redundant action filtering mechanism. Unlike other works, our method constructs the similarity factor by estimating the distance between the state distributions, which requires no prior knowledge. In addition, we combine the modified inverse model to avoid extensive computation in high-dimensional state space. We reveal the underlying structure of action spaces and propose a simple yet efficient redundant action filtering mechanism named No Prior Mask (NPM) based on the above techniques. We show the superior performance of our method by conducting extensive experiments on high-dimensional, pixel-input, and stochastic problems with various action redundancy. Our code is public online at https://github.com/zhongdy15/npm.
Security and Reliability Evaluation of Countermeasures implemented using High-Level Synthesis
Authors: Authors: Amalia Artemis Koufopoulou, Kalliopi Xevgeni, Athanasios Papadimitriou, Mihalis Psarakis, David Hely
Subjects: Cryptography and Security (cs.CR); Hardware Architecture (cs.AR)
Abstract
As the complexity of digital circuits increases, High-Level Synthesis (HLS) is becoming a valuable tool to increase productivity and design reuse by utilizing relevant Electronic Design Automation (EDA) flows, either for Application-Specific Integrated Circuits (ASIC) or for Field Programmable Gate Arrays (FPGA). Side Channel Analysis (SCA) and Fault Injection (FI) attacks are powerful hardware attacks, capable of greatly weakening the theoretical security levels of secure implementations. Furthermore, critical applications demand high levels of reliability including fault tolerance. The lack of security and reliability driven optimizations in HLS tools makes it necessary for the HLS-based designs to validate that the properties of the algorithm and the countermeasures have not been compromised due to the HLS flow. In this work, we provide results on the resilience evaluation of HLS-based FPGA implementations for the aforementioned threats. As a test case, we use multiple versions of an on-the-fly SBOX algorithm integrating different countermeasures (hiding and masking), written in C and implemented using Vivado HLS. We perform extensive evaluations for all the designs and their optimization scenarios. The results provide evidence of issues arising due to HLS optimizations on the security and the reliability of cryptographic implementations. Furthermore, the results put HLS algorithms to the test of designing secure accelerators and can lead to improving them towards the goal of increasing productivity in the domain of secure and reliable cryptographic implementations.
Flow Symmetrization for Parameterized Constrained Diffeomorphisms
Abstract
Diffeomorphisms play a crucial role while searching for shapes with fixed topological properties, allowing for smooth deformation of template shapes. Several approaches use diffeomorphism for shape search. However, these approaches employ only unconstrained diffeomorphisms. In this work, we develop Flow Symmetrization - a method to represent a parametric family of constrained diffeomorphisms that contain additional symmetry constraints such as periodicity, rotation equivariance, and transflection equivariance. Our representation is differentiable in nature, making it suitable for gradient-based optimization approaches for shape search. As these symmetry constraints naturally arise in tiling classes, our method is ideal for representing tile shapes belonging to any tiling class. To demonstrate the efficacy of our method, we design two frameworks for addressing the challenging problems of Escherization and Density Estimation. The first framework is dedicated to the Escherization problem, where we parameterize tile shapes belonging to different isohedral classes. Given a target shape, the template tile is deformed using gradient-based optimization to resemble the target shape. The second framework focuses on density estimation in identification spaces. By leveraging the inherent link between tiling theory and identification topology, we design constrained diffeomorphisms for the plane that result in unconstrained diffeomorphisms on the identification spaces. Specifically, we perform density estimation on identification spaces such as torus, sphere, Klein bottle, and projective plane. Through results and experiments, we demonstrate that our method obtains impressive results for Escherization on the Euclidean plane and density estimation on non-Euclidean identification spaces.
TPRNN: A Top-Down Pyramidal Recurrent Neural Network for Time Series Forecasting
Authors: Authors: Ling Chen, Jiahua Cui
Subjects: Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
Abstract
Time series refer to a series of data points indexed in time order, which can be found in various fields, e.g., transportation, healthcare, and finance. Accurate time series forecasting can enhance optimization planning and decision-making support. Time series have multi-scale characteristics, i.e., different temporal patterns at different scales, which presents a challenge for time series forecasting. In this paper, we propose TPRNN, a Top-down Pyramidal Recurrent Neural Network for time series forecasting. We first construct subsequences of different scales from the input, forming a pyramid structure. Then by executing a multi-scale information interaction module from top to bottom, we model both the temporal dependencies of each scale and the influences of subsequences of different scales, resulting in a complete modeling of multi-scale temporal patterns in time series. Experiments on seven real-world datasets demonstrate that TPRNN has achieved the state-of-the-art performance with an average improvement of 8.13% in MSE compared to the best baseline.
Deep Imbalanced Learning for Multimodal Emotion Recognition in Conversations
Authors: Authors: Tao Meng, Yuntao Shou, Wei Ai, Nan Yin, Keqin Li
Subjects: Sound (cs.SD); Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)
Abstract
The main task of Multimodal Emotion Recognition in Conversations (MERC) is to identify the emotions in modalities, e.g., text, audio, image and video, which is a significant development direction for realizing machine intelligence. However, many data in MERC naturally exhibit an imbalanced distribution of emotion categories, and researchers ignore the negative impact of imbalanced data on emotion recognition. To tackle this problem, we systematically analyze it from three aspects: data augmentation, loss sensitivity, and sampling strategy, and propose the Class Boundary Enhanced Representation Learning (CBERL) model. Concretely, we first design a multimodal generative adversarial network to address the imbalanced distribution of {emotion} categories in raw data. Secondly, a deep joint variational autoencoder is proposed to fuse complementary semantic information across modalities and obtain discriminative feature representations. Finally, we implement a multi-task graph neural network with mask reconstruction and classification optimization to solve the problem of overfitting and underfitting in class boundary learning, and achieve cross-modal emotion recognition. We have conducted extensive experiments on the IEMOCAP and MELD benchmark datasets, and the results show that CBERL has achieved a certain performance improvement in the effectiveness of emotion recognition. Especially on the minority class fear and disgust emotion labels, our model improves the accuracy and F1 value by 10% to 20%.
Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes
Abstract
Pre-trained large language models (LLMs) require fine-tuning to improve their responsiveness to natural language instructions. Federated learning (FL) offers a way to perform fine-tuning using the abundant data on end devices without compromising data privacy. Most existing federated fine-tuning methods for LLMs rely on parameter-efficient fine-tuning techniques, which may not reach the performance heights possible with full-parameter tuning. However, the communication overhead associated with full-parameter tuning is prohibitively high for both servers and clients. This work introduces FedKSeed, a novel approach that employs zeroth-order optimization (ZOO) with a set of random seeds. It enables federated full-parameter tuning of billion-sized LLMs directly on devices. Our method significantly reduces transmission requirements between the server and clients to just a few scalar gradients and random seeds, amounting to only a few thousand bytes. Building on this, we develop a strategy to assess the significance of ZOO perturbations for FL, allowing for probability-differentiated seed sampling. This prioritizes perturbations that have a greater impact on model accuracy. Experiments across six scenarios with different LLMs, datasets and data partitions demonstrate that our approach outperforms existing federated LLM fine-tuning methods in terms of both communication efficiency and new task generalization.
FOSS: A Self-Learned Doctor for Query Optimizer
Authors: Authors: Kai Zhong, Luming Sun, Tao Ji, Cuiping Li, Hong Chen
Abstract
Various works have utilized deep reinforcement learning (DRL) to address the query optimization problem in database system. They either learn to construct plans from scratch in a bottom-up manner or guide the plan generation behavior of traditional optimizer using hints. While these methods have achieved some success, they face challenges in either low training efficiency or limited plan search space. To address these challenges, we introduce FOSS, a novel DRL-based framework for query optimization. FOSS initiates optimization from the original plan generated by a traditional optimizer and incrementally refines suboptimal nodes of the plan through a sequence of actions. Additionally, we devise an asymmetric advantage model to evaluate the advantage between two plans. We integrate it with a traditional optimizer to form a simulated environment. Leveraging this simulated environment, FOSS can bootstrap itself to rapidly generate a large amount of high-quality simulated experiences. FOSS then learns and improves its optimization capability from these simulated experiences. We evaluate the performance of FOSS on Join Order Benchmark, TPC-DS, and Stack Overflow. The experimental results demonstrate that FOSS outperforms the state-of-the-art methods in terms of latency performance and optimization time. Compared to PostgreSQL, FOSS achieves savings ranging from 15% to 83% in total latency across different benchmarks.
Intraoperative 2D/3D Image Registration via Differentiable X-ray Rendering
Abstract
Surgical decisions are informed by aligning rapid portable 2D intraoperative images (e.g., X-rays) to a high-fidelity 3D preoperative reference scan (e.g., CT). 2D/3D image registration often fails in practice: conventional optimization methods are prohibitively slow and susceptible to local minima, while neural networks trained on small datasets fail on new patients or require impractical landmark supervision. We present DiffPose, a self-supervised approach that leverages patient-specific simulation and differentiable physics-based rendering to achieve accurate 2D/3D registration without relying on manually labeled data. Preoperatively, a CNN is trained to regress the pose of a randomly oriented synthetic X-ray rendered from the preoperative CT. The CNN then initializes rapid intraoperative test-time optimization that uses the differentiable X-ray renderer to refine the solution. Our work further proposes several geometrically principled methods for sampling camera poses from $\mathbf{SE}(3)$, for sparse differentiable rendering, and for driving registration in the tangent space $\mathfrak{se}(3)$ with geodesic and multiscale locality-sensitive losses. DiffPose achieves sub-millimeter accuracy across surgical datasets at intraoperative speeds, improving upon existing unsupervised methods by an order of magnitude and even outperforming supervised baselines. Our code is available at https://github.com/eigenvivek/DiffPose.
NVFi: Neural Velocity Fields for 3D Physics Learning from Dynamic Videos
Abstract
In this paper, we aim to model 3D scene dynamics from multi-view videos. Unlike the majority of existing works which usually focus on the common task of novel view synthesis within the training time period, we propose to simultaneously learn the geometry, appearance, and physical velocity of 3D scenes only from video frames, such that multiple desirable applications can be supported, including future frame extrapolation, unsupervised 3D semantic scene decomposition, and dynamic motion transfer. Our method consists of three major components, 1) the keyframe dynamic radiance field, 2) the interframe velocity field, and 3) a joint keyframe and interframe optimization module which is the core of our framework to effectively train both networks. To validate our method, we further introduce two dynamic 3D datasets: 1) Dynamic Object dataset, and 2) Dynamic Indoor Scene dataset. We conduct extensive experiments on multiple datasets, demonstrating the superior performance of our method over all baselines, particularly in the critical tasks of future frame extrapolation and unsupervised 3D semantic scene decomposition.
Compound Text-Guided Prompt Tuning via Image-Adaptive Cues
Authors: Authors: Hao Tan, Jun Li, Yizhuang Zhou, Jun Wan, Zhen Lei, Xiangyu Zhang
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Abstract
Vision-Language Models (VLMs) such as CLIP have demonstrated remarkable generalization capabilities to downstream tasks. However, existing prompt tuning based frameworks need to parallelize learnable textual inputs for all categories, suffering from massive GPU memory consumption when there is a large number of categories in the target dataset. Moreover, previous works require to include category names within prompts, exhibiting subpar performance when dealing with ambiguous category names. To address these shortcomings, we propose Compound Text-Guided Prompt Tuning (TGP-T) that significantly reduces resource demand while achieving superior performance. We introduce text supervision to the optimization of prompts, which enables two benefits: 1) releasing the model reliance on the pre-defined category names during inference, thereby enabling more flexible prompt generation; 2) reducing the number of inputs to the text encoder, which decreases GPU memory consumption significantly. Specifically, we found that compound text supervisions, i.e., category-wise and content-wise, is highly effective, since they provide inter-class separability and capture intra-class variations, respectively. Moreover, we condition the prompt generation on visual features through a module called Bonder, which facilitates the alignment between prompts and visual features. Extensive experiments on few-shot recognition and domain generalization demonstrate that TGP-T achieves superior performance with consistently lower training costs. It reduces GPU memory usage by 93% and attains a 2.5% performance gain on 16-shot ImageNet. The code is available at https://github.com/EricTan7/TGP-T.
DiffVL: Scaling Up Soft Body Manipulation using Vision-Language Driven Differentiable Physics
Authors: Authors: Zhiao Huang, Feng Chen, Yewen Pu, Chunru Lin, Hao Su, Chuang Gan
Abstract
Combining gradient-based trajectory optimization with differentiable physics simulation is an efficient technique for solving soft-body manipulation problems. Using a well-crafted optimization objective, the solver can quickly converge onto a valid trajectory. However, writing the appropriate objective functions requires expert knowledge, making it difficult to collect a large set of naturalistic problems from non-expert users. We introduce DiffVL, a method that enables non-expert users to communicate soft-body manipulation tasks -- a combination of vision and natural language, given in multiple stages -- that can be readily leveraged by a differential physics solver. We have developed GUI tools that enable non-expert users to specify 100 tasks inspired by real-life soft-body manipulations from online videos, which we'll make public. We leverage large language models to translate task descriptions into machine-interpretable optimization objectives. The optimization objectives can help differentiable physics solvers to solve these long-horizon multistage tasks that are challenging for previous baselines.
VisionTraj: A Noise-Robust Trajectory Recovery Framework based on Large-scale Camera Network
Authors: Authors: Zhishuai Li, Ziyue Li, Xiaoru Hu, Guoqing Du, Yunhao Nie, Feng Zhu, Lei Bai, Rui Zhao
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR); Machine Learning (cs.LG)
Abstract
Trajectory recovery based on the snapshots from the city-wide multi-camera network facilitates urban mobility sensing and driveway optimization. The state-of-the-art solutions devoted to such a vision-based scheme typically incorporate predefined rules or unsupervised iterative feedback, struggling with multi-fold challenges such as lack of open-source datasets for training the whole pipeline, and the vulnerability to the noises from visual inputs. In response to the dilemma, this paper proposes VisionTraj, the first learning-based model that reconstructs vehicle trajectories from snapshots recorded by road network cameras. Coupled with it, we elaborate on two rational vision-trajectory datasets, which produce extensive trajectory data along with corresponding visual snapshots, enabling supervised vision-trajectory interplay extraction. Following the data creation, based on the results from the off-the-shelf multi-modal vehicle clustering, we first re-formulate the trajectory recovery problem as a generative task and introduce the canonical Transformer as the autoregressive backbone. Then, to identify clustering noises (e.g., false positives) with the bound on the snapshots' spatiotemporal dependencies, a GCN-based soft-denoising module is conducted based on the fine- and coarse-grained Re-ID clusters. Additionally, we harness strong semantic information extracted from the tracklet to provide detailed insights into the vehicle's entry and exit actions during trajectory recovery. The denoising and tracklet components can also act as plug-and-play modules to boost baselines. Experimental results on the two hand-crafted datasets show that the proposed VisionTraj achieves a maximum +11.5% improvement against the sub-best model.
DreamControl: Control-Based Text-to-3D Generation with 3D Self-Prior
Authors: Authors: Tianyu Huang, Yihan Zeng, Zhilu Zhang, Wan Xu, Hang Xu, Songcen Xu, Rynson W. H. Lau, Wangmeng Zuo
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Abstract
3D generation has raised great attention in recent years. With the success of text-to-image diffusion models, the 2D-lifting technique becomes a promising route to controllable 3D generation. However, these methods tend to present inconsistent geometry, which is also known as the Janus problem. We observe that the problem is caused mainly by two aspects, i.e., viewpoint bias in 2D diffusion models and overfitting of the optimization objective. To address it, we propose a two-stage 2D-lifting framework, namely DreamControl, which optimizes coarse NeRF scenes as 3D self-prior and then generates fine-grained objects with control-based score distillation. Specifically, adaptive viewpoint sampling and boundary integrity metric are proposed to ensure the consistency of generated priors. The priors are then regarded as input conditions to maintain reasonable geometries, in which conditional LoRA and weighted score are further proposed to optimize detailed textures. DreamControl can generate high-quality 3D content in terms of both geometry consistency and texture fidelity. Moreover, our control-based optimization guidance is applicable to more downstream tasks, including user-guided generation and 3D animation. The project page is available at https://github.com/tyhuang0428/DreamControl.
Towards A Flexible Accuracy-Oriented Deep Learning Module Inference Latency Prediction Framework for Adaptive Optimization Algorithms
Abstract
With the rapid development of Deep Learning, more and more applications on the cloud and edge tend to utilize large DNN (Deep Neural Network) models for improved task execution efficiency as well as decision-making quality. Due to memory constraints, models are commonly optimized using compression, pruning, and partitioning algorithms to become deployable onto resource-constrained devices. As the conditions in the computational platform change dynamically, the deployed optimization algorithms should accordingly adapt their solutions. To perform frequent evaluations of these solutions in a timely fashion, RMs (Regression Models) are commonly trained to predict the relevant solution quality metrics, such as the resulted DNN module inference latency, which is the focus of this paper. Existing prediction frameworks specify different RM training workflows, but none of them allow flexible configurations of the input parameters (e.g., batch size, device utilization rate) and of the selected RMs for different modules. In this paper, a deep learning module inference latency prediction framework is proposed, which i) hosts a set of customizable input parameters to train multiple different RMs per DNN module (e.g., convolutional layer) with self-generated datasets, and ii) automatically selects a set of trained RMs leading to the highest possible overall prediction accuracy, while keeping the prediction time / space consumption as low as possible. Furthermore, a new RM, namely MEDN (Multi-task Encoder-Decoder Network), is proposed as an alternative solution. Comprehensive experiment results show that MEDN is fast and lightweight, and capable of achieving the highest overall prediction accuracy and R-squared value. The Time/Space-efficient Auto-selection algorithm also manages to improve the overall accuracy by 2.5% and R-squared by 0.39%, compared to the MEDN single-selection scheme.
Optimal Control of Fluid Dosing in Hemorrhage Resuscitation
Abstract
Fluid resuscitation, also called fluid therapy, is commonly used in treatment of critical care patients. Of the variables inherent to fluid therapy treatment, fluid infusion types are well-studied, but volumetric optimization is not. Automated fluid resuscitation systems employ computational control algorithms along with physiological sensors and actuators to automatically adjust the fluid infusion dosage in hemorrhage scenarios. Most automated hemorrhage resuscitation control algorithms implemented to date are of empiric nature and not effective in the presence of clinical disturbances. This work presents preliminary results of a novel controller based on optimal control approach for automated fluid therapy systems. A receding horizon controller is designed to automatically adjust fluid infusion rate in hemorrhagic situations. The proposed control approach aims to address the inefficiency of current resuscitation control algorithms in finding the accurate fluid dosage adjustments. Performance of the proposed approach is compared against a proportional-integral-derivative (PID) controller for a simulated hemorrhage scenario.
Open Data-Driven Automation of Residential Distribution Grid Modeling with Minimal Data Requirements
Authors: Authors: Moritz Weber, Luc Janecke, Huseyin K. Çakmak, Veit Hagenmeyer
Subjects: Computational Engineering, Finance, and Science (cs.CE)
Abstract
In the present paper, we introduce a new method for the automated generation of residential distribution grid models based on novel building load estimation methods and a two-stage optimization for the generation of the 20 kV and 400 V grid topologies. Using the introduced load estimation methods, various open or proprietary data sources can be utilized to estimate the load of residential buildings. These data sources include available building footprints from OpenStreetMap, 3D building data from OSM Buildings, and the number of electricity meters per address provided by the respective distribution system operator (DSO). For the evaluation of the introduced methods, we compare the resulting grid models by utilizing different available data sources for a specific suburban residential area and the real grid topology provided by the DSO. This evaluation yields two key findings: First, the automated 20 kV network generation methodology works well when compared to the real network. Second, the utilization of public 3D building data for load estimation significantly increases the resulting model accuracy compared to 2D data and enables results similar to models based on DSO-supplied meter data. This substantially reduces the dependence on such normally proprietary data.
Robust Graph Neural Network based on Graph Denoising
Authors: Authors: Victor M. Tenorio, Samuel Rey, Antonio G. Marques
Subjects: Machine Learning (cs.LG); Signal Processing (eess.SP)
Abstract
Graph Neural Networks (GNNs) have emerged as a notorious alternative to address learning problems dealing with non-Euclidean datasets. However, although most works assume that the graph is perfectly known, the observed topology is prone to errors stemming from observational noise, graph-learning limitations, or adversarial attacks. If ignored, these perturbations may drastically hinder the performance of GNNs. To address this limitation, this work proposes a robust implementation of GNNs that explicitly accounts for the presence of perturbations in the observed topology. For any task involving GNNs, our core idea is to i) solve an optimization problem not only over the learnable parameters of the GNN but also over the true graph, and ii) augment the fitting cost with a term accounting for discrepancies on the graph. Specifically, we consider a convolutional GNN based on graph filters and follow an alternating optimization approach to handle the (non-differentiable and constrained) optimization problem by combining gradient descent and projected proximal updates. The resulting algorithm is not limited to a particular type of graph and is amenable to incorporating prior information about the perturbations. Finally, we assess the performance of the proposed method through several numerical experiments.
Sherpa3D: Boosting High-Fidelity Text-to-3D Generation via Coarse 3D Prior
Abstract
Recently, 3D content creation from text prompts has demonstrated remarkable progress by utilizing 2D and 3D diffusion models. While 3D diffusion models ensure great multi-view consistency, their ability to generate high-quality and diverse 3D assets is hindered by the limited 3D data. In contrast, 2D diffusion models find a distillation approach that achieves excellent generalization and rich details without any 3D data. However, 2D lifting methods suffer from inherent view-agnostic ambiguity thereby leading to serious multi-face Janus issues, where text prompts fail to provide sufficient guidance to learn coherent 3D results. Instead of retraining a costly viewpoint-aware model, we study how to fully exploit easily accessible coarse 3D knowledge to enhance the prompts and guide 2D lifting optimization for refinement. In this paper, we propose Sherpa3D, a new text-to-3D framework that achieves high-fidelity, generalizability, and geometric consistency simultaneously. Specifically, we design a pair of guiding strategies derived from the coarse 3D prior generated by the 3D diffusion model: a structural guidance for geometric fidelity and a semantic guidance for 3D coherence. Employing the two types of guidance, the 2D diffusion model enriches the 3D content with diversified and high-quality results. Extensive experiments show the superiority of our Sherpa3D over the state-of-the-art text-to-3D methods in terms of quality and 3D consistency.
Learning Naturally Aggregated Appearance for Efficient 3D Editing
Abstract
Neural radiance fields, which represent a 3D scene as a color field and a density field, have demonstrated great progress in novel view synthesis yet are unfavorable for editing due to the implicitness. In view of such a deficiency, we propose to replace the color field with an explicit 2D appearance aggregation, also called canonical image, with which users can easily customize their 3D editing via 2D image processing. To avoid the distortion effect and facilitate convenient editing, we complement the canonical image with a projection field that maps 3D points onto 2D pixels for texture lookup. This field is carefully initialized with a pseudo canonical camera model and optimized with offset regularity to ensure naturalness of the aggregated appearance. Extensive experimental results on three datasets suggest that our representation, dubbed AGAP, well supports various ways of 3D editing (e.g., stylization, interactive drawing, and content extraction) with no need of re-optimization for each case, demonstrating its generalizability and efficiency. Project page is available at https://felixcheng97.github.io/AGAP/.
Keyword: adam
Structured Inverse-Free Natural Gradient: Memory-Efficient & Numerically-Stable KFAC for Large Neural Nets
Authors: Authors: Wu Lin, Felix Dangel, Runa Eschenhagen, Kirill Neklyudov, Agustinus Kristiadi, Richard E. Turner, Alireza Makhzani
Abstract
Second-order methods for deep learning -- such as KFAC -- can be useful for neural net training. However, they are often memory-inefficient and numerically unstable for low-precision training since their preconditioning Kronecker factors are dense, and require high-precision matrix inversion or decomposition. Consequently, such methods are not widely used for training large neural networks such as transformer-based models. We address these two issues by (i) formulating an inverse-free update of KFAC and (ii) imposing structures in each of the Kronecker factors, resulting in a method we term structured inverse-free natural gradient descent (SINGD). On large modern neural networks, we show that, in contrast to KFAC, SINGD is memory efficient and numerically robust, and often outperforms AdamW even in half precision. Hence, our work closes a gap between first-order and second-order methods in modern low precision training for large neural nets.
Achieving the Fundamental Limit of Lossless Analog Compression via Polarization
Authors: Authors: Shuai Yuan, Liuquan Yao, Yuan Li, Huazi Zhang, Jun Wang, Wen Tong, Zhiming Ma
Abstract
In this paper, we study the lossless analog compression for i.i.d. nonsingular signals via the polarization-based framework. We prove that for nonsingular source, the error probability of maximum a posteriori (MAP) estimation polarizes under the Hadamard transform. Building on this insight, we propose partial Hadamard compression and develop the corresponding analog successive cancellation (SC) decoder. The proposed scheme consists of deterministic measurement matrices and non-iterative reconstruction algorithm, providing benefits in both space and computational complexity. Using the polarization of error probability, we prove that our approach achieves the information-theoretical limit for lossless analog compression developed by Wu and Verdu.
Localization Is All You Evaluate: Data Leakage in Online Mapping Datasets and How to Fix It
Authors: Authors: Adam Lilja, Junsheng Fu, Erik Stenborg, Lars Hammarstrand
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Abstract
Data leakage is a critical issue when training and evaluating any method based on supervised learning. The state-of-the-art methods for online mapping are based on supervised learning and are trained predominantly using two datasets: nuScenes and Argoverse 2. These datasets revisit the same geographic locations across training, validation, and test sets. Specifically, over $80$% of nuScenes and $40$% of Argoverse 2 validation and test samples are located less than $5$ m from a training sample. This allows methods to localize within a memorized implicit map during testing and leads to inflated performance numbers being reported. To reveal the true performance in unseen environments, we introduce geographical splits of the data. Experimental results show significantly lower performance numbers, for some methods dropping with more than $45$ mAP, when retraining and reevaluating existing online mapping models with the proposed split. Additionally, a reassessment of prior design choices reveals diverging conclusions from those based on the original split. Notably, the impact of the lifting method and the support from auxiliary tasks (e.g., depth supervision) on performance appears less substantial or follows a different trajectory than previously perceived. Geographical splits can be found https://github.com/LiljaAdam/geographical-splits
Keyword: gradient
Towards On-device Learning on the Edge: Ways to Select Neurons to Update under a Budget Constraint
Abstract
In the realm of efficient on-device learning under extreme memory and computation constraints, a significant gap in successful approaches persists. Although considerable effort has been devoted to efficient inference, the main obstacle to efficient learning is the prohibitive cost of backpropagation. The resources required to compute gradients and update network parameters often exceed the limits of tightly constrained memory budgets. This paper challenges conventional wisdom and proposes a series of experiments that reveal the existence of superior sub-networks. Furthermore, we hint at the potential for substantial gains through a dynamic neuron selection strategy when fine-tuning a target task. Our efforts extend to the adaptation of a recent dynamic neuron selection strategy pioneered by Bragagnolo et al. (NEq), revealing its effectiveness in the most stringent scenarios. Our experiments demonstrate, in the average case, the superiority of a NEq-inspired approach over a random selection. This observation prompts a compelling avenue for further exploration in the area, highlighting the opportunity to design a new class of algorithms designed to facilitate parameter update selection. Our findings usher in a new era of possibilities in the field of on-device learning under extreme constraints and encourage the pursuit of innovative strategies for efficient, resource-friendly model fine-tuning.
Guaranteed Trust Region Optimization via Two-Phase KL Penalization
Abstract
On-policy reinforcement learning (RL) has become a popular framework for solving sequential decision problems due to its computational efficiency and theoretical simplicity. Some on-policy methods guarantee every policy update is constrained to a trust region relative to the prior policy to ensure training stability. These methods often require computationally intensive non-linear optimization or require a particular form of action distribution. In this work, we show that applying KL penalization alone is nearly sufficient to enforce such trust regions. Then, we show that introducing a "fixup" phase is sufficient to guarantee a trust region is enforced on every policy update while adding fewer than 5% additional gradient steps in practice. The resulting algorithm, which we call FixPO, is able to train a variety of policy architectures and action spaces, is easy to implement, and produces results competitive with other trust region methods.
Rethinking materials simulations: Blending direct numerical simulations with neural operators
Authors: Authors: Vivek Oommen, Khemraj Shukla, Saaketh Desai, Remi Dingreville, George Em Karniadakis
Abstract
Direct numerical simulations (DNS) are accurate but computationally expensive for predicting materials evolution across timescales, due to the complexity of the underlying evolution equations, the nature of multiscale spatio-temporal interactions, and the need to reach long-time integration. We develop a new method that blends numerical solvers with neural operators to accelerate such simulations. This methodology is based on the integration of a community numerical solver with a U-Net neural operator, enhanced by a temporal-conditioning mechanism that enables accurate extrapolation and efficient time-to-solution predictions of the dynamics. We demonstrate the effectiveness of this framework on simulations of microstructure evolution during physical vapor deposition modeled via the phase-field method. Such simulations exhibit high spatial gradients due to the co-evolution of different material phases with simultaneous slow and fast materials dynamics. We establish accurate extrapolation of the coupled solver with up to 16.5$\times$ speed-up compared to DNS. This methodology is generalizable to a broad range of evolutionary models, from solid mechanics, to fluid dynamics, geophysics, climate, and more.
FreeFlow: A Comprehensive Understanding on Diffusion Probabilistic Models via Optimal Transport
Authors: Authors: Bowen Sun, Shibao Zheng
Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Probability (math.PR)
Abstract
The blooming diffusion probabilistic models (DPMs) have garnered significant interest due to their impressive performance and the elegant inspiration they draw from physics. While earlier DPMs relied upon the Markovian assumption, recent methods based on differential equations have been rapidly applied to enhance the efficiency and capabilities of these models. However, a theoretical interpretation encapsulating these diverse algorithms is insufficient yet pressingly required to guide further development of DPMs. In response to this need, we present FreeFlow, a framework that provides a thorough explanation of the diffusion formula as time-dependent optimal transport, where the evolutionary pattern of probability density is given by the gradient flows of a functional defined in Wasserstein space. Crucially, our framework necessitates a unified description that not only clarifies the subtle mechanism of DPMs but also indicates the roots of some defects through creative involvement of Lagrangian and Eulerian views to understand the evolution of probability flow. We particularly demonstrate that the core equation of FreeFlow condenses all stochastic and deterministic DPMs into a single case, showcasing the expansibility of our method. Furthermore, the Riemannian geometry employed in our work has the potential to bridge broader subjects in mathematics, which enable the involvement of more profound tools for the establishment of more outstanding and generalized models in the future.
Multi-dimensional Fair Federated Learning
Authors: Authors: Cong Su, Guoxian Yu, Jun Wang, Hui Li, Qingzhong Li, Han Yu
Abstract
Federated learning (FL) has emerged as a promising collaborative and secure paradigm for training a model from decentralized data without compromising privacy. Group fairness and client fairness are two dimensions of fairness that are important for FL. Standard FL can result in disproportionate disadvantages for certain clients, and it still faces the challenge of treating different groups equitably in a population. The problem of privately training fair FL models without compromising the generalization capability of disadvantaged clients remains open. In this paper, we propose a method, called mFairFL, to address this problem and achieve group fairness and client fairness simultaneously. mFairFL leverages differential multipliers to construct an optimization objective for empirical risk minimization with fairness constraints. Before aggregating locally trained models, it first detects conflicts among their gradients, and then iteratively curates the direction and magnitude of gradients to mitigate these conflicts. Theoretical analysis proves mFairFL facilitates the fairness in model development. The experimental evaluations based on three benchmark datasets show significant advantages of mFairFL compared to seven state-of-the-art baselines.
Abstract
The theory of Bayesian learning incorporates the use of Student-t Processes to model heavy-tailed distributions and datasets with outliers. However, despite Student-t Processes having a similar computational complexity as Gaussian Processes, there has been limited emphasis on the sparse representation of this model. This is mainly due to the increased difficulty in modeling and computation compared to previous sparse Gaussian Processes. Our motivation is to address the need for a sparse representation framework that reduces computational complexity, allowing Student-t Processes to be more flexible for real-world datasets. To achieve this, we leverage the conditional distribution of Student-t Processes to introduce sparse inducing points. Bayesian methods and variational inference are then utilized to derive a well-defined lower bound, facilitating more efficient optimization of our model through stochastic gradient descent. We propose two methods for computing the variational lower bound, one utilizing Monte Carlo sampling and the other employing Jensen's inequality to compute the KL regularization term in the loss function. We propose adopting these approaches as viable alternatives to Gaussian processes when the data might contain outliers or exhibit heavy-tailed behavior, and we provide specific recommendations for their applicability. We evaluate the two proposed approaches on various synthetic and real-world datasets from UCI and Kaggle, demonstrating their effectiveness compared to baseline methods in terms of computational complexity and accuracy, as well as their robustness to outliers.
Abstract
Large Language Models (LLM) exhibit zero-shot mathematical reasoning capacity as a behavior emergent with scale, commonly manifesting as chain-of-thoughts (CoT) reasoning. However, multiple empirical findings suggest that this prowess is exclusive to LLMs with exorbitant sizes (beyond 50 billion parameters). Meanwhile, educational neuroscientists suggest that symbolic algebraic manipulation be introduced around the same time as arithmetic word problems to modularize language-to-formulation, symbolic manipulation of the formulation, and endgame arithmetic. In this paper, we start with the hypothesis that much smaller LMs, which are weak at multi-step reasoning, can achieve reasonable arithmetic reasoning if arithmetic word problems are posed as a formalize-then-solve task. In our architecture, which we call SYRELM, the LM serves the role of a translator to map natural language arithmetic questions into a formal language (FL) description. A symbolic solver then evaluates the FL expression to obtain the answer. A small frozen LM, equipped with an efficient low-rank adapter, is capable of generating FL expressions that incorporate natural language descriptions of the arithmetic problem (e.g., variable names and their purposes, formal expressions combining variables, etc.). We adopt policy-gradient reinforcement learning to train the adapted LM, informed by the non-differentiable symbolic solver. This marks a sharp departure from the recent development in tool-augmented LLMs, in which the external tools (e.g., calculator, Web search, etc.) are essentially detached from the learning phase of the LM. SYRELM shows massive improvements (e.g., +30.65 absolute point improvement in accuracy on the SVAMP dataset using GPT-J 6B model) over base LMs, while keeping our testbed easy to diagnose, interpret and within reach of most researchers.
A Hybrid Method of Sentiment Analysis and Machine Learning Algorithm for the U.S. Presidential Election Forecasting
Authors: Authors: Guocheng Feng, Huaiyu Cai, Kaihao Chen, Zhijian Li
Abstract
U.S. Presidential Election forecasting has been a research interest for several decades. Currently, election prediction consists of two main approaches: traditional models that incorporate economic data and poll surveys, and models that leverage Twitter (or X) and other social media platforms due to their increasing popularity in the past decade. However, traditional approaches have predominantly focused on national-level predictions, while social media-based approaches often oversimplify the nuanced differences between online discourse and the broader voting population's political landscape. In this work, we perform a hybrid method of both the machine learning algorithm and the sentiment analysis on the state level with various independent variables including census data, economic indicators, polling averages, and the newly defined average sentiment scores from Twitter. Our prediction for the 2020 U.S. Presidential Election yielded promising results. Most of our models successfully predicted a victory for the Democratic candidate with 96% accuracy using Gradient Boosting Trees and Multi-Layer Perceptron algorithms. This novel prediction framework addresses the limitations of existing U.S. Presidential Election forecasting approaches, particularly in terms of state-level predictions. It provides a valuable foundation for future research in this field and contributes to advancing our understanding of election dynamics.
NiSNN-A: Non-iterative Spiking Neural Networks with Attention with Application to Motor Imagery EEG Classification
Authors: Authors: Chuhan Zhang, Wei Pan, Cosimo Della Santina
Subjects: Neural and Evolutionary Computing (cs.NE); Machine Learning (cs.LG)
Abstract
Motor imagery, an important category in electroencephalogram (EEG) research, often intersects with scenarios demanding low energy consumption, such as portable medical devices and isolated environment operations. Traditional deep learning algorithms, despite their effectiveness, are characterized by significant computational demands accompanied by high energy usage. As an alternative, spiking neural networks (SNNs), inspired by the biological functions of the brain, emerge as a promising energy-efficient solution. However, SNNs typically exhibit lower accuracy than their counterpart convolutional neural networks (CNNs). Although attention mechanisms successfully increase network accuracy by focusing on relevant features, their integration in the SNN framework remains an open question. In this work, we combine the SNN and the attention mechanisms for the EEG classification, aiming to improve precision and reduce energy consumption. To this end, we first propose a Non-iterative Leaky Integrate-and-Fire (LIF) neuron model, overcoming the gradient issues in the traditional SNNs using the Iterative LIF neurons. Then, we introduce the sequence-based attention mechanisms to refine the feature map. We evaluated the proposed Non-iterative SNN with Attention (NiSNN-A) model on OpenBMI, a large-scale motor imagery dataset. Experiment results demonstrate that 1) our model outperforms other SNN models by achieving higher accuracy, 2) our model increases energy efficiency compared to the counterpart CNN models (i.e., by 2.27 times) while maintaining comparable accuracy.
Position control of an acoustic cavitation bubble by reinforcement learning
Authors: Authors: Kálmán Klapcsik, Bálint Gyires-Tóth, Juan Manuel Rosselló, Ferenc Hegedűs
Subjects: Systems and Control (eess.SY); Machine Learning (cs.LG)
Abstract
A control technique is developed via Reinforcement Learning that allows arbitrary controlling of the position of an acoustic cavitation bubble in a dual-frequency standing acoustic wave field. The agent must choose the optimal pressure amplitude values to manipulate the bubble position in the range of $x/\lambda_0\in[0.05, 0.25]$. To train the agent an actor-critic off-policy algorithm (Deep Deterministic Policy Gradient) was used that supports continuous action space, which allows setting the pressure amplitude values continuously within $0$ and $1\, \mathrm{bar}$. A shaped reward function is formulated that minimizes the distance between the bubble and the target position and implicitly encourages the agent to perform the position control within the shortest amount of time. In some cases, the optimal control can be 7 times faster than the solution expected from the linear theory.
Privacy Preserving Multi-Agent Reinforcement Learning in Supply Chains
Abstract
This paper addresses privacy concerns in multi-agent reinforcement learning (MARL), specifically within the context of supply chains where individual strategic data must remain confidential. Organizations within the supply chain are modeled as agents, each seeking to optimize their own objectives while interacting with others. As each organization's strategy is contingent on neighboring strategies, maintaining privacy of state and action-related information is crucial. To tackle this challenge, we propose a game-theoretic, privacy-preserving mechanism, utilizing a secure multi-party computation (MPC) framework in MARL settings. Our major contribution is the successful implementation of a secure MPC framework, SecFloat on EzPC, to solve this problem. However, simply implementing policy gradient methods such as MADDPG operations using SecFloat, while conceptually feasible, would be programmatically intractable. To overcome this hurdle, we devise a novel approach that breaks down the forward and backward pass of the neural network into elementary operations compatible with SecFloat , creating efficient and secure versions of the MADDPG algorithm. Furthermore, we present a learning mechanism that carries out floating point operations in a privacy-preserving manner, an important feature for successful learning in MARL framework. Experiments reveal that there is on average 68.19% less supply chain wastage in 2 PC compared to no data share, while also giving on average 42.27% better average cumulative revenue for each player. This work paves the way for practical, privacy-preserving MARL, promising significant improvements in secure computation within supply chain contexts and broadly.
Structured Inverse-Free Natural Gradient: Memory-Efficient & Numerically-Stable KFAC for Large Neural Nets
Authors: Authors: Wu Lin, Felix Dangel, Runa Eschenhagen, Kirill Neklyudov, Agustinus Kristiadi, Richard E. Turner, Alireza Makhzani
Abstract
Second-order methods for deep learning -- such as KFAC -- can be useful for neural net training. However, they are often memory-inefficient and numerically unstable for low-precision training since their preconditioning Kronecker factors are dense, and require high-precision matrix inversion or decomposition. Consequently, such methods are not widely used for training large neural networks such as transformer-based models. We address these two issues by (i) formulating an inverse-free update of KFAC and (ii) imposing structures in each of the Kronecker factors, resulting in a method we term structured inverse-free natural gradient descent (SINGD). On large modern neural networks, we show that, in contrast to KFAC, SINGD is memory efficient and numerically robust, and often outperforms AdamW even in half precision. Hence, our work closes a gap between first-order and second-order methods in modern low precision training for large neural nets.
Beyond Gradient and Priors in Privacy Attacks: Leveraging Pooler Layer Inputs of Language Models in Federated Learning
Authors: Authors: Jianwei Li, Sheng Liu, Qi Lei
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Cryptography and Security (cs.CR)
Abstract
Federated learning (FL) emphasizes decentralized training by storing data locally and sending only model updates, underlining user privacy. Recently, a line of works on privacy attacks impairs user privacy by extracting sensitive training text from language models in the context of FL. Yet, these attack techniques face distinct hurdles: some work chiefly with limited batch sizes (e.g., batch size of 1), and others are easily detectable. This paper introduces an innovative approach that is challenging to detect, significantly enhancing the recovery rate of text in various batch-size settings. Building on fundamental gradient matching and domain prior knowledge, we enhance the attack by recovering the input of the Pooler layer of language models, which enables us to provide additional supervised signals at the feature level. Unlike gradient data, these signals do not average across sentences and tokens, thereby offering more nuanced and effective insights. We benchmark our method using text classification tasks on datasets such as CoLA, SST-2, and Rotten Tomatoes. Across different batch sizes and models, our approach consistently outperforms previous state-of-the-art results.
On Convergence Rates of Quadratic Transform and WMMSE Methods
Authors: Authors: Kaiming Shen, Ziping Zhao, Yannan Chen, Zepeng Zhang, Hei Victor Cheng
Abstract
Fractional programming (FP) plays an important role in information science because of the Cramer-Rao bound,the Fisher information, and the signal-to-interference-plus-noise ratio (SINR). A state-of-the-art method called the quadratic transform has been extensively used to address the FP problems. This work aims to accelerate the quadratic transform-based iterative optimization via gradient projection and extrapolation. The main contributions of this work are three-fold. First, we relate the quadratic transform to the gradient projection, thereby eliminating the matrix inverse operation from the iterative optimization; our result generalizes the weighted sum-of-rates (WSR) maximization algorithm in [1] to a wide range of FP problems. Second, based on this connection to gradient projection, we incorporate Nesterov's extrapolation strategy [2] into the quadratic transform so as to accelerate the convergence of the iterative optimization. Third, from a minorization-maximization (MM) point of view, we examine the convergence rates of the conventional quadratic transform methods--which include the weighted minimum mean square error (WMMSE) algorithm as a special case--and the proposed accelerated ones. Moreover, we illustrate the practical use of the accelerated quadratic transform in two popular application cases of future wireless networks: (i) integrated sensing and communication (ISAC) and (ii) massive multiple-input multiple-output (MIMO).
Movable-Antenna Array Enabled Multiuser Uplink: A Low-Complexity Gradient Descent for Total Transmit Power Minimization
Abstract
We investigate multiuser uplink communication from multiple single-antenna users to a base station (BS), which is equipped with a movable-antenna (MA) array and adopts zero-forcing receivers to decode multiple signals. We aim to optimize the MAs' positions at the BS, to minimize the total transmit power of all users subject to the minimum rate requirement. After applying transformations, we show that the problem is equivalent to minimizing the sum of each eigenvalue's reciprocal of a matrix, which is a function of all MAs' positions. Subsequently, the projected gradient descent (PGD) method is utilized to find a locally optimal solution. In particular, different from the latest related work, we exploit the eigenvalue decomposition to successfully derive a closed-form gradient for the PGD, which facilitates the practical implementation greatly. We demonstrate by simulations that via careful optimization for all MAs' positions in our proposed design, the total transmit power of all users can be decreased significantly as compared to competitive benchmarks.
SGNet: Structure Guided Network via Gradient-Frequency Awareness for Depth Map Super-Resolution
Authors: Authors: Zhengxu Wang, Zhiqiang Yan, Jian Yang
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Abstract
Depth super-resolution (DSR) aims to restore high-resolution (HR) depth from low-resolution (LR) one, where RGB image is often used to promote this task. Recent image guided DSR approaches mainly focus on spatial domain to rebuild depth structure. However, since the structure of LR depth is usually blurry, only considering spatial domain is not very sufficient to acquire satisfactory results. In this paper, we propose structure guided network (SGNet), a method that pays more attention to gradient and frequency domains, both of which have the inherent ability to capture high-frequency structure. Specifically, we first introduce the gradient calibration module (GCM), which employs the accurate gradient prior of RGB to sharpen the LR depth structure. Then we present the Frequency Awareness Module (FAM) that recursively conducts multiple spectrum differencing blocks (SDB), each of which propagates the precise high-frequency components of RGB into the LR depth. Extensive experimental results on both real and synthetic datasets demonstrate the superiority of our SGNet, reaching the state-of-the-art. Codes and pre-trained models are available at https://github.com/yanzq95/SGNet.
Abstract
Deep neural networks (DNNs) have demonstrated remarkable success in various fields. However, the large number of floating-point operations (FLOPs) in DNNs poses challenges for their deployment in resource-constrained applications, e.g., edge devices. To address the problem, pruning has been introduced to reduce the computational cost in executing DNNs. Previous pruning strategies are based on weight values, gradient values and activation outputs. Different from previous pruning solutions, in this paper, we propose a class-aware pruning technique to compress DNNs, which provides a novel perspective to reduce the computational cost of DNNs. In each iteration, the neural network training is modified to facilitate the class-aware pruning. Afterwards, the importance of filters with respect to the number of classes is evaluated. The filters that are only important for a few number of classes are removed. The neural network is then retrained to compensate for the incurred accuracy loss. The pruning iterations end until no filter can be removed anymore, indicating that the remaining filters are very important for many classes. This pruning technique outperforms previous pruning solutions in terms of accuracy, pruning ratio and the reduction of FLOPs. Experimental results confirm that this class-aware pruning technique can significantly reduce the number of weights and FLOPs, while maintaining a high inference accuracy.
Three-dimensional numerical schemes for the segmentation of the psoas muscle in X-ray computed tomography images
Abstract
The analysis of the psoas muscle in morphological and functional imaging has proved to be an accurate approach to assess sarcopenia, i.e. a systemic loss of skeletal muscle mass and function that may be correlated to multifactorial etiological aspects. The inclusion of sarcopenia assessment into a radiological workflow would need the implementation of computational pipelines for image processing that guarantee segmentation reliability and a significant degree of automation. The present study utilizes three-dimensional numerical schemes for psoas segmentation in low-dose X-ray computed tomography images. Specifically, here we focused on the level set methodology and compared the performances of two standard approaches, a classical evolution model and a three-dimension geodesic model, with the performances of an original first-order modification of this latter one. The results of this analysis show that these gradient-based schemes guarantee reliability with respect to manual segmentation and that the first-order scheme requires a computational burden that is significantly smaller than the one needed by the second-order approach.
FM-G-CAM: A Holistic Approach for Explainable AI in Computer Vision
Authors: Authors: Ravidu Suien Rammuni Silva, Jordan J. Bird
Abstract
Explainability is an aspect of modern AI that is vital for impact and usability in the real world. The main objective of this paper is to emphasise the need to understand the predictions of Computer Vision models, specifically Convolutional Neural Network (CNN) based models. Existing methods of explaining CNN predictions are mostly based on Gradient-weighted Class Activation Maps (Grad-CAM) and solely focus on a single target class. We show that from the point of the target class selection, we make an assumption on the prediction process, hence neglecting a large portion of the predictor CNN model's thinking process. In this paper, we present an exhaustive methodology called Fused Multi-class Gradient-weighted Class Activation Map (FM-G-CAM) that considers multiple top predicted classes, which provides a holistic explanation of the predictor CNN's thinking rationale. We also provide a detailed and comprehensive mathematical and algorithmic description of our method. Furthermore, along with a concise comparison of existing methods, we compare FM-G-CAM with Grad-CAM, highlighting its benefits through real-world practical use cases. Finally, we present an open-source Python library with FM-G-CAM implementation to conveniently generate saliency maps for CNN-based model predictions.
A Practical Survey on Emerging Threats from AI-driven Voice Attacks: How Vulnerable are Commercial Voice Control Systems?
Authors: Authors: Yuanda Wang, Qiben Yan, Nikolay Ivanov, Xun Chen
Subjects: Cryptography and Security (cs.CR); Sound (cs.SD); Audio and Speech Processing (eess.AS)
Abstract
The emergence of Artificial Intelligence (AI)-driven audio attacks has revealed new security vulnerabilities in voice control systems. While researchers have introduced a multitude of attack strategies targeting voice control systems (VCS), the continual advancements of VCS have diminished the impact of many such attacks. Recognizing this dynamic landscape, our study endeavors to comprehensively assess the resilience of commercial voice control systems against a spectrum of malicious audio attacks. Through extensive experimentation, we evaluate six prominent attack techniques across a collection of voice control interfaces and devices. Contrary to prevailing narratives, our results suggest that commercial voice control systems exhibit enhanced resistance to existing threats. Particularly, our research highlights the ineffectiveness of white-box attacks in black-box scenarios. Furthermore, the adversaries encounter substantial obstacles in obtaining precise gradient estimations during query-based interactions with commercial systems, such as Apple Siri and Samsung Bixby. Meanwhile, we find that current defense strategies are not completely immune to advanced attacks. Our findings contribute valuable insights for enhancing defense mechanisms in VCS. Through this survey, we aim to raise awareness within the academic community about the security concerns of VCS and advocate for continued research in this crucial area.
Complex-valued Neural Networks -- Theory and Analysis
Authors: Authors: Rayyan Abdalla
Subjects: Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE); Systems and Control (eess.SY)
Abstract
Complex-valued neural networks (CVNNs) have recently been successful in various pioneering areas which involve wave-typed information and frequency-domain processing. This work addresses different structures and classification of CVNNs. The theory behind complex activation functions, implications related to complex differentiability and special activations for CVNN output layers are presented. The work also discusses CVNN learning and optimization using gradient and non-gradient based algorithms. Complex Backpropagation utilizing complex chain rule is also explained in terms of Wirtinger calculus. Moreover, special modules for building CVNN models, such as complex batch normalization and complex random initialization are also discussed. The work also highlights libraries and software blocks proposed for CVNN implementations and discusses future directions. The objective of this work is to understand the dynamics and most recent developments of CVNNs.
Concrete Subspace Learning based Interference Elimination for Multi-task Model Fusion
Authors: Authors: Anke Tang, Li Shen, Yong Luo, Liang Ding, Han Hu, Bo Du, Dacheng Tao
Abstract
Merging models fine-tuned from a common, extensively pre-trained large model but specialized for different tasks has been demonstrated as a cheap and scalable strategy to construct a multi-task model that performs well across diverse tasks. Recent research, exemplified by task arithmetic, highlights that this multi-task model can be derived through arithmetic operations on task vectors. Nevertheless, current merging techniques frequently resolve potential conflicts among parameters from task-specific models by evaluating individual attributes, such as the parameters' magnitude or sign, overlooking their collective impact on the overall functionality of the model. In this work, we propose the CONtinuous relaxation of disCRETE (Concrete) subspace learning method to identify a common low-dimensional subspace and utilize its shared information to track the interference problem without sacrificing much performance. Specifically, we model the problem as a bi-level optimization problem and introduce a meta-learning framework to find the Concrete subspace mask through gradient-based techniques. At the upper level, we focus on learning a shared Concrete mask to identify the subspace, while at the inner level, model merging is performed to maximize the performance of the merged model. We conduct extensive experiments on both vision domain and language domain, and the results demonstrate the effectiveness of our method. The code is available at https://github.com/tanganke/subspace_fusion
CSOT: Curriculum and Structure-Aware Optimal Transport for Learning with Noisy Labels
Authors: Authors: Wanxing Chang, Ye Shi, Jingya Wang
Subjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
Abstract
Learning with noisy labels (LNL) poses a significant challenge in training a well-generalized model while avoiding overfitting to corrupted labels. Recent advances have achieved impressive performance by identifying clean labels and correcting corrupted labels for training. However, the current approaches rely heavily on the model's predictions and evaluate each sample independently without considering either the global and local structure of the sample distribution. These limitations typically result in a suboptimal solution for the identification and correction processes, which eventually leads to models overfitting to incorrect labels. In this paper, we propose a novel optimal transport (OT) formulation, called Curriculum and Structure-aware Optimal Transport (CSOT). CSOT concurrently considers the inter- and intra-distribution structure of the samples to construct a robust denoising and relabeling allocator. During the training process, the allocator incrementally assigns reliable labels to a fraction of the samples with the highest confidence. These labels have both global discriminability and local coherence. Notably, CSOT is a new OT formulation with a nonconvex objective function and curriculum constraints, so it is not directly compatible with classical OT solvers. Here, we develop a lightspeed computational method that involves a scaling iteration within a generalized conditional gradient framework to solve CSOT efficiently. Extensive experiments demonstrate the superiority of our method over the current state-of-the-arts in LNL. Code is available at https://github.com/changwxx/CSOT-for-LNL.
Activation Gradient based Poisoned Sample Detection Against Backdoor Attacks
Abstract
This work focuses on defending against the data poisoning based backdoor attacks, which bring in serious security threats to deep neural networks (DNNs). Specifically, given a untrustworthy training dataset, we aim to filter out potential poisoned samples, \ie, poisoned sample detection (PSD). The key solution for this task is to find a discriminative metric between clean and poisoned samples, even though there is no information about the potential poisoned samples (\eg, the attack method, the poisoning ratio). In this work, we develop an innovative detection approach from the perspective of the gradient \wrt activation (\ie, activation gradient direction, AGD) of each sample in the backdoored model trained on the untrustworthy dataset. We present an interesting observation that the circular distribution of AGDs among all samples of the target class is much more dispersed than that of one clean class. Motivated by this observation, we firstly design a novel metric called Cosine similarity Variation towards Basis Transition (CVBT) to measure the circular distribution's dispersion of each class. Then, we design a simple yet effective algorithm with identifying the target class(es) using outlier detection on CVBT scores of all classes, followed by progressively filtering of poisoned samples according to the cosine similarities of AGDs between every potential sample and a few additional clean samples. Extensive experiments under various settings verify that given very few clean samples of each class, the proposed method could filter out most poisoned samples, while avoiding filtering out clean samples, verifying its effectiveness on the PSD task. Codes are available at https://github.com/SCLBD/bdzoo2/blob/dev/detection_pretrain/agpd.py.
Adaptive Annotation Distribution for Weakly Supervised Point Cloud Semantic Segmentation
Authors: Authors: Zhiyi Pan, Nan Zhang, Wei Gao, Shan Liu, Ge Li
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Abstract
Weakly supervised point cloud semantic segmentation has attracted a lot of attention due to its ability to alleviate the heavy reliance on fine-grained annotations of point clouds. However, in practice, sparse annotation usually exhibits a distinct non-uniform distribution in point cloud, which poses challenges for weak supervision. To address these issues, we propose an adaptive annotation distribution method for weakly supervised point cloud semantic segmentation. Specifically, we introduce the probability density function into the gradient sampling approximation analysis and investigate the impact of sparse annotations distributions. Based on our analysis, we propose a label-aware point cloud downsampling strategy to increase the proportion of annotations involved in the training stage. Furthermore, we design the multiplicative dynamic entropy as the gradient calibration function to mitigate the gradient bias caused by non-uniformly distributed sparse annotations and explicitly reduce the epistemic uncertainty. Without any prior restrictions and additional information, our proposed method achieves comprehensive performance improvements at multiple label rates with different annotation distributions on S3DIS, ScanNetV2 and SemanticKITTI.
Flow Symmetrization for Parameterized Constrained Diffeomorphisms
Abstract
Diffeomorphisms play a crucial role while searching for shapes with fixed topological properties, allowing for smooth deformation of template shapes. Several approaches use diffeomorphism for shape search. However, these approaches employ only unconstrained diffeomorphisms. In this work, we develop Flow Symmetrization - a method to represent a parametric family of constrained diffeomorphisms that contain additional symmetry constraints such as periodicity, rotation equivariance, and transflection equivariance. Our representation is differentiable in nature, making it suitable for gradient-based optimization approaches for shape search. As these symmetry constraints naturally arise in tiling classes, our method is ideal for representing tile shapes belonging to any tiling class. To demonstrate the efficacy of our method, we design two frameworks for addressing the challenging problems of Escherization and Density Estimation. The first framework is dedicated to the Escherization problem, where we parameterize tile shapes belonging to different isohedral classes. Given a target shape, the template tile is deformed using gradient-based optimization to resemble the target shape. The second framework focuses on density estimation in identification spaces. By leveraging the inherent link between tiling theory and identification topology, we design constrained diffeomorphisms for the plane that result in unconstrained diffeomorphisms on the identification spaces. Specifically, we perform density estimation on identification spaces such as torus, sphere, Klein bottle, and projective plane. Through results and experiments, we demonstrate that our method obtains impressive results for Escherization on the Euclidean plane and density estimation on non-Euclidean identification spaces.
Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes
Abstract
Pre-trained large language models (LLMs) require fine-tuning to improve their responsiveness to natural language instructions. Federated learning (FL) offers a way to perform fine-tuning using the abundant data on end devices without compromising data privacy. Most existing federated fine-tuning methods for LLMs rely on parameter-efficient fine-tuning techniques, which may not reach the performance heights possible with full-parameter tuning. However, the communication overhead associated with full-parameter tuning is prohibitively high for both servers and clients. This work introduces FedKSeed, a novel approach that employs zeroth-order optimization (ZOO) with a set of random seeds. It enables federated full-parameter tuning of billion-sized LLMs directly on devices. Our method significantly reduces transmission requirements between the server and clients to just a few scalar gradients and random seeds, amounting to only a few thousand bytes. Building on this, we develop a strategy to assess the significance of ZOO perturbations for FL, allowing for probability-differentiated seed sampling. This prioritizes perturbations that have a greater impact on model accuracy. Experiments across six scenarios with different LLMs, datasets and data partitions demonstrate that our approach outperforms existing federated LLM fine-tuning methods in terms of both communication efficiency and new task generalization.
DiffVL: Scaling Up Soft Body Manipulation using Vision-Language Driven Differentiable Physics
Authors: Authors: Zhiao Huang, Feng Chen, Yewen Pu, Chunru Lin, Hao Su, Chuang Gan
Abstract
Combining gradient-based trajectory optimization with differentiable physics simulation is an efficient technique for solving soft-body manipulation problems. Using a well-crafted optimization objective, the solver can quickly converge onto a valid trajectory. However, writing the appropriate objective functions requires expert knowledge, making it difficult to collect a large set of naturalistic problems from non-expert users. We introduce DiffVL, a method that enables non-expert users to communicate soft-body manipulation tasks -- a combination of vision and natural language, given in multiple stages -- that can be readily leveraged by a differential physics solver. We have developed GUI tools that enable non-expert users to specify 100 tasks inspired by real-life soft-body manipulations from online videos, which we'll make public. We leverage large language models to translate task descriptions into machine-interpretable optimization objectives. The optimization objectives can help differentiable physics solvers to solve these long-horizon multistage tasks that are challenging for previous baselines.
A New Matrix Truncation Method for Improving Approximate Factorisation Preconditioners
Authors: Authors: Andreas A. Bock, Martin S. Andersen
Abstract
We present a general framework for preconditioning Hermitian positive definite linear systems based on the Bregman log determinant divergence. This divergence provides a measure of discrepancy between a preconditioner and a target matrix. Given an approximate factorisation of a target matrix, the proposed framework tells us how to construct a low-rank approximation of the typically indefinite factorisation error. The resulting preconditioner is therefore a sum of a Hermitian positive definite matrix given by an approximate factorisation plus a low-rank matrix. Notably, the low-rank term is not generally obtained as a truncated singular value decomposition. This framework leads to a new truncation where principal directions are not based on the magnitude of the singular values. We describe a procedure for determining these \emph{Bregman directions} and prove that preconditioners constructed in this way are minimisers of the aforementioned divergence. Finally, we demonstrate using several numerical examples how the proposed preconditioner performs in terms of convergence of the preconditioned conjugate gradient method (PCG). For the examples we consider, an incomplete Cholesky preconditioner can be greatly improved in this way, and in some cases only a modest low-rank compensation term is required to obtain a considerable improvement in convergence. We also consider matrices arising from interior point methods for linear programming that do not admit such an incomplete factorisation by default, and present a robust incomplete Cholesky preconditioner based on the proposed methodology. The results highlight that the choice of truncation is critical for ill-conditioned matrices. We show numerous examples where PCG converges to a small tolerance by using the proposed preconditioner, whereas PCG with a SVD-based preconditioner fails to do so.
Ensemble Kalman Inversion for Image Guided Guide Wire Navigation in Vascular Systems
Authors: Authors: Matei Hanu, Jürgen Hesser, Guido Kanschat, Javier Moviglia, Claudia Schillings, Jan Stallkamp
Abstract
This paper addresses the challenging task of guide wire navigation in cardiovascular interventions, focusing on the parameter estimation of a guide wire system using Ensemble Kalman Inversion (EKI) with a subsampling technique. The EKI uses an ensemble of particles to estimate the unknown quantities. However since the data misfit has to be computed for each particle in each iteration, the EKI may become computationally infeasible in the case of high-dimensional data, e.g. high-resolution images. This issue can been addressed by randomised algorithms that utilize only a random subset of the data in each iteration. We introduce and analyse a subsampling technique for the EKI, which is based on a continuous-time representation of stochastic gradient methods and apply it to on the parameter estimation of our guide wire system. Numerical experiments with real data from a simplified test setting demonstrate the potential of the method.
Transformers Implement Functional Gradient Descent to Learn Non-Linear Functions In Context
Abstract
Many neural network architectures have been shown to be Turing Complete, and can thus implement arbitrary algorithms. However, Transformers are unique in that they can implement gradient-based learning algorithms \emph{under simple parameter configurations}. A line of recent work shows that linear Transformers naturally learn to implement gradient descent (GD) when trained on a linear regression in-context learning task. But the linearity assumption (either in the Transformer architecture or in the learning task) is far from realistic settings where non-linear activations crucially enable Transformers to learn complicated non-linear functions. In this paper, we provide theoretical and empirical evidence that non-linear Transformers can, and \emph{in fact do}, learn to implement learning algorithms to learn non-linear functions in context. Our results apply to a broad class of combinations of non-linear architectures, and non-linear in-context learning tasks. Interestingly, we show that the optimal choice of non-linear activation depends in a natural way on the non-linearity of the learning task.
Robust Graph Neural Network based on Graph Denoising
Authors: Authors: Victor M. Tenorio, Samuel Rey, Antonio G. Marques
Subjects: Machine Learning (cs.LG); Signal Processing (eess.SP)
Abstract
Graph Neural Networks (GNNs) have emerged as a notorious alternative to address learning problems dealing with non-Euclidean datasets. However, although most works assume that the graph is perfectly known, the observed topology is prone to errors stemming from observational noise, graph-learning limitations, or adversarial attacks. If ignored, these perturbations may drastically hinder the performance of GNNs. To address this limitation, this work proposes a robust implementation of GNNs that explicitly accounts for the presence of perturbations in the observed topology. For any task involving GNNs, our core idea is to i) solve an optimization problem not only over the learnable parameters of the GNN but also over the true graph, and ii) augment the fitting cost with a term accounting for discrepancies on the graph. Specifically, we consider a convolutional GNN based on graph filters and follow an alternating optimization approach to handle the (non-differentiable and constrained) optimization problem by combining gradient descent and projected proximal updates. The resulting algorithm is not limited to a particular type of graph and is amenable to incorporating prior information about the perturbations. Finally, we assess the performance of the proposed method through several numerical experiments.
Keyword: super-resolution
Iterative Token Evaluation and Refinement for Real-World Super-Resolution
Abstract
Real-world image super-resolution (RWSR) is a long-standing problem as low-quality (LQ) images often have complex and unidentified degradations. Existing methods such as Generative Adversarial Networks (GANs) or continuous diffusion models present their own issues including GANs being difficult to train while continuous diffusion models requiring numerous inference steps. In this paper, we propose an Iterative Token Evaluation and Refinement (ITER) framework for RWSR, which utilizes a discrete diffusion model operating in the discrete token representation space, i.e., indexes of features extracted from a VQGAN codebook pre-trained with high-quality (HQ) images. We show that ITER is easier to train than GANs and more efficient than continuous diffusion models. Specifically, we divide RWSR into two sub-tasks, i.e., distortion removal and texture generation. Distortion removal involves simple HQ token prediction with LQ images, while texture generation uses a discrete diffusion model to iteratively refine the distortion removal output with a token refinement network. In particular, we propose to include a token evaluation network in the discrete diffusion process. It learns to evaluate which tokens are good restorations and helps to improve the iterative refinement results. Moreover, the evaluation network can first check status of the distortion removal output and then adaptively select total refinement steps needed, thereby maintaining a good balance between distortion removal and texture generation. Extensive experimental results show that ITER is easy to train and performs well within just 8 iterative steps. Our codes will be available publicly.
SGNet: Structure Guided Network via Gradient-Frequency Awareness for Depth Map Super-Resolution
Authors: Authors: Zhengxu Wang, Zhiqiang Yan, Jian Yang
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Abstract
Depth super-resolution (DSR) aims to restore high-resolution (HR) depth from low-resolution (LR) one, where RGB image is often used to promote this task. Recent image guided DSR approaches mainly focus on spatial domain to rebuild depth structure. However, since the structure of LR depth is usually blurry, only considering spatial domain is not very sufficient to acquire satisfactory results. In this paper, we propose structure guided network (SGNet), a method that pays more attention to gradient and frequency domains, both of which have the inherent ability to capture high-frequency structure. Specifically, we first introduce the gradient calibration module (GCM), which employs the accurate gradient prior of RGB to sharpen the LR depth structure. Then we present the Frequency Awareness Module (FAM) that recursively conducts multiple spectrum differencing blocks (SDB), each of which propagates the precise high-frequency components of RGB into the LR depth. Extensive experimental results on both real and synthetic datasets demonstrate the superiority of our SGNet, reaching the state-of-the-art. Codes and pre-trained models are available at https://github.com/yanzq95/SGNet.
Transformer-based Selective Super-Resolution for Efficient Image Refinement
Authors: Authors: Tianyi Zhang, Kishore Kasichainula, Yaoxin Zhuo, Baoxin Li, Jae-sun Seo, Yu Cao
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Abstract
Conventional super-resolution methods suffer from two drawbacks: substantial computational cost in upscaling an entire large image, and the introduction of extraneous or potentially detrimental information for downstream computer vision tasks during the refinement of the background. To solve these issues, we propose a novel transformer-based algorithm, Selective Super-Resolution (SSR), which partitions images into non-overlapping tiles, selects tiles of interest at various scales with a pyramid architecture, and exclusively reconstructs these selected tiles with deep features. Experimental results on three datasets demonstrate the efficiency and robust performance of our approach for super-resolution. Compared to the state-of-the-art methods, the FID score is reduced from 26.78 to 10.41 with 40% reduction in computation cost for the BDD100K dataset. The source code is available at https://github.com/destiny301/SSR.
Study of Multiuser Multiple-Antenna Wireless Communications Systems Based on Super-Resolution Arrays
Authors: Authors: S. Pinto, R. C. de Lamare
Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Abstract
This work studies multiple-antenna wireless communication systems based on super-resolution arrays (SRAs). We consider the uplink of a multiple-antenna system in which users communicate with a multiple-antenna base station equipped with SRAs. In particular, we develop linear minimum mean-square error (MMSE) receive filters along with linear and successive interference cancellation receivers for processing signals with the difference co-array originating from the SRAs. We then derive analytical expressions to assess the achievable sum-rates associated with the proposed multiple-antenna systems with SRAs. Simulations show that the proposed multiple-antenna systems with SRAs outperform existing systems with standard arrays that have a larger number of antenna elements.
Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution
Abstract
Text-based diffusion models have exhibited remarkable success in generation and editing, showing great promise for enhancing visual content with their generative prior. However, applying these models to video super-resolution remains challenging due to the high demands for output fidelity and temporal consistency, which is complicated by the inherent randomness in diffusion models. Our study introduces Upscale-A-Video, a text-guided latent diffusion framework for video upscaling. This framework ensures temporal coherence through two key mechanisms: locally, it integrates temporal layers into U-Net and VAE-Decoder, maintaining consistency within short sequences; globally, without training, a flow-guided recurrent latent propagation module is introduced to enhance overall video stability by propagating and fusing latent across the entire sequences. Thanks to the diffusion paradigm, our model also offers greater flexibility by allowing text prompts to guide texture creation and adjustable noise levels to balance restoration and generation, enabling a trade-off between fidelity and quality. Extensive experiments show that Upscale-A-Video surpasses existing methods in both synthetic and real-world benchmarks, as well as in AI-generated videos, showcasing impressive visual realism and temporal consistency.
Photorealistic Video Generation with Diffusion Models
Authors: Authors: Agrim Gupta, Lijun Yu, Kihyuk Sohn, Xiuye Gu, Meera Hahn, Li Fei-Fei, Irfan Essa, Lu Jiang, José Lezama
Abstract
We present W.A.L.T, a transformer-based approach for photorealistic video generation via diffusion modeling. Our approach has two key design decisions. First, we use a causal encoder to jointly compress images and videos within a unified latent space, enabling training and generation across modalities. Second, for memory and training efficiency, we use a window attention architecture tailored for joint spatial and spatiotemporal generative modeling. Taken together these design decisions enable us to achieve state-of-the-art performance on established video (UCF-101 and Kinetics-600) and image (ImageNet) generation benchmarks without using classifier free guidance. Finally, we also train a cascade of three models for the task of text-to-video generation consisting of a base latent video diffusion model, and two video super-resolution diffusion models to generate videos of $512 \times 896$ resolution at $8$ frames per second.
Keyword: sgd
There is no result
Keyword: optimization
Optimizing the Passenger Flow for Airport Security Check
0.1% Data Makes Segment Anything Slim
A Data-Driven Framework for Improving Public EV Charging Infrastructure: Modeling and Forecasting
Guaranteed Trust Region Optimization via Two-Phase KL Penalization
Real-time Scheduling of Building Battery Systems Under Electrical Load Uncertainty Using Dynamic Markov Decision Process
On the calibration of compartmental epidemiological models
Poisoning $\times$ Evasion: Symbiotic Adversarial Robustness for Graph Neural Networks
Flexible Base Station Sleeping and Resource Cooperation Enabled Green Fully-Decoupled RAN
Federated Causality Learning with Explainable Adaptive Optimization
DPoser: Diffusion Model as Robust 3D Human Pose Prior
Multi-dimensional Fair Federated Learning
Long-Term Rate-Fairness-Aware Beamforming Based Massive MIMO Systems
Sparse Variational Student-t Processes
Dynamic Adjustment of Matching Radii under the Broadcasting Mode: A Novel Multitask Learning Strategy and Temporal Modeling Approach
Not All Data Matters: An End-to-End Adaptive Dataset Pruning Framework for Enhancing Model Performance and Efficiency
Spectral methods for Neural Integral Equations
Leveraging Reinforcement Learning and Large Language Models for Code Optimization
Minimum-Time Trajectory Optimization With Data-Based Models: A Linear Programming Approach
On Convergence Rates of Quadratic Transform and WMMSE Methods
A Scalable MARL Solution for Scheduling in Conflict Graphs
IL-NeRF: Incremental Learning for Neural Radiance Fields with Camera Pose Alignment
Movable-Antenna Array Enabled Multiuser Uplink: A Low-Complexity Gradient Descent for Total Transmit Power Minimization
Hacking Task Confounder in Meta-Learning
Autotuning by Changing Directives and Number of Threads in OpenMP using ppOpen-AT
Graph-based Prediction and Planning Policy Network (GP3Net) for scalable self-driving in dynamic environments using Deep Reinforcement Learning
Physics-Aware Multifidelity Bayesian Optimization: a Generalized Formulation
Data-Driven Robust Covariance Control for Uncertain Linear Systems
Composite Survival Analysis: Learning with Auxiliary Aggregated Baselines and Survival Scores
Optimization for the Metaverse over Mobile Edge Computing with Play to Earn
Learning for CasADi: Data-driven Models in Numerical Optimization
Maximum capacity path problem with loss factors
The survival of scientific stylization
Uncertainty Propagation through Trained Deep Neural Networks Using Factor Graphs
Neural Architecture Codesign for Fast Bragg Peak Analysis
Constructing Vec-tionaries to Extract Latent Message Features from Texts: A Case Study of Moral Appeals
From Correspondences to Pose: Non-minimal Certifiably Optimal Relative Pose without Disambiguation
Multiplier Optimization via E-Graph Rewriting
Software issues report for bug fixing process: An empirical study of machine-learning libraries
Complex-valued Neural Networks -- Theory and Analysis
ROSE: A Recognition-Oriented Speech Enhancement Framework in Air Traffic Control Using Multi-Objective Learning
Can LLMs Configure Software Tools
Efficient Estimation of Pairwise Effective Resistance
Pre-Evolved Model for Complex Multi-objective Optimization Problems
Order Matters in the Presence of Dataset Imbalance for Multilingual Learning
Long-Term Carbon-Efficient Planning for Geographically Shiftable Resources: A Monte Carlo Tree Search Approach
Motion Planning for Multiple Mobile Manipulator System in Complex Flipping Manipulation
Concrete Subspace Learning based Interference Elimination for Multi-task Model Fusion
Optimized View and Geometry Distillation from Multi-view Diffuser
Towards Transferable Adversarial Attacks with Centralized Perturbation
Transforms for Multiplicative and Fractional Programming with Broad Applications in Edge Computing and Communication Networks
Offloading and Quality Control for AI Generated Content Services in Edge Computing Networks
Invariant Representation Learning via Decoupling Style and Spurious Features
No Prior Mask: Eliminate Redundant Action for Deep Reinforcement Learning
Security and Reliability Evaluation of Countermeasures implemented using High-Level Synthesis
Flow Symmetrization for Parameterized Constrained Diffeomorphisms
TPRNN: A Top-Down Pyramidal Recurrent Neural Network for Time Series Forecasting
Deep Imbalanced Learning for Multimodal Emotion Recognition in Conversations
Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes
FOSS: A Self-Learned Doctor for Query Optimizer
Intraoperative 2D/3D Image Registration via Differentiable X-ray Rendering
NVFi: Neural Velocity Fields for 3D Physics Learning from Dynamic Videos
Compound Text-Guided Prompt Tuning via Image-Adaptive Cues
DiffVL: Scaling Up Soft Body Manipulation using Vision-Language Driven Differentiable Physics
VisionTraj: A Noise-Robust Trajectory Recovery Framework based on Large-scale Camera Network
DreamControl: Control-Based Text-to-3D Generation with 3D Self-Prior
Towards A Flexible Accuracy-Oriented Deep Learning Module Inference Latency Prediction Framework for Adaptive Optimization Algorithms
Optimal Control of Fluid Dosing in Hemorrhage Resuscitation
Open Data-Driven Automation of Residential Distribution Grid Modeling with Minimal Data Requirements
Robust Graph Neural Network based on Graph Denoising
Sherpa3D: Boosting High-Fidelity Text-to-3D Generation via Coarse 3D Prior
Learning Naturally Aggregated Appearance for Efficient 3D Editing
Keyword: adam
Structured Inverse-Free Natural Gradient: Memory-Efficient & Numerically-Stable KFAC for Large Neural Nets
Achieving the Fundamental Limit of Lossless Analog Compression via Polarization
Localization Is All You Evaluate: Data Leakage in Online Mapping Datasets and How to Fix It
Keyword: gradient
Towards On-device Learning on the Edge: Ways to Select Neurons to Update under a Budget Constraint
Guaranteed Trust Region Optimization via Two-Phase KL Penalization
Rethinking materials simulations: Blending direct numerical simulations with neural operators
FreeFlow: A Comprehensive Understanding on Diffusion Probabilistic Models via Optimal Transport
Multi-dimensional Fair Federated Learning
Sparse Variational Student-t Processes
Frugal LMs Trained to Invoke Symbolic Solvers Achieve Parameter-Efficient Arithmetic Reasoning
A Hybrid Method of Sentiment Analysis and Machine Learning Algorithm for the U.S. Presidential Election Forecasting
NiSNN-A: Non-iterative Spiking Neural Networks with Attention with Application to Motor Imagery EEG Classification
Position control of an acoustic cavitation bubble by reinforcement learning
Privacy Preserving Multi-Agent Reinforcement Learning in Supply Chains
Structured Inverse-Free Natural Gradient: Memory-Efficient & Numerically-Stable KFAC for Large Neural Nets
Beyond Gradient and Priors in Privacy Attacks: Leveraging Pooler Layer Inputs of Language Models in Federated Learning
On Convergence Rates of Quadratic Transform and WMMSE Methods
Movable-Antenna Array Enabled Multiuser Uplink: A Low-Complexity Gradient Descent for Total Transmit Power Minimization
SGNet: Structure Guided Network via Gradient-Frequency Awareness for Depth Map Super-Resolution
Class-Aware Pruning for Efficient Neural Networks
Three-dimensional numerical schemes for the segmentation of the psoas muscle in X-ray computed tomography images
FM-G-CAM: A Holistic Approach for Explainable AI in Computer Vision
A Practical Survey on Emerging Threats from AI-driven Voice Attacks: How Vulnerable are Commercial Voice Control Systems?
Complex-valued Neural Networks -- Theory and Analysis
Concrete Subspace Learning based Interference Elimination for Multi-task Model Fusion
CSOT: Curriculum and Structure-Aware Optimal Transport for Learning with Noisy Labels
Activation Gradient based Poisoned Sample Detection Against Backdoor Attacks
Adaptive Annotation Distribution for Weakly Supervised Point Cloud Semantic Segmentation
Flow Symmetrization for Parameterized Constrained Diffeomorphisms
Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes
DiffVL: Scaling Up Soft Body Manipulation using Vision-Language Driven Differentiable Physics
A New Matrix Truncation Method for Improving Approximate Factorisation Preconditioners
Ensemble Kalman Inversion for Image Guided Guide Wire Navigation in Vascular Systems
Transformers Implement Functional Gradient Descent to Learn Non-Linear Functions In Context
Robust Graph Neural Network based on Graph Denoising
Keyword: super-resolution
Iterative Token Evaluation and Refinement for Real-World Super-Resolution
SGNet: Structure Guided Network via Gradient-Frequency Awareness for Depth Map Super-Resolution
Transformer-based Selective Super-Resolution for Efficient Image Refinement
Study of Multiuser Multiple-Antenna Wireless Communications Systems Based on Super-Resolution Arrays
Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution
Photorealistic Video Generation with Diffusion Models