New submissions for Tue, 4 Apr 23

[zoq]

Keyword: sgd

Doubly Stochastic Models: Learning with Unbiased Label Noises and Inference Stability

• Authors: Authors: Haoyi Xiong, Xuhong Li, Boyang Yu, Zhanxing Zhu, Dongrui Wu, Dejing Dou
• Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2304.00320
• Pdf link: https://arxiv.org/pdf/2304.00320
• Abstract Random label noises (or observational noises) widely exist in practical machine learning settings. While previous studies primarily focus on the affects of label noises to the performance of learning, our work intends to investigate the implicit regularization effects of the label noises, under mini-batch sampling settings of stochastic gradient descent (SGD), with assumptions that label noises are unbiased. Specifically, we analyze the learning dynamics of SGD over the quadratic loss with unbiased label noises, where we model the dynamics of SGD as a stochastic differentiable equation (SDE) with two diffusion terms (namely a Doubly Stochastic Model). While the first diffusion term is caused by mini-batch sampling over the (label-noiseless) loss gradients as many other works on SGD, our model investigates the second noise term of SGD dynamics, which is caused by mini-batch sampling over the label noises, as an implicit regularizer. Our theoretical analysis finds such implicit regularizer would favor some convergence points that could stabilize model outputs against perturbation of parameters (namely inference stability). Though similar phenomenon have been investigated, our work doesn't assume SGD as an Ornstein-Uhlenbeck like process and achieve a more generalizable result with convergence of approximation proved. To validate our analysis, we design two sets of empirical studies to analyze the implicit regularizer of SGD with unbiased random label noises for deep neural networks training and linear regression.

  Fast Convergence of Random Reshuffling under Over-Parameterization and the Polyak-Łojasiewicz Condition

• Authors: Authors: Chen Fan, Christos Thrampoulidis, Mark Schmidt
• Subjects: Machine Learning (cs.LG); Optimization and Control (math.OC)
• Arxiv link: https://arxiv.org/abs/2304.00459
• Pdf link: https://arxiv.org/pdf/2304.00459
• Abstract Modern machine learning models are often over-parameterized and as a result they can interpolate the training data. Under such a scenario, we study the convergence properties of a sampling-without-replacement variant of stochastic gradient descent (SGD) known as random reshuffling (RR). Unlike SGD that samples data with replacement at every iteration, RR chooses a random permutation of data at the beginning of each epoch and each iteration chooses the next sample from the permutation. For under-parameterized models, it has been shown RR can converge faster than SGD under certain assumptions. However, previous works do not show that RR outperforms SGD in over-parameterized settings except in some highly-restrictive scenarios. For the class of Polyak-\L ojasiewicz (PL) functions, we show that RR can outperform SGD in over-parameterized settings when either one of the following holds: (i) the number of samples ($n$) is less than the product of the condition number ($\kappa$) and the parameter ($\alpha$) of a weak growth condition (WGC), or (ii) $n$ is less than the parameter ($\rho$) of a strong growth condition (SGC).

  Keyword: optimization

  Almost Linear Constant-Factor Sketching for $\ell_1$ and Logistic Regression

• Authors: Authors: Alexander Munteanu, Simon Omlor, David Woodruff
• Subjects: Data Structures and Algorithms (cs.DS); Machine Learning (cs.LG)
• Arxiv link: https://arxiv.org/abs/2304.00051
• Pdf link: https://arxiv.org/pdf/2304.00051
• Abstract We improve upon previous oblivious sketching and turnstile streaming results for $\ell_1$ and logistic regression, giving a much smaller sketching dimension achieving $O(1)$-approximation and yielding an efficient optimization problem in the sketch space. Namely, we achieve for any constant $c>0$ a sketching dimension of $\tilde{O}(d^{1+c})$ for $\ell_1$ regression and $\tilde{O}(\mu d^{1+c})$ for logistic regression, where $\mu$ is a standard measure that captures the complexity of compressing the data. For $\ell_1$-regression our sketching dimension is near-linear and improves previous work which either required $\Omega(\log d)$-approximation with this sketching dimension, or required a larger $\operatorname{poly}(d)$ number of rows. Similarly, for logistic regression previous work had worse $\operatorname{poly}(\mu d)$ factors in its sketching dimension. We also give a tradeoff that yields a $1+\varepsilon$ approximation in input sparsity time by increasing the total size to $(d\log(n)/\varepsilon)^{O(1/\varepsilon)}$ for $\ell_1$ and to $(\mu d\log(n)/\varepsilon)^{O(1/\varepsilon)}$ for logistic regression. Finally, we show that our sketch can be extended to approximate a regularized version of logistic regression where the data-dependent regularizer corresponds to the variance of the individual logistic losses.

  A Physics-Informed Machine Learning for Electricity Markets: A NYISO Case Study

• Authors: Authors: Robert Ferrando, Laurent Pagnier, Robert Mieth, Zhirui Liang, Yury Dvorkin, Daniel Bienstock, Michael Chertkov
• Subjects: Machine Learning (cs.LG); Systems and Control (eess.SY); Optimization and Control (math.OC)
• Arxiv link: https://arxiv.org/abs/2304.00062
• Pdf link: https://arxiv.org/pdf/2304.00062
• Abstract This paper addresses the challenge of efficiently solving the optimal power flow problem in real-time electricity markets. The proposed solution, named Physics-Informed Market-Aware Active Set learning OPF (PIMA-AS-OPF), leverages physical constraints and market properties to ensure physical and economic feasibility of market-clearing outcomes. Specifically, PIMA-AS-OPF employs the active set learning technique and expands its capabilities to account for curtailment in load or renewable power generation, which is a common challenge in real-world power systems. The core of PIMA-AS-OPF is a fully-connected neural network that takes the net load and the system topology as input. The outputs of this neural network include active constraints such as saturated generators and transmission lines, as well as non-zero load shedding and wind curtailments. These outputs allow for reducing the original market-clearing optimization to a system of linear equations, which can be solved efficiently and yield both the dispatch decisions and the locational marginal prices (LMPs). The dispatch decisions and LMPs are then tested for their feasibility with respect to the requirements for efficient market-clearing results. The accuracy and scalability of the proposed method is tested on a realistic 1814-bus NYISO system with current and future renewable energy penetration levels.

  Robust Tracking Control for Nonlinear Systems: Performance optimization via extremum seeking

• Authors: Authors: Jiapeng Xu, Ying Tan, Xiang Chen
• Subjects: Systems and Control (eess.SY)
• Arxiv link: https://arxiv.org/abs/2304.00138
• Pdf link: https://arxiv.org/pdf/2304.00138
• Abstract This paper presents a controller design and optimization framework for nonlinear dynamic systems to track a given reference signal in the presence of disturbances when the task is repeated over a finite-time interval. This novel framework mainly consists of two steps. The first step is to design a robust linear quadratic tracking controller based on the existing control structure with a Youla-type filter $\tilde Q$. Secondly, an extra degree of freedom: a parameterization in terms of $\tilde Q$, is added to this design framework. This extra design parameter is tuned iteratively from measured tracking cost function with the given disturbances and modeling uncertainties to achieve the best transient performance. The proposed method is validated with simulation placed on a Furuta inverted pendulum, showing significant tracking performance improvement.

  Data-Driven Output Regulation using Single-Gain Tuning Regulators

• Authors: Authors: Liangjie Chen, John W. Simpson-Porco
• Subjects: Systems and Control (eess.SY)
• Arxiv link: https://arxiv.org/abs/2304.00169
• Pdf link: https://arxiv.org/pdf/2304.00169
• Abstract Current approaches to data-driven control are geared towards optimal performance, and often integrate aspects of machine learning and large-scale convex optimization, leading to complex implementations. In many applications, it may be preferable to sacrifice performance to obtain significantly simpler controller designs. We focus here on the problem of output regulation for linear systems, and revisit the so-called tuning regulator of E. J. Davison as a minimal-order data-driven design for tracking and disturbance rejection. Our proposed modification of the tuning regulator relies only on samples of the open-loop plant frequency response for design, is tuned online by adjusting a single scalar gain, and comes with a guaranteed margin of stability; this provides a faithful extension of tuning procedures for SISO integral controllers to MIMO systems with mixed constant and harmonic disturbances. The results are illustrated via application to a four-tank water control process.

  Practical Conformer: Optimizing size, speed and flops of Conformer for on-Device and cloud ASR

• Authors: Authors: Rami Botros, Anmol Gulati, Tara N. Sainath, Krzysztof Choromanski, Ruoming Pang, Trevor Strohman, Weiran Wang, Jiahui Yu
• Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)
• Arxiv link: https://arxiv.org/abs/2304.00171
• Pdf link: https://arxiv.org/pdf/2304.00171
• Abstract Conformer models maintain a large number of internal states, the vast majority of which are associated with self-attention layers. With limited memory bandwidth, reading these from memory at each inference step can slow down inference. In this paper, we design an optimized conformer that is small enough to meet on-device restrictions and has fast inference on TPUs. We explore various ideas to improve the execution speed, including replacing lower conformer blocks with convolution-only blocks, strategically downsizing the architecture, and utilizing an RNNAttention-Performer. Our optimized conformer can be readily incorporated into a cascaded-encoder setting, allowing a second-pass decoder to operate on its output and improve the accuracy whenever more resources are available. Altogether, we find that these optimizations can reduce latency by a factor of 6.8x, and come at a reasonable trade-off in quality. With the cascaded second-pass, we show that the recognition accuracy is completely recoverable. Thus, our proposed encoder can double as a strong standalone encoder in on device, and as the first part of a high-performance ASR pipeline.

  Subject-driven Text-to-Image Generation via Apprenticeship Learning

• Authors: Authors: Wenhu Chen, Hexiang Hu, Yandong Li, Nataniel Rui, Xuhui Jia, Ming-Wei Chang, William W. Cohen
• Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2304.00186
• Pdf link: https://arxiv.org/pdf/2304.00186
• Abstract Recent text-to-image generation models like DreamBooth have made remarkable progress in generating highly customized images of a target subject, by fine-tuning an ``expert model'' for a given subject from a few examples. However, this process is expensive, since a new expert model must be learned for each subject. In this paper, we present SuTI, a Subject-driven Text-to-Image generator that replaces subject-specific fine tuning with \emph{in-context} learning. Given a few demonstrations of a new subject, SuTI can instantly generate novel renditions of the subject in different scenes, without any subject-specific optimization. SuTI is powered by {\em apprenticeship learning}, where a single apprentice model is learned from data generated by massive amount of subject-specific expert models. Specifically, we mine millions of image clusters from the Internet, each centered around a specific visual subject. We adopt these clusters to train massive amount of expert models specialized on different subjects. The apprentice model SuTI then learns to mimic the behavior of these experts through the proposed apprenticeship learning algorithm. SuTI can generate high-quality and customized subject-specific images 20x faster than optimization-based SoTA methods. On the challenging DreamBench and DreamBench-v2, our human evaluation shows that SuTI can significantly outperform existing approaches like InstructPix2Pix, Textual Inversion, Imagic, Prompt2Prompt, Re-Imagen while performing on par with DreamBooth.

  Leveraging Neo4j and deep learning for traffic congestion simulation & optimization

• Authors: Authors: Shyam Pratap Singh, Arshad Ali Khan, Riad Souissi, Syed Adnan Yusuf
• Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
• Arxiv link: https://arxiv.org/abs/2304.00192
• Pdf link: https://arxiv.org/pdf/2304.00192
• Abstract Traffic congestion has been a major challenge in many urban road networks. Extensive research studies have been conducted to highlight traffic-related congestion and address the issue using data-driven approaches. Currently, most traffic congestion analyses are done using simulation software that offers limited insight due to the limitations in the tools and utilities being used to render various traffic congestion scenarios. All that impacts the formulation of custom business problems which vary from place to place and country to country. By exploiting the power of the knowledge graph, we model a traffic congestion problem into the Neo4j graph and then use the load balancing, optimization algorithm to identify congestion-free road networks. We also show how traffic propagates backward in case of congestion or accident scenarios and its overall impact on other segments of the roads. We also train a sequential RNN-LSTM (Long Short-Term Memory) deep learning model on the real-time traffic data to assess the accuracy of simulation results based on a road-specific congestion. Our results show that graph-based traffic simulation, supplemented by AI ML-based traffic prediction can be more effective in estimating the congestion level in a road network.

  Applications of No-Collision Transportation Maps in Manifold Learning

• Authors: Authors: Elisa Negrini, Levon Nurbekyan
• Subjects: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (stat.ML)
• Arxiv link: https://arxiv.org/abs/2304.00199
• Pdf link: https://arxiv.org/pdf/2304.00199
• Abstract In this work, we investigate applications of no-collision transportation maps introduced in [Nurbekyan et. al., 2020] in manifold learning for image data. Recently, there has been a surge in applying transportation-based distances and features for data representing motion-like or deformation-like phenomena. Indeed, comparing intensities at fixed locations often does not reveal the data structure. No-collision maps and distances developed in [Nurbekyan et. al., 2020] are sensitive to geometric features similar to optimal transportation (OT) maps but much cheaper to compute due to the absence of optimization. In this work, we prove that no-collision distances provide an isometry between translations (respectively dilations) of a single probability measure and the translation (respectively dilation) vectors equipped with a Euclidean distance. Furthermore, we prove that no-collision transportation maps, as well as OT and linearized OT maps, do not in general provide an isometry for rotations. The numerical experiments confirm our theoretical findings and show that no-collision distances achieve similar or better performance on several manifold learning tasks compared to other OT and Euclidean-based methods at a fraction of a computational cost.

  Precoder Design with Matrix Manifold Optimization in Massive MIMO

• Authors: Authors: Rui Sun, Chen Wang, An-An Lu, Xiqi Gao, Xiang-Gen Xia
• Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)
• Arxiv link: https://arxiv.org/abs/2304.00201
• Pdf link: https://arxiv.org/pdf/2304.00201
• Abstract We investigate the weighted sum-rate (WSR) maximization linear precoder design for massive MIMO downlink and propose a unified matrix manifold optimization framework applicable to total power constraint (TPC), per-user power constraint (PUPC) and per-antenna power constraint (PAPC). Particularly, we prove that the precoders under TPC, PUPC and PAPC are on different Riemannian submanifolds, and transform the constrained problems in Euclidean space to unconstrained ones on manifolds. In accordance with this, we derive Riemannian ingredients including orthogonal projection, Riemannian gradient, Riemannian Hessian, retraction and vector transport, which are needed for precoder design in matrix manifold framework. Then, Riemannian design methods using Riemannian steepest descent, Riemannian conjugate gradient and Riemannian trust region are provided to design the WSR-maximization precoders under TPC, PUPC or PAPC. Riemannian methods are free of the inverse of large dimensional matrix, which is of great significance for practice. Complexity analysis and performance simulations demonstrate the advantages of the proposed precoder design.

  Q-DETR: An Efficient Low-Bit Quantized Detection Transformer

• Authors: Authors: Sheng Xu, Yanjing Li, Mingbao Lin, Peng Gao, Guodong Guo, Jinhu Lu, Baochang Zhang
• Subjects: Computer Vision and Pattern Recognition (cs.CV)
• Arxiv link: https://arxiv.org/abs/2304.00253
• Pdf link: https://arxiv.org/pdf/2304.00253
• Abstract The recent detection transformer (DETR) has advanced object detection, but its application on resource-constrained devices requires massive computation and memory resources. Quantization stands out as a solution by representing the network in low-bit parameters and operations. However, there is a significant performance drop when performing low-bit quantized DETR (Q-DETR) with existing quantization methods. We find that the bottlenecks of Q-DETR come from the query information distortion through our empirical analyses. This paper addresses this problem based on a distribution rectification distillation (DRD). We formulate our DRD as a bi-level optimization problem, which can be derived by generalizing the information bottleneck (IB) principle to the learning of Q-DETR. At the inner level, we conduct a distribution alignment for the queries to maximize the self-information entropy. At the upper level, we introduce a new foreground-aware query matching scheme to effectively transfer the teacher information to distillation-desired features to minimize the conditional information entropy. Extensive experimental results show that our method performs much better than prior arts. For example, the 4-bit Q-DETR can theoretically accelerate DETR with ResNet-50 backbone by 6.6x and achieve 39.4% AP, with only 2.6% performance gaps than its real-valued counterpart on the COCO dataset.

  DOAD: Decoupled One Stage Action Detection Network

• Authors: Authors: Shuning Chang, Pichao Wang, Fan Wang, Jiashi Feng, Mike Zheng Show
• Subjects: Computer Vision and Pattern Recognition (cs.CV)
• Arxiv link: https://arxiv.org/abs/2304.00254
• Pdf link: https://arxiv.org/pdf/2304.00254
• Abstract Localizing people and recognizing their actions from videos is a challenging task towards high-level video understanding. Existing methods are mostly two-stage based, with one stage for person bounding box generation and the other stage for action recognition. However, such two-stage methods are generally with low efficiency. We observe that directly unifying detection and action recognition normally suffers from (i) inferior learning due to different desired properties of context representation for detection and action recognition; (ii) optimization difficulty with insufficient training data. In this work, we present a decoupled one-stage network dubbed DOAD, to mitigate above issues and improve the efficiency for spatio-temporal action detection. To achieve it, we decouple detection and action recognition into two branches. Specifically, one branch focuses on detection representation for actor detection, and the other one for action recognition. For the action branch, we design a transformer-based module (TransPC) to model pairwise relationships between people and context. Different from commonly used vector-based dot product in self-attention, it is built upon a novel matrix-based key and value for Hadamard attention to model person-context information. It not only exploits relationships between person pairs but also takes into account context and relative position information. The results on AVA and UCF101-24 datasets show that our method is competitive with two-stage state-of-the-art methods with significant efficiency improvement.

  Gaussian Mechanism Design for Prescribed Privacy Sets in Data Releasing Systems

• Authors: Authors: Teimour Hosseinalizadeh, Nima Monshizadeh
• Subjects: Systems and Control (eess.SY)
• Arxiv link: https://arxiv.org/abs/2304.00260
• Pdf link: https://arxiv.org/pdf/2304.00260
• Abstract The data transmitted by cyber-physical systems can be intercepted and exploited by malicious individuals to infer privacy-sensitive information regarding the physical system. This motivates us to study the problem of preserving privacy in data releasing of linear dynamical system using stochastic perturbation. In this study, the privacy sensitive quantity is the initial state value of the system. For protecting its privacy, we directly design the covariance matrix of a Gaussian output noise to achieve a prescribed uncertainty set in the form of hyper-ellipsoids. This is done by correlated noise and through a convex optimization problem by considering the utility of released signals. Compared to other available methods, our proposed technique for designing the Gaussian output noise provides enhanced flexibility for system designers. As a case study, the results are applied to a heating ventilation and air conditioning system.

  Energy Consumption Optimization in Radio Access Networks (ECO-RAN)

• Authors: Authors: Anders Mariegaard, Kim G. Larsen, Marco Muniz, Thomas Dyhre Nielsen
• Subjects: Networking and Internet Architecture (cs.NI)
• Arxiv link: https://arxiv.org/abs/2304.00277
• Pdf link: https://arxiv.org/pdf/2304.00277
• Abstract In recent years, mobile network operators are showing interest in reducing energy consumption. Toward this goal, in cooperation with the Danish company 2Operate we have developed a stochastic simulation environment for mobile networks. Our simulator interacts with historical data from 2Operate and allow us to turn on and off network cells, replay traffic loads, etc. We have developed an optimization tool which is based on stochastic and distributed controllers computed by \uppaal. We have conducted experiments in our simulation tool. Experiments show that there is a potential to save up to 10\% of energy. We observe that for larger networks, there exists a larger potential for saving energy. Our simulator and \uppaal controllers, have been constructed in accordance to the 2Operate data and infrastructure. However, a main difference is that current equipment do not support updating schedulers on hourly bases. Nevertheless, new equipment e.g. new Huawei equipment do support changing schedulers on hourly basis. Therefore, integrating our solution in the production server of 2Operate is possible. However, rigorous testing in the production system is required.

  Predictive Heterogeneity: Measures and Applications

• Authors: Authors: Jiashuo Liu, Jiayun Wu, Bo Li, Peng Cui
• Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Information Theory (cs.IT)
• Arxiv link: https://arxiv.org/abs/2304.00305
• Pdf link: https://arxiv.org/pdf/2304.00305
• Abstract As an intrinsic and fundamental property of big data, data heterogeneity exists in a variety of real-world applications, such as precision medicine, autonomous driving, financial applications, etc. For machine learning algorithms, the ignorance of data heterogeneity will greatly hurt the generalization performance and the algorithmic fairness, since the prediction mechanisms among different sub-populations are likely to differ from each other. In this work, we focus on the data heterogeneity that affects the prediction of machine learning models, and firstly propose the \emph{usable predictive heterogeneity}, which takes into account the model capacity and computational constraints. We prove that it can be reliably estimated from finite data with probably approximately correct (PAC) bounds. Additionally, we design a bi-level optimization algorithm to explore the usable predictive heterogeneity from data. Empirically, the explored heterogeneity provides insights for sub-population divisions in income prediction, crop yield prediction and image classification tasks, and leveraging such heterogeneity benefits the out-of-distribution generalization performance.

  Convergent iLQR for Safe Trajectory Planning and Control of Legged Robots

• Authors: Authors: James Zhu, J. Joe Payne, Aaron M. Johnson
• Subjects: Robotics (cs.RO)
• Arxiv link: https://arxiv.org/abs/2304.00346
• Pdf link: https://arxiv.org/pdf/2304.00346
• Abstract In order to perform highly dynamic and agile maneuvers, legged robots typically spend time in underactuated domains (e.g. with feet off the ground) where the system has limited command of its acceleration and a constrained amount of time before transitioning to a new domain (e.g. foot touchdown). Meanwhile, these transitions can have instantaneous, unbounded effects on perturbations. These properties make it difficult for local feedback controllers to effectively recover from disturbances as the system evolves through underactuated domains and hybrid impact events. To address this, we utilize the fundamental solution matrix that characterizes the evolution of perturbations through a hybrid trajectory and its 2-norm, which represents the worst-case growth of perturbations. In this paper, the worst-case perturbation analysis is used to explicitly reason about the tracking performance of a hybrid trajectory and is incorporated in an iLQR framework to optimize a trajectory while taking into account the closed-loop convergence of the trajectory under an LQR tracking controller. The generated convergent trajectories are able to recover more effectively from perturbations, are more robust to large disturbances, and use less feedback control effort than trajectories generated with traditional optimization methods.

  Human-Centric Resource Allocation in the Metaverse over Wireless Communications

• Authors: Authors: Jun Zhao, Liangxin Qian, Wenhan Yu
• Subjects: Social and Information Networks (cs.SI); Networking and Internet Architecture (cs.NI)
• Arxiv link: https://arxiv.org/abs/2304.00355
• Pdf link: https://arxiv.org/pdf/2304.00355
• Abstract The Metaverse will provide numerous immersive applications for human users, by consolidating technologies like extended reality (XR), video streaming, and cellular networks. Optimizing wireless communications to enable the human-centric Metaverse is important to satisfy the demands of mobile users. In this paper, we formulate the optimization of the system utility-cost ratio (UCR) for the Metaverse over wireless networks. Our human-centric utility measure for virtual reality (VR) applications of the Metaverse represents users' perceptual assessment of the VR video quality as a function of the data rate and the video resolution, and is learnt from real datasets. The variables jointly optimized in our problem include the allocation of both communication and computation resources as well as VR video resolutions. The system cost in our problem comprises the energy consumption and delay, and is non-convex with respect to the optimization variables due to fractions in the mathematical expressions. To solve the non-convex optimization, we develop a novel fractional programming technique, which contributes to optimization theory and has broad applicability beyond our paper. Our proposed algorithm for the system UCR optimization is computationally efficient and finds a stationary point to the constrained optimization. Through extensive simulations, our algorithm is demonstrated to outperform other approaches.

  HaLP: Hallucinating Latent Positives for Skeleton-based Self-Supervised Learning of Actions

• Authors: Authors: Anshul Shah, Aniket Roy, Ketul Shah, Shlok Kumar Mishra, David Jacobs, Anoop Cherian, Rama Chellappa
• Subjects: Computer Vision and Pattern Recognition (cs.CV)
• Arxiv link: https://arxiv.org/abs/2304.00387
• Pdf link: https://arxiv.org/pdf/2304.00387
• Abstract Supervised learning of skeleton sequence encoders for action recognition has received significant attention in recent times. However, learning such encoders without labels continues to be a challenging problem. While prior works have shown promising results by applying contrastive learning to pose sequences, the quality of the learned representations is often observed to be closely tied to data augmentations that are used to craft the positives. However, augmenting pose sequences is a difficult task as the geometric constraints among the skeleton joints need to be enforced to make the augmentations realistic for that action. In this work, we propose a new contrastive learning approach to train models for skeleton-based action recognition without labels. Our key contribution is a simple module, HaLP - to Hallucinate Latent Positives for contrastive learning. Specifically, HaLP explores the latent space of poses in suitable directions to generate new positives. To this end, we present a novel optimization formulation to solve for the synthetic positives with an explicit control on their hardness. We propose approximations to the objective, making them solvable in closed form with minimal overhead. We show via experiments that using these generated positives within a standard contrastive learning framework leads to consistent improvements across benchmarks such as NTU-60, NTU-120, and PKU-II on tasks like linear evaluation, transfer learning, and kNN evaluation. Our code will be made available at https://github.com/anshulbshah/HaLP.

  GreenScale: Carbon-Aware Systems for Edge Computing

• Authors: Authors: Young Geun Kim, Udit Gupta, Andrew McCrabb, Yonglak Son, Valeria Bertacco, David Brooks, Carole-Jean Wu
• Subjects: Distributed, Parallel, and Cluster Computing (cs.DC); Hardware Architecture (cs.AR)
• Arxiv link: https://arxiv.org/abs/2304.00404
• Pdf link: https://arxiv.org/pdf/2304.00404
• Abstract To improve the environmental implications of the growing demand of computing, future applications need to improve the carbon-efficiency of computing infrastructures. State-of-the-art approaches, however, do not consider the intermittent nature of renewable energy. The time and location-based carbon intensity of energy fueling computing has been ignored when determining how computation is carried out. This poses a new challenge -- deciding when and where to run applications across consumer devices at the edge and servers in the cloud. Such scheduling decisions become more complicated with the stochastic runtime variance and the amortization of the rising embodied emissions. This work proposes GreenScale, a framework to understand the design and optimization space of carbon-aware scheduling for green applications across the edge-cloud infrastructure. Based on the quantified carbon output of the infrastructure components, we demonstrate that optimizing for carbon, compared to performance and energy efficiency, yields unique scheduling solutions. Our evaluation with three representative categories of applications (i.e., AI, Game, and AR/VR) demonstrate that the carbon emissions of the applications can be reduced by up to 29.1% with the GreenScale. The analysis in this work further provides a detailed road map for edge-cloud application developers to build green applications.

  Finding Pareto Efficient Redistricting Plans with Short Bursts

• Authors: Authors: Cory McCartan
• Subjects: Computers and Society (cs.CY); Optimization and Control (math.OC); Applications (stat.AP)
• Arxiv link: https://arxiv.org/abs/2304.00427
• Pdf link: https://arxiv.org/pdf/2304.00427
• Abstract Redistricting practitioners must balance many competing constraints and criteria when drawing district boundaries. To aid in this process, researchers have developed many methods for optimizing districting plans according to one or more criteria. This research note extends a recently-proposed single-criterion optimization method, short bursts (Cannon et al., 2023), to handle the multi-criterion case, and in doing so approximate the Pareto frontier for any set of constraints. We study the empirical performance of the method in a realistic setting and find it behaves as expected and is not very sensitive to algorithmic parameters. The proposed approach, which is implemented in open-source software, should allow researchers and practitioners to better understand the tradeoffs inherent to the redistricting process.

  On the Optimal Recovery of Graph Signals

• Authors: Authors: Simon Foucart, Chunyang Liao, Nate Veldt
• Subjects: Machine Learning (cs.LG); Signal Processing (eess.SP)
• Arxiv link: https://arxiv.org/abs/2304.00474
• Pdf link: https://arxiv.org/pdf/2304.00474
• Abstract Learning a smooth graph signal from partially observed data is a well-studied task in graph-based machine learning. We consider this task from the perspective of optimal recovery, a mathematical framework for learning a function from observational data that adopts a worst-case perspective tied to model assumptions on the function to be learned. Earlier work in the optimal recovery literature has shown that minimizing a regularized objective produces optimal solutions for a general class of problems, but did not fully identify the regularization parameter. Our main contribution provides a way to compute regularization parameters that are optimal or near-optimal (depending on the setting), specifically for graph signal processing problems. Our results offer a new interpretation for classical optimization techniques in graph-based learning and also come with new insights for hyperparameter selection. We illustrate the potential of our methods in numerical experiments on several semi-synthetic graph signal processing datasets.

  Learning by Grouping: A Multilevel Optimization Framework for Improving Fairness in Classification without Losing Accuracy

• Authors: Authors: Ramtin Hosseini, Li Zhang, Bhanu Garg, Pengtao Xie
• Subjects: Machine Learning (cs.LG); Computers and Society (cs.CY)
• Arxiv link: https://arxiv.org/abs/2304.00486
• Pdf link: https://arxiv.org/pdf/2304.00486
• Abstract The integration of machine learning models in various real-world applications is becoming more prevalent to assist humans in their daily decision-making tasks as a result of recent advancements in this field. However, it has been discovered that there is a tradeoff between the accuracy and fairness of these decision-making tasks. In some cases, these AI systems can be unfair by exhibiting bias or discrimination against certain social groups, which can have severe consequences in real life. Inspired by one of the most well-known human learning skills called grouping, we address this issue by proposing a novel machine learning framework where the ML model learns to group a diverse set of problems into distinct subgroups to solve each subgroup using its specific sub-model. Our proposed framework involves three stages of learning, which are formulated as a three-level optimization problem: (i) learning to group problems into different subgroups; (ii) learning group-specific sub-models for problem-solving; and (iii) updating group assignments of training examples by minimizing the validation loss. These three learning stages are performed end-to-end in a joint manner using gradient descent. To improve fairness and accuracy, we develop an efficient optimization algorithm to solve this three-level optimization problem. To further reduce the risk of overfitting in small datasets, we incorporate domain adaptation techniques in the second stage of training. We further apply our method to neural architecture search. Extensive experiments on various datasets demonstrate our method's effectiveness and performance improvements in both fairness and accuracy. Our proposed Learning by Grouping can reduce overfitting and achieve state-of-the-art performances with fixed human-designed network architectures and searchable network architectures on various datasets.

  Level set-based shape optimization of deformable structures for manipulating sound propagation

• Authors: Authors: Yuki Noguchi, Takayuki Yamada
• Subjects: Computational Engineering, Finance, and Science (cs.CE)
• Arxiv link: https://arxiv.org/abs/2304.00497
• Pdf link: https://arxiv.org/pdf/2304.00497
• Abstract In this paper, we propose a level set-based shape optimization method for acoustic wave propagation problems with a deformable structure. First, we propose a mathematical model for acoustic wave propagation with a deformed structure based on coordinate transformation and the Eulerian approach. Next, we formulate the shape optimization problem and perform sensitivity analysis based on the shape derivative concept. We then construct an optimization algorithm based on the framework of a level set-based shape and topology optimization method. Finally, we provide two-dimensional optimization examples that demonstrate the effectiveness of our proposed method in providing optimized designs with desired functionality, while considering the structural deformation.

  Distributed Optimization for Quadratic Cost Functions over Large-Scale Networks with Quantized Communication and Finite-Time Convergence

• Authors: Authors: Apostolos I. Rikos, Andreas Grammenos, Evangelia Kalyvianaki, Christoforos N. Hadjicostis, Themistoklis Charalambous, Karl H. Johansson
• Subjects: Systems and Control (eess.SY)
• Arxiv link: https://arxiv.org/abs/2304.00596
• Pdf link: https://arxiv.org/pdf/2304.00596
• Abstract We propose two distributed iterative algorithms that can be used to solve, in finite time, the distributed optimization problem over quadratic local cost functions in large-scale networks. The first algorithm exhibits synchronous operation whereas the second one exhibits asynchronous operation. Both algorithms share salient features. Specifically, the algorithms operate exclusively with quantized values, which means that the information stored, processed and exchanged between neighboring nodes is subject to deterministic uniform quantization. The algorithms rely on event-driven updates in order to reduce energy consumption, communication bandwidth, network congestion, and/or processor usage. Finally, once the algorithms converge, nodes distributively terminate their operation. We prove that our algorithms converge in a finite number of iterations to the exact optimal solution depending on the quantization level, and we present applications of our algorithms to (i) optimal task scheduling for data centers, and (ii) global model aggregation for distributed federated learning. We provide simulations of these applications to illustrate the operation, performance, and advantages of the proposed algorithms. Additionally, it is shown that our proposed algorithms compare favorably to algorithms in the current literature. Quantized communication and asynchronous updates increase the required time to completion, but finite-time operation is maintained.

  Recurrence without Recurrence: Stable Video Landmark Detection with Deep Equilibrium Models

• Authors: Authors: Paul Micaelli, Arash Vahdat, Hongxu Yin, Jan Kautz, Pavlo Molchanov
• Subjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
• Arxiv link: https://arxiv.org/abs/2304.00600
• Pdf link: https://arxiv.org/pdf/2304.00600
• Abstract Cascaded computation, whereby predictions are recurrently refined over several stages, has been a persistent theme throughout the development of landmark detection models. In this work, we show that the recently proposed Deep Equilibrium Model (DEQ) can be naturally adapted to this form of computation. Our Landmark DEQ (LDEQ) achieves state-of-the-art performance on the challenging WFLW facial landmark dataset, reaching $3.92$ NME with fewer parameters and a training memory cost of $\mathcal{O}(1)$ in the number of recurrent modules. Furthermore, we show that DEQs are particularly suited for landmark detection in videos. In this setting, it is typical to train on still images due to the lack of labelled videos. This can lead to a ``flickering'' effect at inference time on video, whereby a model can rapidly oscillate between different plausible solutions across consecutive frames. By rephrasing DEQs as a constrained optimization, we emulate recurrence at inference time, despite not having access to temporal data at training time. This Recurrence without Recurrence (RwR) paradigm helps in reducing landmark flicker, which we demonstrate by introducing a new metric, normalized mean flicker (NMF), and contributing a new facial landmark video dataset (WFLW-V) targeting landmark uncertainty. On the WFLW-V hard subset made up of $500$ videos, our LDEQ with RwR improves the NME and NMF by $10$ and $13\%$ respectively, compared to the strongest previously published model using a hand-tuned conventional filter.

  Partial-View Object View Synthesis via Filtered Inversion

• Authors: Authors: Fan-Yun Sun, Jonathan Tremblay, Valts Blukis, Kevin Lin, Danfei Xu, Boris Ivanovic, Peter Karkus, Stan Birchfield, Dieter Fox, Ruohan Zhang, Yunzhu Li, Jiajun Wu, Marco Pavone, Nick Haber
• Subjects: Computer Vision and Pattern Recognition (cs.CV)
• Arxiv link: https://arxiv.org/abs/2304.00673
• Pdf link: https://arxiv.org/pdf/2304.00673
• Abstract We propose Filtering Inversion (FINV), a learning framework and optimization process that predicts a renderable 3D object representation from one or few partial views. FINV addresses the challenge of synthesizing novel views of objects from partial observations, spanning cases where the object is not entirely in view, is partially occluded, or is only observed from similar views. To achieve this, FINV learns shape priors by training a 3D generative model. At inference, given one or more views of a novel real-world object, FINV first finds a set of latent codes for the object by inverting the generative model from multiple initial seeds. Maintaining the set of latent codes, FINV filters and resamples them after receiving each new observation, akin to particle filtering. The generator is then finetuned for each latent code on the available views in order to adapt to novel objects. We show that FINV successfully synthesizes novel views of real-world objects (e.g., chairs, tables, and cars), even if the generative prior is trained only on synthetic objects. The ability to address the sim-to-real problem allows FINV to be used for object categories without real-world datasets. FINV achieves state-of-the-art performance on multiple real-world datasets, recovers object shape and texture from partial and sparse views, is robust to occlusion, and is able to incrementally improve its representation with more observations.

  Soft-Minimum Barrier Functions for Safety-Critical Control Subject to Actuation Constraints

• Authors: Authors: Pedram Rabiee, Jesse B. Hoagg
• Subjects: Systems and Control (eess.SY)
• Arxiv link: https://arxiv.org/abs/2304.00693
• Pdf link: https://arxiv.org/pdf/2304.00693
• Abstract This paper presents a new control approach for guaranteed safety (remaining in a safe set) subject to actuator constraints (the control is in a convex polytope). The control signals are computed using real-time optimization, including linear and quadratic programs subject to affine constraints, which are shown to be feasible. The control method relies on a new soft-minimum barrier function that is constructed using a finite-time-horizon prediction of the system trajectories under a known backup control. The main result shows that: (i) the control is continuous and satisfies the actuator constraints, and (ii) a subset of the safe set is forward invariant under the control. We also demonstrate this control on numerical simulations of an inverted pendulum and a double-integrator ground robot.

  Guaranteeing the Õ(AGM/OUT) Runtime for Uniform Sampling and OUT Size Estimation over Joins

• Authors: Authors: Kyoungmin Kim, Jaehyun Ha, George Fletcher, Wook-Shin Han
• Subjects: Databases (cs.DB); Computational Complexity (cs.CC)
• Arxiv link: https://arxiv.org/abs/2304.00715
• Pdf link: https://arxiv.org/pdf/2304.00715
• Abstract We propose a new method for estimating the number of answers OUT of a small join query Q in a large database D, and for uniform sampling over joins. Our method is the first to satisfy all the following statements. - Support arbitrary Q, which can be either acyclic or cyclic, and contain binary and non-binary relations. - Guarantee an arbitrary small error with a high probability always in \~O(AGM/OUT) time, where AGM is the AGM bound OUT (an upper bound of OUT), and \~O hides the polylogarithmic factor of input size. We also explain previous join size estimators in a unified framework. All methods including ours rely on certain indexes on relations in D, which take linear time to build offline. Additionally, we extend our method using generalized hypertree decompositions (GHDs) to achieve a lower complexity than \~O(AGM/OUT) when OUT is small, and present optimization techniques for improving estimation efficiency and accuracy.

  Two-scale data-driven design for heat manipulation

• Authors: Authors: Daicong Da, Wei Chen
• Subjects: Computational Engineering, Finance, and Science (cs.CE)
• Arxiv link: https://arxiv.org/abs/2304.00747
• Pdf link: https://arxiv.org/pdf/2304.00747
• Abstract Data-driven methods have gained increasing attention in computational mechanics and design. This study investigates a two-scale data-driven design for thermal metamaterials with various functionalities. To address the complexity of multiscale design, the design variables are chosen as the components of the homogenized thermal conductivity matrix originating from the lower scale unit cells. Multiple macroscopic functionalities including thermal cloak, thermal concentrator, thermal rotator/inverter, and their combinations, are achieved using the developed approach. Sensitivity analysis is performed to determine the effect of each design variable on the desired functionalities, which is then incorporated into topology optimization. Geometric extraction demonstrates an excellent matching between the optimized homogenized conductivity and the extraction from the constructed database containing both architecture and property information. The designed heterostructures exhibit multiple thermal meta-functionalities that can be applied to a wide range of heat transfer fields from personal computers to aerospace engineering.

  Small but Mighty: Enhancing 3D Point Clouds Semantic Segmentation with U-Next Framework

• Authors: Authors: Ziyin Zeng, Qingyong Hu, Zhong Xie, Jian Zhou, Yongyang Xu
• Subjects: Computer Vision and Pattern Recognition (cs.CV)
• Arxiv link: https://arxiv.org/abs/2304.00749
• Pdf link: https://arxiv.org/pdf/2304.00749
• Abstract We study the problem of semantic segmentation of large-scale 3D point clouds. In recent years, significant research efforts have been directed toward local feature aggregation, improved loss functions and sampling strategies. While the fundamental framework of point cloud semantic segmentation has been largely overlooked, with most existing approaches rely on the U-Net architecture by default. In this paper, we propose U-Next, a small but mighty framework designed for point cloud semantic segmentation. The key to this framework is to learn multi-scale hierarchical representations from semantically similar feature maps. Specifically, we build our U-Next by stacking multiple U-Net $L^1$ codecs in a nested and densely arranged manner to minimize the semantic gap, while simultaneously fusing the feature maps across scales to effectively recover the fine-grained details. We also devised a multi-level deep supervision mechanism to further smooth gradient propagation and facilitate network optimization. Extensive experiments conducted on three large-scale benchmarks including S3DIS, Toronto3D, and SensatUrban demonstrate the superiority and the effectiveness of the proposed U-Next architecture. Our U-Next architecture shows consistent and visible performance improvements across different tasks and baseline models, indicating its great potential to serve as a general framework for future research.

  Robust Optimal Control for Nonlinear Systems with Parametric Uncertainties via System Level Synthesis

• Authors: Authors: Antoine P. Leeman, Jerome Sieber, Samir Bennani, Melanie N. Zeilinger
• Subjects: Systems and Control (eess.SY); Optimization and Control (math.OC)
• Arxiv link: https://arxiv.org/abs/2304.00752
• Pdf link: https://arxiv.org/pdf/2304.00752
• Abstract This paper addresses the problem of optimally controlling nonlinear systems with norm-bounded disturbances and parametric uncertainties while robustly satisfying constraints. The proposed approach jointly optimizes a nominal trajectory of the nonlinear system and an error feedback, requiring minimal offline design effort and offering low conservatism. This is achieved by reformulating the uncertain nonlinear system as an uncertain linear time-varying system accounting for linearization errors, additive disturbances, and parametric uncertainties. This decomposition enables the application of established tools from system level synthesis to convexly over-bound all the uncertainties online, leading to a tractable nonlinear optimization problem. With this novel controller parameterization, we can formulate a convex constraint to ensure robust performance guarantees for the nonlinear system. The presented method is relevant for numerous applications related to trajectory optimization, e.g., in robotics and aerospace engineering. We demonstrate the performance of the approach and its low conservatism through the simulation example of a post-capture satellite stabilization.

  Effective and Stable Role-Based Multi-Agent Collaboration by Structural Information Principles

• Authors: Authors: Xianghua Zeng, Hao Peng, Angsheng Li
• Subjects: Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2304.00755
• Pdf link: https://arxiv.org/pdf/2304.00755
• Abstract Role-based learning is a promising approach to improving the performance of Multi-Agent Reinforcement Learning (MARL). Nevertheless, without manual assistance, current role-based methods cannot guarantee stably discovering a set of roles to effectively decompose a complex task, as they assume either a predefined role structure or practical experience for selecting hyperparameters. In this article, we propose a mathematical Structural Information principles-based Role Discovery method, namely SIRD, and then present a SIRD optimizing MARL framework, namely SR-MARL, for multi-agent collaboration. The SIRD transforms role discovery into a hierarchical action space clustering. Specifically, the SIRD consists of structuralization, sparsification, and optimization modules, where an optimal encoding tree is generated to perform abstracting to discover roles. The SIRD is agnostic to specific MARL algorithms and flexibly integrated with various value function factorization approaches. Empirical evaluations on the StarCraft II micromanagement benchmark demonstrate that, compared with state-of-the-art MARL algorithms, the SR-MARL framework improves the average test win rate by 0.17%, 6.08%, and 3.24%, and reduces the deviation by 16.67%, 30.80%, and 66.30%, under easy, hard, and super hard scenarios.

  FedIN: Federated Intermediate Layers Learning for Model Heterogeneity

• Authors: Authors: Chan Yun-Hin, Jiang Zhihan, Deng Jing, Ngai C.-H. Edith
• Subjects: Machine Learning (cs.LG)
• Arxiv link: https://arxiv.org/abs/2304.00759
• Pdf link: https://arxiv.org/pdf/2304.00759
• Abstract Federated learning (FL) facilitates edge devices to cooperatively train a global shared model while maintaining the training data locally and privately. However, a common but impractical assumption in FL is that the participating edge devices possess the same required resources and share identical global model architecture. In this study, we propose a novel FL method called Federated Intermediate Layers Learning (FedIN), supporting heterogeneous models without utilizing any public dataset. The training models in FedIN are divided into three parts, including an extractor, the intermediate layers, and a classifier. The model architectures of the extractor and classifier are the same in all devices to maintain the consistency of the intermediate layer features, while the architectures of the intermediate layers can vary for heterogeneous devices according to their resource capacities. To exploit the knowledge from features, we propose IN training, training the intermediate layers in line with the features from other clients. Additionally, we formulate and solve a convex optimization problem to mitigate the gradient divergence problem induced by the conflicts between the IN training and the local training. The experiment results show that FedIN achieves the best performance in the heterogeneous model environment compared with the state-of-the-art algorithms. Furthermore, our ablation study demonstrates the effectiveness of IN training and the solution to the convex optimization problem.

  Disorder-invariant Implicit Neural Representation

• Authors: Authors: Hao Zhu, Shaowen Xie, Zhen Liu, Fengyi Liu, Qi Zhang, You Zhou, Yi Lin, Zhan Ma, Xun Cao
• Subjects: Computer Vision and Pattern Recognition (cs.CV); Signal Processing (eess.SP)
• Arxiv link: https://arxiv.org/abs/2304.00837
• Pdf link: https://arxiv.org/pdf/2304.00837
• Abstract Implicit neural representation (INR) characterizes the attributes of a signal as a function of corresponding coordinates which emerges as a sharp weapon for solving inverse problems. However, the expressive power of INR is limited by the spectral bias in the network training. In this paper, we find that such a frequency-related problem could be greatly solved by re-arranging the coordinates of the input signal, for which we propose the disorder-invariant implicit neural representation (DINER) by augmenting a hash-table to a traditional INR backbone. Given discrete signals sharing the same histogram of attributes and different arrangement orders, the hash-table could project the coordinates into the same distribution for which the mapped signal can be better modeled using the subsequent INR network, leading to significantly alleviated spectral bias. Furthermore, the expressive power of the DINER is determined by the width of the hash-table. Different width corresponds to different geometrical elements in the attribute space, \textit{e.g.}, 1D curve, 2D curved-plane and 3D curved-volume when the width is set as $1$, $2$ and $3$, respectively. More covered areas of the geometrical elements result in stronger expressive power. Experiments not only reveal the generalization of the DINER for different INR backbones (MLP vs. SIREN) and various tasks (image/video representation, phase retrieval, refractive index recovery, and neural radiance field optimization) but also show the superiority over the state-of-the-art algorithms both in quality and speed. \textit{Project page:} \url{https://ezio77.github.io/DINER-website/}

  A Hierarchical Multi-Vehicle Coordinated Motion Planning Method based on Interactive Spatio-Temporal Corridors

• Authors: Authors: Xiang Zhang, Boyang Wang, Yaomin Lu, Haiou Liu, Jianwei Gong, Huiyan Chen
• Subjects: Robotics (cs.RO)
• Arxiv link: https://arxiv.org/abs/2304.00843
• Pdf link: https://arxiv.org/pdf/2304.00843
• Abstract Multi-vehicle coordinated motion planning has always been challenged to safely and efficiently resolve conflicts under non-holonomic dynamic constraints. Constructing spatial-temporal corridors for multi-vehicle can decouple the high-dimensional conflicts and further reduce the difficulty of obtaining feasible trajectories. Therefore, this paper proposes a novel hierarchical method based on interactive spatio-temporal corridors (ISTCs). In the first layer, based on the initial guidance trajectories, Mixed Integer Quadratic Programming is designed to construct ISTCs capable of resolving conflicts in generic multi-vehicle scenarios. And then in the second layer, Non-Linear Programming is settled to generate in-corridor trajectories that satisfy the vehicle dynamics. By introducing ISTCs, the multi-vehicle coordinated motion planning problem is able to be decoupled into single-vehicle trajectory optimization problems, which greatly decentralizes the computational pressure and has great potential for real-world applications. Besides, the proposed method searches for feasible solutions in the 3-D $(x,y,t)$ configuration space, preserving more possibilities than the traditional velocity-path decoupling method. Simulated experiments in unsignalized intersection and challenging dense scenarios have been conduced to verify the feasibility and adaptability of the proposed framework.

  Bubble Explorer: Fast UAV Exploration in Large-Scale and Cluttered 3D-Environments using Occlusion-Free Spheres

• Authors: Authors: Benxu Tang, Yunfan Ren, Fangcheng Zhu, Rui He, Siqi Liang, Fanze Kong, Fu Zhang
• Subjects: Robotics (cs.RO)
• Arxiv link: https://arxiv.org/abs/2304.00852
• Pdf link: https://arxiv.org/pdf/2304.00852
• Abstract Autonomous exploration is a crucial aspect of robotics that has numerous applications. Most of the existing methods greedily choose goals that maximize immediate reward. This strategy is computationally efficient but insufficient for overall exploration efficiency. In recent years, some state-of-the-art methods are proposed, which generate a global coverage path and significantly improve overall exploration efficiency. However, global optimization produces high computational overhead, leading to low-frequency planner updates and inconsistent planning motion. In this work, we propose a novel method to support fast UAV exploration in large-scale and cluttered 3-D environments. We introduce a computationally low-cost viewpoints generation method using novel occlusion-free spheres. Additionally, we combine greedy strategy with global optimization, which considers both computational and exploration efficiency. We benchmark our method against state-of-the-art methods to showcase its superiority in terms of exploration efficiency and computational time. We conduct various real-world experiments to demonstrate the excellent performance of our method in large-scale and cluttered environments.

  Optimizing data-flow in Binary Neural Networks

• Authors: Authors: L. Vorabbi, D. Maltoni, S. Santi
• Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2304.00952
• Pdf link: https://arxiv.org/pdf/2304.00952
• Abstract Binary Neural Networks (BNNs) can significantly accelerate the inference time of a neural network by replacing its expensive floating-point arithmetic with bitwise operations. Most existing solutions, however, do not fully optimize data flow through the BNN layers, and intermediate conversions from 1 to 16/32 bits often further hinder efficiency. We propose a novel training scheme that can increase data flow and parallelism in the BNN pipeline; specifically, we introduce a clipping block that decreases the data-width from 32 bits to 8. Furthermore, we reduce the internal accumulator size of a binary layer, usually kept using 32-bit to prevent data overflow without losing accuracy. Additionally, we provide an optimization of the Batch Normalization layer that both reduces latency and simplifies deployment. Finally, we present an optimized implementation of the Binary Direct Convolution for ARM instruction sets. Our experiments show a consistent improvement of the inference speed (up to 1.91 and 2.73x compared to two state-of-the-art BNNs frameworks) with no drop in accuracy for at least one full-precision model.

  Kinetostatic Optimization for Kinematic Redundancy Planning of Nimbl'Bot Robot

• Authors: Authors: Angelica Ginnante (Nantes Univ - ECN), Stéphane Caro (LS2N, CNRS, LS2N - équipe RoMas), Enrico Simetti (DIBRIS), François Leborne
• Subjects: Robotics (cs.RO)
• Arxiv link: https://arxiv.org/abs/2304.00995
• Pdf link: https://arxiv.org/pdf/2304.00995
• Abstract Abstract In manufacturing industry, Computer Numerical Control (CNC) machines are often preferred over Industrial Serial Robots (ISR) for machining tasks. Indeed, CNC machines offer high positioning accuracy, which leads to slight dimensional deviation on the final product. However, these machines have a restricted workspace generating limitations in the machining work. Conversely, ISR are typically characterized by a larger workspace. ISR have already shown satisfactory performance in tasks like polishing, grinding and deburring. This paper proposes a kinematic redundant robot composed of a novel two degrees-of-freedom mechanism with a closed-loop kinematic chain. After describing a task priority inverse kinematic control framework used for joint trajectory planning exploiting the robot kinematic redundancy, the paper analyses the kinetostatic performance of this robot depending on the considered control tasks. Moreover, two kinetostatic tasks are introduced and employed to improve the robot performance. Simulation results show how the robot better performs when the optimization tasks are active. ### Adaptive structure evolution and biologically plausible synaptic plasticity for recurrent spiking neural networks
• Authors: Authors: Wenxuan Pan, Feifei Zhao, Yi Zeng, Bing Han
• Subjects: Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2304.01015
• Pdf link: https://arxiv.org/pdf/2304.01015
• Abstract The architecture design and multi-scale learning principles of the human brain that evolved over hundreds of millions of years are crucial to realizing human-like intelligence. Spiking Neural Network (SNN) based Liquid State Machine (LSM) serves as a suitable architecture to study brain-inspired intelligence because of its brain-inspired structure and the potential for integrating multiple biological principles. Existing researches on LSM focus on different certain perspectives, including high-dimensional encoding or optimization of the liquid layer, network architecture search, and application to hardware devices. There is still a lack of in-depth inspiration from the learning and structural evolution mechanism of the brain. Considering these limitations, this paper presents a novel LSM learning model that integrates adaptive structural evolution and multi-scale biological learning rules. For structural evolution, an adaptive evolvable LSM model is developed to optimize the neural architecture design of liquid layer with separation property. For brain-inspired learning of LSM, we propose a dopamine-modulated Bienenstock-Cooper-Munros (DA-BCM) method that incorporates global long-term dopamine regulation and local trace-based BCM synaptic plasticity. Comparative experimental results on different decision-making tasks show that introducing structural evolution of the liquid layer, and the DA-BCM regulation of the liquid layer and the readout layer could improve the decision-making ability of LSM and flexibly adapt to rule reversal. This work is committed to exploring how evolution can help to design more appropriate network architectures and how multi-scale neuroplasticity principles coordinated to enable the optimization and learning of LSMs for relatively complex decision-making tasks.

  Integrated Decision-Making and Control for Urban Autonomous Driving with Traffic Rules Compliance

• Authors: Authors: Haichao Liu, Kai Chen, Yulin Li, Zhenmin Huang, Jianghua Duan, Jun Ma
• Subjects: Robotics (cs.RO); Systems and Control (eess.SY)
• Arxiv link: https://arxiv.org/abs/2304.01041
• Pdf link: https://arxiv.org/pdf/2304.01041
• Abstract In urban driving scenarios, autonomous vehicles are expected to conform to traffic rules covering traffic lights, traversable and non-traversable traffic lines, etc. In this article, we propose an optimization-based integrated decision-making and control scheme for urban autonomous driving. Inherently, to ensure the compliance with traffic rules, an innovative design of potential functions (PFs) is presented to characterize various traffic rules that are commonly encountered in urban driving scenarios, and these PFs are further incorporated as part of the model predictive control (MPC) formulation. In this sense, it circumvents the necessity of typical hand-crafted rule design, and high-level decision-making is attained implicitly along with control as an integrated architecture, facilitating flexible maneuvers with safety guarantees. As demonstrated from a series of simulations in CARLA, it is noteworthy that the proposed framework admits real-time performance and high generalizability.

  Prediction-Based Leader-Follower Rendezvous Model Predictive Control with Robustness to Communication Losses

• Authors: Authors: Dženan Lapandić, Christos K. Verginis, Dimos V. Dimarogonas, Bo Wahlberg
• Subjects: Systems and Control (eess.SY); Robotics (cs.RO)
• Arxiv link: https://arxiv.org/abs/2304.01045
• Pdf link: https://arxiv.org/pdf/2304.01045
• Abstract In this paper we propose a novel distributed model predictive control (DMPC) based algorithm with a trajectory predictor for a scenario of landing of unmanned aerial vehicles (UAVs) on a moving unmanned surface vehicle (USV). The algorithm is executing DMPC with exchange of trajectories between the agents at a sufficient rate. In the case of loss of communication, and given the sensor setup, agents are predicting the trajectories of other agents based on the available measurements and prior information. The predictions are then used as the reference inputs to DMPC. During the landing, the followers are tasked with avoidance of USV-dependent obstacles and inter-agent collisions. In the proposed distributed algorithm, all agents solve their local optimization problem in parallel and we prove the convergence of the proposed algorithm. Finally, the simulation results support the theoretical findings.

  Optimal control of quasi-1D Bose gases in optical box potentials

• Authors: Authors: Andreas Deutschmann-Olek, Katharina Schrom, Nikolaus Würkner, Jörg Schmiedmayer, Sebastian Erne, Andreas Kugi
• Subjects: Systems and Control (eess.SY); Quantum Gases (cond-mat.quant-gas); Quantum Physics (quant-ph)
• Arxiv link: https://arxiv.org/abs/2304.01051
• Pdf link: https://arxiv.org/pdf/2304.01051
• Abstract In this paper, we investigate the manipulation of quasi-1D Bose gases that are trapped in a highly elongated potential by optimal control methods. The effective meanfield dynamics of the gas can be described by a one-dimensional non-polynomial Schr\"odinger equation. We extend the indirect optimal control method for the Gross-Pitaevskii equation by Winckel and Borzi (2008) to obtain necessary optimality conditions for state and energy cost functionals. This approach is then applied to optimally compress a quasi-1D Bose gase in an (optical) box potential, i.e., to find a so-called short-cut to adiabaticity, by solving the optimality conditions numerically. The behavior of the proposed method is finally analyzed and compared to simple direct optimization strategies using reduced basis functions. Simulations results demonstrate the feasibility of the proposed approach.

  Eigen-Factors an Alternating Optimization for Back-end Plane SLAM of 3D Point Clouds

• Authors: Authors: Gonzalo Ferrer, Dmitrii Iarosh, Anastasiia Kornilova
• Subjects: Robotics (cs.RO)
• Arxiv link: https://arxiv.org/abs/2304.01055
• Pdf link: https://arxiv.org/pdf/2304.01055
• Abstract Modern depth sensors can generate a huge number of 3D points in few seconds to be latter processed by Localization and Mapping algorithms. Ideally, these algorithms should handle efficiently large sizes of Point Clouds under the assumption that using more points implies more information available. The Eigen Factors (EF) is a new algorithm that solves SLAM by using planes as the main geometric primitive. To do so, EF exhaustively calculates the error of all points at complexity $O(1)$, thanks to the {\em Summation matrix} $S$ of homogeneous points. The solution of EF is highly efficient: i) the state variables are only the sensor poses -- trajectory, while the plane parameters are estimated previously in closed from and ii) EF alternating optimization uses a Newton-Raphson method by a direct analytical calculation of the gradient and the Hessian, which turns out to be a block diagonal matrix. Since we require to differentiate over eigenvalues and matrix elements, we have developed an intuitive methodology to calculate partial derivatives in the manifold of rigid body transformations $SE(3)$, which could be applied to unrelated problems that require analytical derivatives of certain complexity. We evaluate EF and other state-of-the-art plane SLAM back-end algorithms in a synthetic environment. The evaluation is extended to ICL dataset (RGBD) and LiDAR KITTI dataset. Code is publicly available at https://github.com/prime-slam/EF-plane-SLAM.

  Conformal Prediction Regions for Time Series using Linear Complementarity Programming

• Authors: Authors: Matthew Cleaveland, Insup Lee, George J. Pappas, Lars Lindemann
• Subjects: Systems and Control (eess.SY); Machine Learning (cs.LG)
• Arxiv link: https://arxiv.org/abs/2304.01075
• Pdf link: https://arxiv.org/pdf/2304.01075
• Abstract Conformal prediction is a statistical tool for producing prediction regions of machine learning models that are valid with high probability. However, applying conformal prediction to time series data leads to conservative prediction regions. In fact, to obtain prediction regions over $T$ time steps with confidence $1-\delta$, {previous works require that each individual prediction region is valid} with confidence $1-\delta/T$. We propose an optimization-based method for reducing this conservatism to enable long horizon planning and verification when using learning-enabled time series predictors. Instead of considering prediction errors individually at each time step, we consider a parameterized prediction error over multiple time steps. By optimizing the parameters over an additional dataset, we find prediction regions that are not conservative. We show that this problem can be cast as a mixed integer linear complementarity program (MILCP), which we then relax into a linear complementarity program (LCP). Additionally, we prove that the relaxed LP has the same optimal cost as the original MILCP. Finally, we demonstrate the efficacy of our method on a case study using pedestrian trajectory predictors.

  Evolving Artificial Neural Networks To Imitate Human Behaviour In Shinobi III : Return of the Ninja Master

• Authors: Authors: Maximilien Le Clei
• Subjects: Neural and Evolutionary Computing (cs.NE); Machine Learning (cs.LG)
• Arxiv link: https://arxiv.org/abs/2304.01096
• Pdf link: https://arxiv.org/pdf/2304.01096
• Abstract Our society is increasingly fond of computational tools. This phenomenon has greatly increased over the past decade following, among other factors, the emergence of a new Artificial Intelligence paradigm. Specifically, the coupling of two algorithmic techniques, Deep Neural Networks and Stochastic Gradient Descent, thrusted by an exponentially increasing computing capacity, has and is continuing to become a major asset in many modern technologies. However, as progress takes its course, some still wonder whether other methods could similarly or even more greatly benefit from these various hardware advances. In order to further this study, we delve in this thesis into Evolutionary Algorithms and their application to Dynamic Neural Networks, two techniques which despite enjoying many advantageous properties have yet to find their niche in contemporary Artificial Intelligence. We find that by elaborating new methods while exploiting strong computational resources, it becomes possible to develop strongly performing agents on a variety of benchmarks but also some other agents behaving very similarly to human subjects on the video game Shinobi III : Return of The Ninja Master, typical complex tasks previously out of reach for non-gradient-based optimization.

  Process Channels: A New Layer for Process Enactment Based on Blockchain State Channels

• Authors: Authors: Fabian Stiehle, Ingo Weber
• Subjects: Cryptography and Security (cs.CR); Software Engineering (cs.SE)
• Arxiv link: https://arxiv.org/abs/2304.01107
• Pdf link: https://arxiv.org/pdf/2304.01107
• Abstract For the enactment of inter-organizational processes, blockchain can guarantee the enforcement of process models and the integrity of execution traces. However, existing solutions come with downsides regarding throughput scalability, latency, and suboptimal tradeoffs between confidentiality and transparency. To address these issues, we propose to change the foundation of blockchain-based process enactment: from on-chain smart contracts to state channels, an overlay network on top of a blockchain. State channels allow conducting most transactions off-chain while mostly retaining the core security properties offered by blockchain. Our proposal, process channels, is a model-driven approach to enacting processes on state channels, with the aim to retain the desired blockchain properties while reducing the on-chain footprint as much as possible. We here focus on the principled approach of state channels as a platform, to enable manifold future optimizations in various directions, like latency and confidentiality. We implemented our approach prototypically and evaluate it both qualitatively (w.r.t. assumptions and guarantees) and quantitatively (w.r.t. correctness and gas cost). In short, while the initial deployment effort is higher with state channels, it typically pays off after a few process instances; and as long as the new assumptions hold, so do the guarantees.

  Interpretable Symbolic Regression for Data Science: Analysis of the 2022 Competition

• Authors: Authors: F. O. de Franca, M. Virgolin, M. Kommenda, M. S. Majumder, M. Cranmer, G. Espada, L. Ingelse, A. Fonseca, M. Landajuela, B. Petersen, R. Glatt, N. Mundhenk, C. S. Lee, J. D. Hochhalter, D. L. Randall, P. Kamienny, H. Zhang, G. Dick, A. Simon, B. Burlacu, Jaan Kasak, Meera Machado, Casper Wilstrup, W. G. La Cava
• Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2304.01117
• Pdf link: https://arxiv.org/pdf/2304.01117
• Abstract Symbolic regression searches for analytic expressions that accurately describe studied phenomena. The main attraction of this approach is that it returns an interpretable model that can be insightful to users. Historically, the majority of algorithms for symbolic regression have been based on evolutionary algorithms. However, there has been a recent surge of new proposals that instead utilize approaches such as enumeration algorithms, mixed linear integer programming, neural networks, and Bayesian optimization. In order to assess how well these new approaches behave on a set of common challenges often faced in real-world data, we hosted a competition at the 2022 Genetic and Evolutionary Computation Conference consisting of different synthetic and real-world datasets which were blind to entrants. For the real-world track, we assessed interpretability in a realistic way by using a domain expert to judge the trustworthiness of candidate models.We present an in-depth analysis of the results obtained in this competition, discuss current challenges of symbolic regression algorithms and highlight possible improvements for future competitions.

  Resource Allocation in IRS-aided Optical Wireless Communication Systems

• Authors: Authors: Ahrar N. Hamad, Ahmad Adnan Qidan, Taisir E.H. El-Gorashi, Jaafar M. H. Elmirghani
• Subjects: Systems and Control (eess.SY)
• Arxiv link: https://arxiv.org/abs/2304.01137
• Pdf link: https://arxiv.org/pdf/2304.01137
• Abstract One of the main challenges facing optical wireless communication (OWC) systems is service disconnection in high blockage probability scenarios where users might lose the line of sight (LoS) connection with their corresponding access points (APs). In this work, we study the deployment of passive reflecting surfaces referred to as Intelligent Reflecting Surfaces (IRSs) in indoor visible light communication (VLC) to boost users signal to noise ratio (SNR) and ensure service continuity. We formulate an optimization problem to allocate APs and the mirrors of IRSs to users such that the sum rate is increased. The results show a 35% increase in the sum rate of the IRS-aided OWC system compared to the sum rate achieved by only considering the LoS channel components. The results also shows that the deployment of IRSs improves the sum rate under LoS blockage.

  Bringing Telepresence to Every Desk

• Authors: Authors: Shengze Wang, Ziheng Wang, Ryan Schmelzle, Liujie Zheng, YoungJoong Kwon, Soumyadip Sengupta, Henry Fuchs
• Subjects: Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR)
• Arxiv link: https://arxiv.org/abs/2304.01197
• Pdf link: https://arxiv.org/pdf/2304.01197
• Abstract In this paper, we work to bring telepresence to every desktop. Unlike commercial systems, personal 3D video conferencing systems must render high-quality videos while remaining financially and computationally viable for the average consumer. To this end, we introduce a capturing and rendering system that only requires 4 consumer-grade RGBD cameras and synthesizes high-quality free-viewpoint videos of users as well as their environments. Experimental results show that our system renders high-quality free-viewpoint videos without using object templates or heavy pre-processing. While not real-time, our system is fast and does not require per-video optimizations. Moreover, our system is robust to complex hand gestures and clothing, and it can generalize to new users. This work provides a strong basis for further optimization, and it will help bring telepresence to every desk in the near future. The code and dataset will be made available on our website https://mcmvmc.github.io/PersonalTelepresence/.

  Keyword: adam

  DOAD: Decoupled One Stage Action Detection Network

• Authors: Authors: Shuning Chang, Pichao Wang, Fan Wang, Jiashi Feng, Mike Zheng Show
• Subjects: Computer Vision and Pattern Recognition (cs.CV)
• Arxiv link: https://arxiv.org/abs/2304.00254
• Pdf link: https://arxiv.org/pdf/2304.00254
• Abstract Localizing people and recognizing their actions from videos is a challenging task towards high-level video understanding. Existing methods are mostly two-stage based, with one stage for person bounding box generation and the other stage for action recognition. However, such two-stage methods are generally with low efficiency. We observe that directly unifying detection and action recognition normally suffers from (i) inferior learning due to different desired properties of context representation for detection and action recognition; (ii) optimization difficulty with insufficient training data. In this work, we present a decoupled one-stage network dubbed DOAD, to mitigate above issues and improve the efficiency for spatio-temporal action detection. To achieve it, we decouple detection and action recognition into two branches. Specifically, one branch focuses on detection representation for actor detection, and the other one for action recognition. For the action branch, we design a transformer-based module (TransPC) to model pairwise relationships between people and context. Different from commonly used vector-based dot product in self-attention, it is built upon a novel matrix-based key and value for Hadamard attention to model person-context information. It not only exploits relationships between person pairs but also takes into account context and relative position information. The results on AVA and UCF101-24 datasets show that our method is competitive with two-stage state-of-the-art methods with significant efficiency improvement.

  Keyword: gradient

  Towards Reasonable Budget Allocation in Untargeted Graph Structure Attacks via Gradient Debias

• Authors: Authors: Zihan Liu, Yun Luo, Lirong Wu, Zicheng Liu, Stan Z. Li
• Subjects: Machine Learning (cs.LG); Cryptography and Security (cs.CR)
• Arxiv link: https://arxiv.org/abs/2304.00010
• Pdf link: https://arxiv.org/pdf/2304.00010
• Abstract It has become cognitive inertia to employ cross-entropy loss function in classification related tasks. In the untargeted attacks on graph structure, the gradients derived from the attack objective are the attacker's basis for evaluating a perturbation scheme. Previous methods use negative cross-entropy loss as the attack objective in attacking node-level classification models. However, the suitability of the cross-entropy function for constructing the untargeted attack objective has yet been discussed in previous works. This paper argues about the previous unreasonable attack objective from the perspective of budget allocation. We demonstrate theoretically and empirically that negative cross-entropy tends to produce more significant gradients from nodes with lower confidence in the labeled classes, even if the predicted classes of these nodes have been misled. To free up these inefficient attack budgets, we propose a simple attack model for untargeted attacks on graph structure based on a novel attack objective which generates unweighted gradients on graph structures that are not affected by the node confidence. By conducting experiments in gray-box poisoning attack scenarios, we demonstrate that a reasonable budget allocation can significantly improve the effectiveness of gradient-based edge perturbations without any extra hyper-parameter.

  Soft-Bellman Equilibrium in Affine Markov Games: Forward Solutions and Inverse Learning

• Authors: Authors: Shenghui Chen, Yue Yu, David Fridovich-Keil, Ufuk Topcu
• Subjects: Computer Science and Game Theory (cs.GT); Machine Learning (cs.LG)
• Arxiv link: https://arxiv.org/abs/2304.00163
• Pdf link: https://arxiv.org/pdf/2304.00163
• Abstract Markov games model interactions among multiple players in a stochastic, dynamic environment. Each player in a Markov game maximizes its expected total discounted reward, which depends upon the policies of the other players. We formulate a class of Markov games, termed affine Markov games, where an affine reward function couples the players' actions. We introduce a novel solution concept, the soft-Bellman equilibrium, where each player is boundedly rational and chooses a soft-Bellman policy rather than a purely rational policy as in the well-known Nash equilibrium concept. We provide conditions for the existence and uniqueness of the soft-Bellman equilibrium and propose a nonlinear least squares algorithm to compute such an equilibrium in the forward problem. We then solve the inverse game problem of inferring the players' reward parameters from observed state-action trajectories via a projected gradient algorithm. Experiments in a predator-prey OpenAI Gym environment show that the reward parameters inferred by the proposed algorithm outperform those inferred by a baseline algorithm: they reduce the Kullback-Leibler divergence between the equilibrium policies and observed policies by at least two orders of magnitude.

  Precoder Design with Matrix Manifold Optimization in Massive MIMO

• Authors: Authors: Rui Sun, Chen Wang, An-An Lu, Xiqi Gao, Xiang-Gen Xia
• Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)
• Arxiv link: https://arxiv.org/abs/2304.00201
• Pdf link: https://arxiv.org/pdf/2304.00201
• Abstract We investigate the weighted sum-rate (WSR) maximization linear precoder design for massive MIMO downlink and propose a unified matrix manifold optimization framework applicable to total power constraint (TPC), per-user power constraint (PUPC) and per-antenna power constraint (PAPC). Particularly, we prove that the precoders under TPC, PUPC and PAPC are on different Riemannian submanifolds, and transform the constrained problems in Euclidean space to unconstrained ones on manifolds. In accordance with this, we derive Riemannian ingredients including orthogonal projection, Riemannian gradient, Riemannian Hessian, retraction and vector transport, which are needed for precoder design in matrix manifold framework. Then, Riemannian design methods using Riemannian steepest descent, Riemannian conjugate gradient and Riemannian trust region are provided to design the WSR-maximization precoders under TPC, PUPC or PAPC. Riemannian methods are free of the inverse of large dimensional matrix, which is of great significance for practice. Complexity analysis and performance simulations demonstrate the advantages of the proposed precoder design.

  Bipartite Graph Convolutional Hashing for Effective and Efficient Top-N Search in Hamming Space

• Authors: Authors: Yankai Chen, Yixiang Fang, Yifei Zhang, Irwin King
• Subjects: Information Retrieval (cs.IR)
• Arxiv link: https://arxiv.org/abs/2304.00241
• Pdf link: https://arxiv.org/pdf/2304.00241
• Abstract Searching on bipartite graphs is basal and versatile to many real-world Web applications, e.g., online recommendation, database retrieval, and query-document searching. Given a query node, the conventional approaches rely on the similarity matching with the vectorized node embeddings in the continuous Euclidean space. To efficiently manage intensive similarity computation, developing hashing techniques for graph structured data has recently become an emerging research direction. Despite the retrieval efficiency in Hamming space, prior work is however confronted with catastrophic performance decay. In this work, we investigate the problem of hashing with Graph Convolutional Network on bipartite graphs for effective Top-N search. We propose an end-to-end Bipartite Graph Convolutional Hashing approach, namely BGCH, which consists of three novel and effective modules: (1) adaptive graph convolutional hashing, (2) latent feature dispersion, and (3) Fourier serialized gradient estimation. Specifically, the former two modules achieve the substantial retention of the structural information against the inevitable information loss in hash encoding; the last module develops Fourier Series decomposition to the hashing function in the frequency domain mainly for more accurate gradient estimation. The extensive experiments on six real-world datasets not only show the performance superiority over the competing hashing-based counterparts, but also demonstrate the effectiveness of all proposed model components contained therein.

  Reusing Deep Neural Network Models through Model Re-engineering

• Authors: Authors: Binhang Qi, Hailong Sun, Xiang Gao, Hongyu Zhang, Zhaotian Li, Xudong Liu
• Subjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2304.00245
• Pdf link: https://arxiv.org/pdf/2304.00245
• Abstract Training deep neural network (DNN) models, which has become an important task in today's software development, is often costly in terms of computational resources and time. With the inspiration of software reuse, building DNN models through reusing existing ones has gained increasing attention recently. Prior approaches to DNN model reuse have two main limitations: 1) reusing the entire model, while only a small part of the model's functionalities (labels) are required, would cause much overhead (e.g., computational and time costs for inference), and 2) model reuse would inherit the defects and weaknesses of the reused model, and hence put the new system under threats of security attack. To solve the above problem, we propose SeaM, a tool that re-engineers a trained DNN model to improve its reusability. Specifically, given a target problem and a trained model, SeaM utilizes a gradient-based search method to search for the model's weights that are relevant to the target problem. The re-engineered model that only retains the relevant weights is then reused to solve the target problem. Evaluation results on widely-used models show that the re-engineered models produced by SeaM only contain 10.11% weights of the original models, resulting 42.41% reduction in terms of inference time. For the target problem, the re-engineered models even outperform the original models in classification accuracy by 5.85%. Moreover, reusing the re-engineered models inherits an average of 57% fewer defects than reusing the entire model. We believe our approach to reducing reuse overhead and defect inheritance is one important step forward for practical model reuse.

  Doubly Stochastic Models: Learning with Unbiased Label Noises and Inference Stability

• Authors: Authors: Haoyi Xiong, Xuhong Li, Boyang Yu, Zhanxing Zhu, Dongrui Wu, Dejing Dou
• Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2304.00320
• Pdf link: https://arxiv.org/pdf/2304.00320
• Abstract Random label noises (or observational noises) widely exist in practical machine learning settings. While previous studies primarily focus on the affects of label noises to the performance of learning, our work intends to investigate the implicit regularization effects of the label noises, under mini-batch sampling settings of stochastic gradient descent (SGD), with assumptions that label noises are unbiased. Specifically, we analyze the learning dynamics of SGD over the quadratic loss with unbiased label noises, where we model the dynamics of SGD as a stochastic differentiable equation (SDE) with two diffusion terms (namely a Doubly Stochastic Model). While the first diffusion term is caused by mini-batch sampling over the (label-noiseless) loss gradients as many other works on SGD, our model investigates the second noise term of SGD dynamics, which is caused by mini-batch sampling over the label noises, as an implicit regularizer. Our theoretical analysis finds such implicit regularizer would favor some convergence points that could stabilize model outputs against perturbation of parameters (namely inference stability). Though similar phenomenon have been investigated, our work doesn't assume SGD as an Ornstein-Uhlenbeck like process and achieve a more generalizable result with convergence of approximation proved. To validate our analysis, we design two sets of empirical studies to analyze the implicit regularizer of SGD with unbiased random label noises for deep neural networks training and linear regression.

  JacobiNeRF: NeRF Shaping with Mutual Information Gradients

• Authors: Authors: Xiaomeng Xu, Yanchao Yang, Kaichun Mo, Boxiao Pan, Li Yi, Leonidas Guibas
• Subjects: Computer Vision and Pattern Recognition (cs.CV)
• Arxiv link: https://arxiv.org/abs/2304.00341
• Pdf link: https://arxiv.org/pdf/2304.00341
• Abstract We propose a method that trains a neural radiance field (NeRF) to encode not only the appearance of the scene but also semantic correlations between scene points, regions, or entities -- aiming to capture their mutual co-variation patterns. In contrast to the traditional first-order photometric reconstruction objective, our method explicitly regularizes the learning dynamics to align the Jacobians of highly-correlated entities, which proves to maximize the mutual information between them under random scene perturbations. By paying attention to this second-order information, we can shape a NeRF to express semantically meaningful synergies when the network weights are changed by a delta along the gradient of a single entity, region, or even a point. To demonstrate the merit of this mutual information modeling, we leverage the coordinated behavior of scene entities that emerges from our shaping to perform label propagation for semantic and instance segmentation. Our experiments show that a JacobiNeRF is more efficient in propagating annotations among 2D pixels and 3D points compared to NeRFs without mutual information shaping, especially in extremely sparse label regimes -- thus reducing annotation burden. The same machinery can further be used for entity selection or scene modifications.

  Fast Convergence of Random Reshuffling under Over-Parameterization and the Polyak-Łojasiewicz Condition

• Authors: Authors: Chen Fan, Christos Thrampoulidis, Mark Schmidt
• Subjects: Machine Learning (cs.LG); Optimization and Control (math.OC)
• Arxiv link: https://arxiv.org/abs/2304.00459
• Pdf link: https://arxiv.org/pdf/2304.00459
• Abstract Modern machine learning models are often over-parameterized and as a result they can interpolate the training data. Under such a scenario, we study the convergence properties of a sampling-without-replacement variant of stochastic gradient descent (SGD) known as random reshuffling (RR). Unlike SGD that samples data with replacement at every iteration, RR chooses a random permutation of data at the beginning of each epoch and each iteration chooses the next sample from the permutation. For under-parameterized models, it has been shown RR can converge faster than SGD under certain assumptions. However, previous works do not show that RR outperforms SGD in over-parameterized settings except in some highly-restrictive scenarios. For the class of Polyak-\L ojasiewicz (PL) functions, we show that RR can outperform SGD in over-parameterized settings when either one of the following holds: (i) the number of samples ($n$) is less than the product of the condition number ($\kappa$) and the parameter ($\alpha$) of a weak growth condition (WGC), or (ii) $n$ is less than the parameter ($\rho$) of a strong growth condition (SGC).

  Learning by Grouping: A Multilevel Optimization Framework for Improving Fairness in Classification without Losing Accuracy

• Authors: Authors: Ramtin Hosseini, Li Zhang, Bhanu Garg, Pengtao Xie
• Subjects: Machine Learning (cs.LG); Computers and Society (cs.CY)
• Arxiv link: https://arxiv.org/abs/2304.00486
• Pdf link: https://arxiv.org/pdf/2304.00486
• Abstract The integration of machine learning models in various real-world applications is becoming more prevalent to assist humans in their daily decision-making tasks as a result of recent advancements in this field. However, it has been discovered that there is a tradeoff between the accuracy and fairness of these decision-making tasks. In some cases, these AI systems can be unfair by exhibiting bias or discrimination against certain social groups, which can have severe consequences in real life. Inspired by one of the most well-known human learning skills called grouping, we address this issue by proposing a novel machine learning framework where the ML model learns to group a diverse set of problems into distinct subgroups to solve each subgroup using its specific sub-model. Our proposed framework involves three stages of learning, which are formulated as a three-level optimization problem: (i) learning to group problems into different subgroups; (ii) learning group-specific sub-models for problem-solving; and (iii) updating group assignments of training examples by minimizing the validation loss. These three learning stages are performed end-to-end in a joint manner using gradient descent. To improve fairness and accuracy, we develop an efficient optimization algorithm to solve this three-level optimization problem. To further reduce the risk of overfitting in small datasets, we incorporate domain adaptation techniques in the second stage of training. We further apply our method to neural architecture search. Extensive experiments on various datasets demonstrate our method's effectiveness and performance improvements in both fairness and accuracy. Our proposed Learning by Grouping can reduce overfitting and achieve state-of-the-art performances with fixed human-designed network architectures and searchable network architectures on various datasets.

  Saddle-to-Saddle Dynamics in Diagonal Linear Networks

• Authors: Authors: Scott Pesme, Nicolas Flammarion
• Subjects: Machine Learning (cs.LG); Optimization and Control (math.OC)
• Arxiv link: https://arxiv.org/abs/2304.00488
• Pdf link: https://arxiv.org/pdf/2304.00488
• Abstract In this paper we fully describe the trajectory of gradient flow over diagonal linear networks in the limit of vanishing initialisation. We show that the limiting flow successively jumps from a saddle of the training loss to another until reaching the minimum $\ell_1$-norm solution. This saddle-to-saddle dynamics translates to an incremental learning process as each saddle corresponds to the minimiser of the loss constrained to an active set outside of which the coordinates must be zero. We explicitly characterise the visited saddles as well as the jumping times through a recursive algorithm reminiscent of the Homotopy algorithm used for computing the Lasso path. Our proof leverages a convenient arc-length time-reparametrisation which enables to keep track of the heteroclinic transitions between the jumps. Our analysis requires negligible assumptions on the data, applies to both under and overparametrised settings and covers complex cases where there is no monotonicity of the number of active coordinates. We provide numerical experiments to support our findings.

  SparDL: Distributed Deep Learning Training with Efficient Sparse Communication

• Authors: Authors: Minjun Zhao, Yichen Yin, Yuren Mao, Lu Chen, Yunjun Gao
• Subjects: Machine Learning (cs.LG); Distributed, Parallel, and Cluster Computing (cs.DC)
• Arxiv link: https://arxiv.org/abs/2304.00737
• Pdf link: https://arxiv.org/pdf/2304.00737
• Abstract Top-$k$ sparsification has recently been widely used to reduce the communication volume in distributed deep learning; however, due to Gradient Accumulation (GA) dilemma, the performance of top-$k$ sparsification is still limited. Several methods have been proposed to handle the GA dilemma but have two drawbacks: (1) they are frustrated by the high communication complexity as they introduce a large amount of extra transmission; (2) they are not flexible for non-power-of-two numbers of workers. To solve these two problems, we propose a flexible and efficient sparse communication framework, dubbed SparDL. SparDL uses the Spar-Reduce-Scatter algorithm to solve the GA dilemma without additional communication operations and is flexible to any number of workers. Besides, to further reduce the communication complexity and adjust the proportion of latency and bandwidth cost in communication complexity, we propose the Spar-All-Gather algorithm as part of SparDL. Extensive experiments validate the superiority of SparDL.

  Small but Mighty: Enhancing 3D Point Clouds Semantic Segmentation with U-Next Framework

• Authors: Authors: Ziyin Zeng, Qingyong Hu, Zhong Xie, Jian Zhou, Yongyang Xu
• Subjects: Computer Vision and Pattern Recognition (cs.CV)
• Arxiv link: https://arxiv.org/abs/2304.00749
• Pdf link: https://arxiv.org/pdf/2304.00749
• Abstract We study the problem of semantic segmentation of large-scale 3D point clouds. In recent years, significant research efforts have been directed toward local feature aggregation, improved loss functions and sampling strategies. While the fundamental framework of point cloud semantic segmentation has been largely overlooked, with most existing approaches rely on the U-Net architecture by default. In this paper, we propose U-Next, a small but mighty framework designed for point cloud semantic segmentation. The key to this framework is to learn multi-scale hierarchical representations from semantically similar feature maps. Specifically, we build our U-Next by stacking multiple U-Net $L^1$ codecs in a nested and densely arranged manner to minimize the semantic gap, while simultaneously fusing the feature maps across scales to effectively recover the fine-grained details. We also devised a multi-level deep supervision mechanism to further smooth gradient propagation and facilitate network optimization. Extensive experiments conducted on three large-scale benchmarks including S3DIS, Toronto3D, and SensatUrban demonstrate the superiority and the effectiveness of the proposed U-Next architecture. Our U-Next architecture shows consistent and visible performance improvements across different tasks and baseline models, indicating its great potential to serve as a general framework for future research.

  FedIN: Federated Intermediate Layers Learning for Model Heterogeneity

• Authors: Authors: Chan Yun-Hin, Jiang Zhihan, Deng Jing, Ngai C.-H. Edith
• Subjects: Machine Learning (cs.LG)
• Arxiv link: https://arxiv.org/abs/2304.00759
• Pdf link: https://arxiv.org/pdf/2304.00759
• Abstract Federated learning (FL) facilitates edge devices to cooperatively train a global shared model while maintaining the training data locally and privately. However, a common but impractical assumption in FL is that the participating edge devices possess the same required resources and share identical global model architecture. In this study, we propose a novel FL method called Federated Intermediate Layers Learning (FedIN), supporting heterogeneous models without utilizing any public dataset. The training models in FedIN are divided into three parts, including an extractor, the intermediate layers, and a classifier. The model architectures of the extractor and classifier are the same in all devices to maintain the consistency of the intermediate layer features, while the architectures of the intermediate layers can vary for heterogeneous devices according to their resource capacities. To exploit the knowledge from features, we propose IN training, training the intermediate layers in line with the features from other clients. Additionally, we formulate and solve a convex optimization problem to mitigate the gradient divergence problem induced by the conflicts between the IN training and the local training. The experiment results show that FedIN achieves the best performance in the heterogeneous model environment compared with the state-of-the-art algorithms. Furthermore, our ablation study demonstrates the effectiveness of IN training and the solution to the convex optimization problem.

  Learning Sparsity of Representations with Discrete Latent Variables

• Authors: Authors: Zhao Xu, Daniel Onoro Rubio, Giuseppe Serra, Mathias Niepert
• Subjects: Machine Learning (cs.LG)
• Arxiv link: https://arxiv.org/abs/2304.00935
• Pdf link: https://arxiv.org/pdf/2304.00935
• Abstract Deep latent generative models have attracted increasing attention due to the capacity of combining the strengths of deep learning and probabilistic models in an elegant way. The data representations learned with the models are often continuous and dense. However in many applications, sparse representations are expected, such as learning sparse high dimensional embedding of data in an unsupervised setting, and learning multi-labels from thousands of candidate tags in a supervised setting. In some scenarios, there could be further restriction on degree of sparsity: the number of non-zero features of a representation cannot be larger than a pre-defined threshold $L_0$. In this paper we propose a sparse deep latent generative model SDLGM to explicitly model degree of sparsity and thus enable to learn the sparse structure of the data with the quantified sparsity constraint. The resulting sparsity of a representation is not fixed, but fits to the observation itself under the pre-defined restriction. In particular, we introduce to each observation $i$ an auxiliary random variable $L_i$, which models the sparsity of its representation. The sparse representations are then generated with a two-step sampling process via two Gumbel-Softmax distributions. For inference and learning, we develop an amortized variational method based on MC gradient estimator. The resulting sparse representations are differentiable with backpropagation. The experimental evaluation on multiple datasets for unsupervised and supervised learning problems shows the benefits of the proposed method.

  Eigen-Factors an Alternating Optimization for Back-end Plane SLAM of 3D Point Clouds

• Authors: Authors: Gonzalo Ferrer, Dmitrii Iarosh, Anastasiia Kornilova
• Subjects: Robotics (cs.RO)
• Arxiv link: https://arxiv.org/abs/2304.01055
• Pdf link: https://arxiv.org/pdf/2304.01055
• Abstract Modern depth sensors can generate a huge number of 3D points in few seconds to be latter processed by Localization and Mapping algorithms. Ideally, these algorithms should handle efficiently large sizes of Point Clouds under the assumption that using more points implies more information available. The Eigen Factors (EF) is a new algorithm that solves SLAM by using planes as the main geometric primitive. To do so, EF exhaustively calculates the error of all points at complexity $O(1)$, thanks to the {\em Summation matrix} $S$ of homogeneous points. The solution of EF is highly efficient: i) the state variables are only the sensor poses -- trajectory, while the plane parameters are estimated previously in closed from and ii) EF alternating optimization uses a Newton-Raphson method by a direct analytical calculation of the gradient and the Hessian, which turns out to be a block diagonal matrix. Since we require to differentiate over eigenvalues and matrix elements, we have developed an intuitive methodology to calculate partial derivatives in the manifold of rigid body transformations $SE(3)$, which could be applied to unrelated problems that require analytical derivatives of certain complexity. We evaluate EF and other state-of-the-art plane SLAM back-end algorithms in a synthetic environment. The evaluation is extended to ICL dataset (RGBD) and LiDAR KITTI dataset. Code is publicly available at https://github.com/prime-slam/EF-plane-SLAM.

  Evolving Artificial Neural Networks To Imitate Human Behaviour In Shinobi III : Return of the Ninja Master

• Authors: Authors: Maximilien Le Clei
• Subjects: Neural and Evolutionary Computing (cs.NE); Machine Learning (cs.LG)
• Arxiv link: https://arxiv.org/abs/2304.01096
• Pdf link: https://arxiv.org/pdf/2304.01096
• Abstract Our society is increasingly fond of computational tools. This phenomenon has greatly increased over the past decade following, among other factors, the emergence of a new Artificial Intelligence paradigm. Specifically, the coupling of two algorithmic techniques, Deep Neural Networks and Stochastic Gradient Descent, thrusted by an exponentially increasing computing capacity, has and is continuing to become a major asset in many modern technologies. However, as progress takes its course, some still wonder whether other methods could similarly or even more greatly benefit from these various hardware advances. In order to further this study, we delve in this thesis into Evolutionary Algorithms and their application to Dynamic Neural Networks, two techniques which despite enjoying many advantageous properties have yet to find their niche in contemporary Artificial Intelligence. We find that by elaborating new methods while exploiting strong computational resources, it becomes possible to develop strongly performing agents on a variety of benchmarks but also some other agents behaving very similarly to human subjects on the video game Shinobi III : Return of The Ninja Master, typical complex tasks previously out of reach for non-gradient-based optimization.

  WeakTr: Exploring Plain Vision Transformer for Weakly-supervised Semantic Segmentation

• Authors: Authors: Lianghui Zhu, Yingyue Li, Jieming Fang, Yan Liu, Hao Xin, Wenyu Liu, Xinggang Wang
• Subjects: Computer Vision and Pattern Recognition (cs.CV)
• Arxiv link: https://arxiv.org/abs/2304.01184
• Pdf link: https://arxiv.org/pdf/2304.01184
• Abstract This paper explores the properties of the plain Vision Transformer (ViT) for Weakly-supervised Semantic Segmentation (WSSS). The class activation map (CAM) is of critical importance for understanding a classification network and launching WSSS. We observe that different attention heads of ViT focus on different image areas. Thus a novel weight-based method is proposed to end-to-end estimate the importance of attention heads, while the self-attention maps are adaptively fused for high-quality CAM results that tend to have more complete objects. Besides, we propose a ViT-based gradient clipping decoder for online retraining with the CAM results to complete the WSSS task. We name this plain Transformer-based Weakly-supervised learning framework WeakTr. It achieves the state-of-the-art WSSS performance on standard benchmarks, i.e., 78.4% mIoU on the val set of PASCAL VOC 2012 and 50.3% mIoU on the val set of COCO 2014. Code is available at https://github.com/hustvl/WeakTr.

  Keyword: super-resolution

  AUDIT: Audio Editing by Following Instructions with Latent Diffusion Models

• Authors: Authors: Yuancheng Wang, Zeqian Ju, Xu Tan, Lei He, Zhizheng Wu, Jiang Bian, Sheng Zhao
• Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)
• Arxiv link: https://arxiv.org/abs/2304.00830
• Pdf link: https://arxiv.org/pdf/2304.00830
• Abstract Audio editing is applicable for various purposes, such as adding background sound effects, replacing a musical instrument, and repairing damaged audio. Recently, some diffusion-based methods achieved zero-shot audio editing by using a diffusion and denoising process conditioned on the text description of the output audio. However, these methods still have some problems: 1) they have not been trained on editing tasks and cannot ensure good editing effects; 2) they can erroneously modify audio segments that do not require editing; 3) they need a complete description of the output audio, which is not always available or necessary in practical scenarios. In this work, we propose AUDIT, an instruction-guided audio editing model based on latent diffusion models. Specifically, AUDIT has three main design features: 1) we construct triplet training data (instruction, input audio, output audio) for different audio editing tasks and train a diffusion model using instruction and input (to be edited) audio as conditions and generating output (edited) audio; 2) it can automatically learn to only modify segments that need to be edited by comparing the difference between the input and output audio; 3) it only needs edit instructions instead of full target audio descriptions as text input. AUDIT achieves state-of-the-art results in both objective and subjective metrics for several audio editing tasks (e.g., adding, dropping, replacement, inpainting, super-resolution). Demo samples are available at https://audit-demo.github.io/.

  Tunable Convolutions with Parametric Multi-Loss Optimization

• Authors: Authors: Matteo Maggioni, Thomas Tanay, Francesca Babiloni, Steven McDonagh, Aleš Leonardis
• Subjects: Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV)
• Arxiv link: https://arxiv.org/abs/2304.00898
• Pdf link: https://arxiv.org/pdf/2304.00898
• Abstract Behavior of neural networks is irremediably determined by the specific loss and data used during training. However it is often desirable to tune the model at inference time based on external factors such as preferences of the user or dynamic characteristics of the data. This is especially important to balance the perception-distortion trade-off of ill-posed image-to-image translation tasks. In this work, we propose to optimize a parametric tunable convolutional layer, which includes a number of different kernels, using a parametric multi-loss, which includes an equal number of objectives. Our key insight is to use a shared set of parameters to dynamically interpolate both the objectives and the kernels. During training, these parameters are sampled at random to explicitly optimize all possible combinations of objectives and consequently disentangle their effect into the corresponding kernels. During inference, these parameters become interactive inputs of the model hence enabling reliable and consistent control over the model behavior. Extensive experimental results demonstrate that our tunable convolutions effectively work as a drop-in replacement for traditional convolutions in existing neural networks at virtually no extra computational cost, outperforming state-of-the-art control strategies in a wide range of applications; including image denoising, deblurring, super-resolution, and style transfer.

  HyperThumbnail: Real-time 6K Image Rescaling with Rate-distortion Optimization

• Authors: Authors: Chenyang Qi, Xin Yang, Ka Leong Cheng, Ying-Cong Chen, Qifeng Chen
• Subjects: Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV)
• Arxiv link: https://arxiv.org/abs/2304.01064
• Pdf link: https://arxiv.org/pdf/2304.01064
• Abstract Contemporary image rescaling aims at embedding a high-resolution (HR) image into a low-resolution (LR) thumbnail image that contains embedded information for HR image reconstruction. Unlike traditional image super-resolution, this enables high-fidelity HR image restoration faithful to the original one, given the embedded information in the LR thumbnail. However, state-of-the-art image rescaling methods do not optimize the LR image file size for efficient sharing and fall short of real-time performance for ultra-high-resolution (e.g., 6K) image reconstruction. To address these two challenges, we propose a novel framework (HyperThumbnail) for real-time 6K rate-distortion-aware image rescaling. Our framework first embeds an HR image into a JPEG LR thumbnail by an encoder with our proposed quantization prediction module, which minimizes the file size of the embedding LR JPEG thumbnail while maximizing HR reconstruction quality. Then, an efficient frequency-aware decoder reconstructs a high-fidelity HR image from the LR one in real time. Extensive experiments demonstrate that our framework outperforms previous image rescaling baselines in rate-distortion performance and can perform 6K image reconstruction in real time.

  Burstormer: Burst Image Restoration and Enhancement Transformer

• Authors: Authors: Akshay Dudhane, Syed Waqas Zamir, Salman Khan, Fahad Shahbaz Khan, Ming-Hsuan Yang
• Subjects: Computer Vision and Pattern Recognition (cs.CV)
• Arxiv link: https://arxiv.org/abs/2304.01194
• Pdf link: https://arxiv.org/pdf/2304.01194
• Abstract On a shutter press, modern handheld cameras capture multiple images in rapid succession and merge them to generate a single image. However, individual frames in a burst are misaligned due to inevitable motions and contain multiple degradations. The challenge is to properly align the successive image shots and merge their complimentary information to achieve high-quality outputs. Towards this direction, we propose Burstormer: a novel transformer-based architecture for burst image restoration and enhancement. In comparison to existing works, our approach exploits multi-scale local and non-local features to achieve improved alignment and feature fusion. Our key idea is to enable inter-frame communication in the burst neighborhoods for information aggregation and progressive fusion while modeling the burst-wide context. However, the input burst frames need to be properly aligned before fusing their information. Therefore, we propose an enhanced deformable alignment module for aligning burst features with regards to the reference frame. Unlike existing methods, the proposed alignment module not only aligns burst features but also exchanges feature information and maintains focused communication with the reference frame through the proposed reference-based feature enrichment mechanism, which facilitates handling complex motions. After multi-level alignment and enrichment, we re-emphasize on inter-frame communication within burst using a cyclic burst sampling module. Finally, the inter-frame information is aggregated using the proposed burst feature fusion module followed by progressive upsampling. Our Burstormer outperforms state-of-the-art methods on burst super-resolution, burst denoising and burst low-light enhancement. Our codes and pretrained models are available at https:// github.com/akshaydudhane16/Burstormer

[AegonC]

Dear Zoq,

I am the author of Optimal Resource Allocation between Two Nonfully Cooperative Wireless Networks under Malicious Attacks: A Gestalt Game Perspective. It will be very nice if you could remove the title and abstract of our paper from your GitHub. When we submit our paper to a journal, it warns us the repeat with your sharing. It really bothers us.

Best regards,

Yukang

[zoq]

Hello,

Removed, but I'm surprised, this is public data (rss feed from arxiv) - and I can see you published the paper on arxiv. Probably a good idea to inform the journal to update their check.