Martch 23, 2022 | Virtual
MS: Analysis, Modeling and Decomposition of Nonlinear Flows
Nonlinear flows appear in various applications of image processing and machine learning, most notably in optimization algorithms and training of neural networks. In this minisymposium we present several approaches for analyzing nonlinear flows. The talks will cover recent advances related to Koopman operators, mode decomposition, nonlinear spectral analysis and convex optimization.
This work binds the existence of Koopman Eigenfunctions (KEF), the geometric of the dynamics, and the validity of Dynamic Mode Decomposition (DMD) to one coherent theory. Viewing the dynamic as a curve in the state-space allows us to formulate an existence condition of KEF and their multiplicities. These conditions lay the foundations for system reconstruction, global controllability, and observability for nonlinear dynamics. DMD can be interpreted as a finite dimension approximation of Koopman Mode Decomposition (KMD). However, this method is limited to the case when KEF are linear combinations of the observations. We examine the limitations of DMD through the analysis of Koopman theory. We propose a new mode decomposition technique based on the typical time profile of the dynamics. An overcomplete dictionary of decay profiles is used to sparsely represent the dynamic. This analysis is also valid in the full continuous setting of Koopman theory, which is based on variational calculus. We demonstrate applications of this analysis, such as finding KEF and their multiplicities, calculating KMD, dynamics reconstruction, global linearization, and controllability.
January 6, 2022 | Imperial College, London
Koopman Analysis of Gradient Descent Optimization
Recently applications of tools from dynamical systems, such as Dynamic Mode Decomposition (DMD) and Koopman analysis, to Neural Networks (NNs) are becoming increasingly popular. Such works attempt to accelerate the training, while avoiding the blackbox nature of NNs by employing well understood mathematical tools. This paper presents a bottom-up theoretical analysis of the training of NNs via the application of Koopman Theory, and its reconstruction by DMD. we then use this reconstruction to extrapolate the trajectory of the weights during training, to achieve acceleration, save computation power and reach better stability.
July 17, 2020 | Virtual
MS: Nonlinear Spectral Analysis with Applications in Imaging and Data Science
In recent years, nonlinear spectral analysis has emerged as promising tool to tackle problems in image processing. On one hand, spectral decompositions emerging from the total variation flow have been used for face fusion, nonlinear spectral filtering, and for designing invariant feature descriptors. On the other hand, so-called ground states of p-Laplacian operators have undergone thorough theoretical analysis and have found applications in image segmentation and data clustering. From a theoretical point of view, studying eigenfunctions of a subdifferential operator strengthens the understanding of regularizing properties of the associated functional. A main objective for the future is to bridge the gap to other disciplines which use spectral techniques to create synergies and find new applications. In this two-part minisymposium researchers working on nonlinear spectral analysis and relates fields will present their latest results and discuss future trends.
In the last decade a lot of attention was dedicated to formulating a signal decomposition that is based on the eigenfunctions of a nonlinear operator. In this talk, we show how concepts from Koopman theory leads to an efficient, effective and theoretically sound framework for signal decomposition. Key to our approach is a novel Dynamic Mode Decomposition (DMD) algorithm which involves a nonequispaced temporal sampling of observations. In our setup, we can naturally generalize signal analysis tools such as the transform and its inverse, a signal filtering and the spectrum of a signal. Moreover, we establish links between our framework and the discrete Fourier transform. For instance, we prove that Parseval’s identity holds exactly in our formulation. Finally, we evaluate our machinery on several examples and discuss its advantages with respect to previous work.
November 20, 2019 | LA, CA
Fluid dynamics meets image processing through nonlinear mode decomposition.
In recent years, an intense effort in image and signal processing was dedicated to formulating nonlinear signal decompositions, related to eigenfunctions of nonlinear operators. Dynamic Mode Decomposition (DMD) is frequently used in fluid dynamics to analyze nonlinear flows. In this talk, we show how a new configuration of DMD can approximate nonlinear signal decompositions. This configuration involves nonequispaced sampling as an essential condition for the existence of DMD. With tools borrowed from fluid dynamics, we define a signal analysis framework, including transform, inverse-transform, filtering and spectrum. Some commonalities are shown between this framework and the discrete Fourier transform. Moreover, we establish a Parseval identity for this transform.
July 18, 2019 | Valencia, Spain
MS: Nonlinear Spectral Decompositions with Applications in Imaging and Data Science
Spectral representations for p-homogeneous regularization
The TV-transform is based on linear decay and finite extinction of an eigenfunction under the TV-flow. We show that signals under the gradient flow of 𝑝-homogeneous functionals, 𝑝 ∈ (1,2), get extinct in finite time. Moreover, an eigenfunction of the variational derivative operator decays polynomially under this flow. Based on these attributes, we define a nonlinear transform, the 𝑝-transform, associated with eigenfunctions of the operator. This framework rigorously defines the notions of decomposition, reconstruction, filtering, and spectrum.
