and prospective/current/enrolled master students in the fields of modern mathematics and computational science:
Scientific Learning
The Institute of Computational Science (ICS) at the Università della Svizzera italiana (USI) announces six seminars about Machine Learning, Neural Networks, Physics Informed Neural Networks
Topics
Machine Learning, Neural Networks, Physics Informed Neural Networks
PROGRAM
Time 
Speaker & Place 
May 19
5:007:00 pm
UTC+2:00 Zurich time

Prof. Paris Perdikaris
Lecture: 5:006:00 pm
Seminar: 6:007:00 pm

June 2
5:007:00 pm
UTC+2:00 Zurich time

Prof. Jonathan Siegel
Lecture: 5:006:00 pm
Seminar: 6:007:00 pm

June 16
5:007:00 pm
UTC+2:00 Zurich time

Dr. Eric Cyr
Lecture: 5:006:00 pm
Seminar: 6:007:00 pm

June 23
5:007:00 pm
UTC+2:00 Zurich time

Prof. Mishra Siddhartha
Lecture: 5:006:00 pm
Seminar: 6:007:00 pm

List of Speakers
Prof. Paris Perdikaris
assistant professor at Department of Mechanical Engineering and Applied Mechanics, University of Pennsylvania.
Part A:Lecture
Making neural networks physicsinformed
Leveraging advances in automatic differentiation, physicsinformed neural networks are introducing a new paradigm in tackling forward and inverse problems in computational mechanics. Under this emerging paradigm, unknown quantities of interest are typically parametrized by deep neural networks, and a multitask learning problem is posed with the dual goal of fitting observational data and approximately satisfying a given physical law, mathematically expressed via systems of partial differential equations (PDEs). PINNs have demonstrated remarkable flexibility across diverse applications, but, despite some empirical success, a concrete mathematical understanding of the mechanisms that render such constrained neural network models effective is still lacking. In fact, more often than not, PINNs are notoriously hard to train, especially for forward problems exhibiting highfrequency or multiscale behavior. In this talk we will discuss the basic principles of making neural networks physics informed with an emphasis on the caveats one should be aware of and how those can be addressed in practice.
Part B: Seminar
Learning the solution operator of parametric partial differential equations with physicsinformed DeepONets
Deep operator networks (DeepONets) are receiving increased attention thanks to their demonstrated capability to approximate nonlinear operators between infinitedimensional Banach spaces. However, despite their remarkable early promise, they typically require large training datasets consisting of paired inputoutput observations which may be expensive to obtain, while their predictions may not be consistent with the underlying physical principles that generated the observed data. In this work, we propose a novel model class coined as physicsinformed DeepONets, which introduces an effective regularization mechanism for biasing the outputs of DeepOnet models towards ensuring physical consistency. This is accomplished by leveraging automatic differentiation to impose the underlying physical laws via soft penalty constraints during model training. We demonstrate that this simple, yet remarkably effective extension can not only yield a significant improvement in the predictive accuracy of DeepOnets, but also greatly reduce the need for large training datasets. To this end, a remarkable observation is that physicsinformed DeepONets are capable of solving parametric partial differential equations (PDEs) without any paired inputoutput observations, except for a set of given initial or boundary conditions. We illustrate the effectiveness of the proposed framework through a series of comprehensive numerical studies across various types of PDEs. Strikingly, a trained physics informed DeepOnet model can predict the solution of O(1000) timedependent PDEs in a fraction of a second  up to three orders of magnitude faster compared to a conventional PDE solver.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Prof. Jonathan Siegel
assistant professor at the Department of Mathematics, Penn State.
Part A:Lecture
The Approximation Theory of Neural Networks
An important component in understanding the potential and limitations of neural networks is a study of how efficiently neural networks can approximate a given target function class. Classical results which we will cover include the universal approximation property of neural networks and the theory of Barron and Jones which gives quantitative approximation rates for shallow neural networks. We will also discuss approximation rates for deep ReLU networks and their relationship with finite elements methods. Finally, we will introduce the notions of metric entropy and nwidths which are fundamental in approximation theory and permit a comparison between neural networks and traditional methods of approximation.
Part B: Seminar
The Metric Entropy and nwidths of Shallow Neural Networks
The metric entropy and nwidths are fundamental quantities in approximation theory which control the fundamental limits of linear and nonlinear approximation and statistical estimation for a given class of functions. In this talk, we will derive the spaces of functions which can be efficiently approximated by shallow neural networks for a wide variety of activation functions, and we will calculate the metric entropy and nwidths for these spaces. Consequences of these results include: the optimal approximation rates which can be attained for shallow neural networks, that shallow neural networks dramatically outperform linear methods of approximation, and indeed that shallow neural networks outperform all stable methods of approximation on these spaces. Finally, we will discuss insights into how neural networks break the curse of dimensionality.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Dr. Eric Cyr
Part A:Lecture
An Adaptive Basis Perspective to Improve Initialization and Accelerate Training of DNNs
Motivated by the gap between theoretical optimal approximation rates of deep neural networks (DNNs) and the accuracy realized in practice, we seek to improve the training of DNNs. The adoption of an adaptive basis viewpoint of DNNs leads to novel initializations and a hybrid least squares/gradient descent optimizer. We provide analysis of these techniques and illustrate via numerical examples dramatic increases in accuracy and convergence rate for benchmarks characterizing scientific applications where DNNs are currently used, including regression problems and physicsinformed neural networks for the solution of partial differential equations. In addition, we present a partition of unity (POU) architecture capable of achieving hp like convergence. This methodology, introduces traditional polynomial approximations on partitions learned by deep neural networks. The architecture is designed to play to the strengths of each approach, while still achieving good convergence rates as demonstrated in the results.
Part B: Seminar
A LayerParallel Approach for Training Deep Neural Networks
Deep neural networks are a powerful machine learning tool with the capacity to “learn” complex nonlinear relationships described by large data sets. Despite their success training these models remains a challenging and computationally intensive undertaking. In this talk we will present a layerparallel training algorithm that exploits a multigrid scheme to accelerate both forward and backward propagation. Introducing a parallel decomposition between layers requires inexact propagation of the neural network. The multigrid method used in this approach stitches these subdomains together with sufficient accuracy to ensure rapid convergence. We demonstrate an order of magnitude wallclock time speedup over the serial approach, opening a new avenue for parallelism that is complementary to existing approaches. In a more recent development, we discuss applying the layerparallel methodology to recurrent neural networks. In particular, we study the generalized recurrent unit (GRU) architecture. We demonstrate its relation to a simple ODE formulation that facilitates application of the layerparallel approach. Results are demonstrating performance improvements on a human activity recognition (HAR) data set are presented.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Prof. Misha Siddhartha
professor at the Department of Mathematics, ETH Zurich.
Part A:Lecture
Deep Learning and Computations of PDEs.
Abstract: In this talk, we will review some recent results on computing PDEs with deep neural networks. The focus will be on the design of neural networks as fast and efficient surrogates for PDEs. We will start with parametric PDEs and talk about novel modifications that enable standard deep neural networks to provide efficient PDE surrogates and apply them for prediction, uncertainty quantification and PDE constrained optimization. We will also cover very recent results on operator regression using novel architectures such as DeepOnets and Fourier Neural Operators, and their application to PDEs.
Part B: Seminar
On Physical Informed Neural Networks (PINNs) for computing PDEs.
Abstract: We will describe PINNs and illustrate several examples for using PINNs for solving PDEs. Our aim would be to elucidate mechanisms that underpin the success of PINNs in approximating classical solutions to PDEs by deriving bounds on the resulting error. Examples of a variety of linear and nonlinear PDEs will be provided including PDEs in high dimensions and inverse problems for PDEs.
MATERIAL
You can download the content of the lectures here
pwd is pinn
Venue
Classes will take place online
Registration
Please, send an email to
Organizers:
Rolf Krause, Maria Nestola, Patrick Zulian, Alena Kopaničáková., Simone Pezzuto, Luca Gambardella