Share

Conjugate Gradient Algorithms in Nonconvex Optimization

2008-11-18 Mathematics
Download Conjugate Gradient Algorithms in Nonconvex Optimization PDF Online Free

Author :
Release : 2008-11-18
Genre : Mathematics
Kind : eBook
Book Rating : 34X/5 ( reviews)

GET EBOOK


Book Synopsis Conjugate Gradient Algorithms in Nonconvex Optimization by : Radoslaw Pytlak

Download or read book Conjugate Gradient Algorithms in Nonconvex Optimization written by Radoslaw Pytlak. This book was released on 2008-11-18. Available in PDF, EPUB and Kindle. Book excerpt: This book details algorithms for large-scale unconstrained and bound constrained optimization. It shows optimization techniques from a conjugate gradient algorithm perspective as well as methods of shortest residuals, which have been developed by the author.

Conjugate Gradient Algorithms and Finite Element Methods

2012-12-06 Science
Download Conjugate Gradient Algorithms and Finite Element Methods PDF Online Free

Author :
Release : 2012-12-06
Genre : Science
Kind : eBook
Book Rating : 606/5 ( reviews)

GET EBOOK


Book Synopsis Conjugate Gradient Algorithms and Finite Element Methods by : Michal Krizek

Download or read book Conjugate Gradient Algorithms and Finite Element Methods written by Michal Krizek. This book was released on 2012-12-06. Available in PDF, EPUB and Kindle. Book excerpt: The position taken in this collection of pedagogically written essays is that conjugate gradient algorithms and finite element methods complement each other extremely well. Via their combinations practitioners have been able to solve complicated, direct and inverse, multidemensional problems modeled by ordinary or partial differential equations and inequalities, not necessarily linear, optimal control and optimal design being part of these problems. The aim of this book is to present both methods in the context of complicated problems modeled by linear and nonlinear partial differential equations, to provide an in-depth discussion on their implementation aspects. The authors show that conjugate gradient methods and finite element methods apply to the solution of real-life problems. They address graduate students as well as experts in scientific computing.

Algorithms for Smooth Nonconvex Optimization with Worst-case Guarantees

2020
Download Algorithms for Smooth Nonconvex Optimization with Worst-case Guarantees PDF Online Free

Author :
Release : 2020
Genre :
Kind : eBook
Book Rating : /5 ( reviews)

GET EBOOK


Book Synopsis Algorithms for Smooth Nonconvex Optimization with Worst-case Guarantees by : Michael John O'Neill

Download or read book Algorithms for Smooth Nonconvex Optimization with Worst-case Guarantees written by Michael John O'Neill. This book was released on 2020. Available in PDF, EPUB and Kindle. Book excerpt: The nature of global convergence guarantees for nonconvex optimization algorithms has changed significantly in recent years. New results characterize the maximum computational cost required for algorithms to satisfy approximate optimality conditions, instead of focusing on the limiting behavior of the iterates. In many contexts, such as those arising from machine learning, convergence to approximate second order points is desired. Algorithms designed for these problems must avoid saddle points efficiently to achieve optimal worst-case guarantees. In this dissertation, we develop and analyze a number of nonconvex optimization algorithms. First, we focus on accelerated gradient algorithms and provide results related to the avoidance of "strict saddle points''. In addition, the rate of divergence these accelerated gradient algorithms exhibit when in a neighborhood of strict saddle points is proven. Subsequently, we propose three new algorithms for smooth, nonconvex optimization with worst-case complexity guarantees. The first algorithm is developed for unconstrained optimization and is based on the classical Newton Conjugate Gradient method. This approach is then extended to bound constrained optimization by modifying the primal-log barrier method. Finally, we present a method for a special class of ``strict saddle functions'' which does not require knowledge of the parameters defining the optimization landscape. These algorithms converge to approximate second-order points in the best known computational complexity for their respective problem classes.

Nonlinear Conjugate Gradient Methods for Unconstrained Optimization

2020-06-23 Mathematics
Download Nonlinear Conjugate Gradient Methods for Unconstrained Optimization PDF Online Free

Author :
Release : 2020-06-23
Genre : Mathematics
Kind : eBook
Book Rating : 504/5 ( reviews)

GET EBOOK


Book Synopsis Nonlinear Conjugate Gradient Methods for Unconstrained Optimization by : Neculai Andrei

Download or read book Nonlinear Conjugate Gradient Methods for Unconstrained Optimization written by Neculai Andrei. This book was released on 2020-06-23. Available in PDF, EPUB and Kindle. Book excerpt: Two approaches are known for solving large-scale unconstrained optimization problems—the limited-memory quasi-Newton method (truncated Newton method) and the conjugate gradient method. This is the first book to detail conjugate gradient methods, showing their properties and convergence characteristics as well as their performance in solving large-scale unconstrained optimization problems and applications. Comparisons to the limited-memory and truncated Newton methods are also discussed. Topics studied in detail include: linear conjugate gradient methods, standard conjugate gradient methods, acceleration of conjugate gradient methods, hybrid, modifications of the standard scheme, memoryless BFGS preconditioned, and three-term. Other conjugate gradient methods with clustering the eigenvalues or with the minimization of the condition number of the iteration matrix, are also treated. For each method, the convergence analysis, the computational performances and the comparisons versus other conjugate gradient methods are given. The theory behind the conjugate gradient algorithms presented as a methodology is developed with a clear, rigorous, and friendly exposition; the reader will gain an understanding of their properties and their convergence and will learn to develop and prove the convergence of his/her own methods. Numerous numerical studies are supplied with comparisons and comments on the behavior of conjugate gradient algorithms for solving a collection of 800 unconstrained optimization problems of different structures and complexities with the number of variables in the range [1000,10000]. The book is addressed to all those interested in developing and using new advanced techniques for solving unconstrained optimization complex problems. Mathematical programming researchers, theoreticians and practitioners in operations research, practitioners in engineering and industry researchers, as well as graduate students in mathematics, Ph.D. and master students in mathematical programming, will find plenty of information and practical applications for solving large-scale unconstrained optimization problems and applications by conjugate gradient methods.

On the Relationship Between Conjugate Gradient and Optimal First-Order Methods for Convex Optimization

2013
Download On the Relationship Between Conjugate Gradient and Optimal First-Order Methods for Convex Optimization PDF Online Free

Author :
Release : 2013
Genre :
Kind : eBook
Book Rating : /5 ( reviews)

GET EBOOK


Book Synopsis On the Relationship Between Conjugate Gradient and Optimal First-Order Methods for Convex Optimization by : Sahar Karimi

Download or read book On the Relationship Between Conjugate Gradient and Optimal First-Order Methods for Convex Optimization written by Sahar Karimi. This book was released on 2013. Available in PDF, EPUB and Kindle. Book excerpt: In a series of work initiated by Nemirovsky and Yudin, and later extended by Nesterov, first-order algorithms for unconstrained minimization with optimal theoretical complexity bound have been proposed. On the other hand, conjugate gradient algorithms as one of the widely used first-order techniques suffer from the lack of a finite complexity bound. In fact their performance can possibly be quite poor. This dissertation is partially on tightening the gap between these two classes of algorithms, namely the traditional conjugate gradient methods and optimal first-order techniques. We derive conditions under which conjugate gradient methods attain the same complexity bound as in Nemirovsky-Yudin's and Nesterov's methods. Moreover, we propose a conjugate gradient-type algorithm named CGSO, for Conjugate Gradient with Subspace Optimization, achieving the optimal complexity bound with the payoff of a little extra computational cost. We extend the theory of CGSO to convex problems with linear constraints. In particular we focus on solving $l_1$-regularized least square problem, often referred to as Basis Pursuit Denoising (BPDN) problem in the optimization community. BPDN arises in many practical fields including sparse signal recovery, machine learning, and statistics. Solving BPDN is fairly challenging because the size of the involved signals can be quite large; therefore first order methods are of particular interest for these problems. We propose a quasi-Newton proximal method for solving BPDN. Our numerical results suggest that our technique is computationally effective, and can compete favourably with the other state-of-the-art solvers.

You may also like...