Convergence of Limited Communication Gradient Methods

Abstract

Distributed optimization increasingly plays a centralrole in economical and sustainable operation of cyber-physicalsystems. Nevertheless, the complete potential of the technologyhas not yet been fully exploited in practice due to communicationlimitations posed by the real-world infrastructures. This workinvestigates fundamental properties of distributed optimizationbased on gradient methods, where gradient information iscommunicated using limited number of bits. In particular, ageneral class of quantized gradient methods are studied wherethe gradient direction is approximated by a finite quantizationset. Sufficient and necessary conditions are provided on sucha quantization set to guarantee that the methods minimize anyconvex objective function with Lipschitz continuous gradient anda nonempty and bounded set of optimizers. A lower bound on thecardinality of the quantization set is provided, along with specificexamples of minimal quantizations. Convergence rate results areestablished that connect the fineness of the quantization andthe number of iterations needed to reach a predefined solutionaccuracy. Generalizations of the results to a relevant class ofconstrained problems using projections are considered. Finally,the results are illustrated by simulations of practical systems.

Publication
In IEEE Transactions on Automatic Control

Related