Analytic Expressions for Probabilistic Moments of PL-DNN with Gaussian Input

A. Bibi, M. Alfadly, B. Ghanem
Conference on Computer Vision and Pattern Recognition (CVPR), (2018)

Analytic Expressions for Probabilistic Moments of PL-DNN with Gaussian Input

Keywords

Deep neural network, Probabilistic moments

Abstract

​The outstanding performance of deep neural networks (DNNs), for the visual recognition task in particular, has been demonstrated on several large-scale benchmarks. This performance has immensely strengthened the line of research that aims to understand and analyze the driving rea- sons behind the effectiveness of these networks. One impor- tant aspect of this analysis has recently gained much atten- tion, namely the reaction of a DNN to noisy input. This has spawned research on developing adversarial input attacks as well as training strategies that make DNNs more robust against these attacks. To this end, we derive in this paper exact analytic expressions for the first and second mo- ments (mean and variance) of a small piecewise linear (PL) network (Affine, ReLU, Affine) subject to general Gaussian input. We experimentally show that these expressions are tight under simple linearizations of deeper PL-DNNs, es- pecially popular architectures in the literature (e.g. LeNet and AlexNet). Extensive experiments on image classifica- tion show that these expressions can be used to study the behaviour of the output mean of the logits for each class, the interclass confusion and the pixel-level spatial noise sensi- tivity of the network. Moreover, we show how these expres- sions can be used to systematically construct targeted and non-targeted adversarial attacks.

Code

DOI:

Sources

Website PDF

See all publications 2018