Abstract—Recently, we have analyzed the convergence properties and derived the objective functions of the weight noise injection-based algorithms. In this paper, we generalize our work to perturbed gradient systems (PGS). For original gradient systems (GS), the change of a vector
x(
t) at time
t is defined as the gradient vector of a function
F(
x) times a small negative constant. For PGS, it is assumed that the vector
x(
t) is perturbed by mean zero Gaussian noise. The noise can either be multiplicative or additive. In this paper, the corresponding energy functions for both cases are derived. It is found that their energy functions are very difference from
F(
x). For the case of multiplicative noise, the energy function consists of three terms: (i)
F(
x), (ii) a regularization term and (iii) a de-regularization term. For the case of additive noise, the energy function consists of only two terms: (i)
F(
x) and (ii) a regularization term. Note that de-regularization could lead to divergence behavior while regularization will improve the convergence behavior of a system. Our results suggest that special caution should be done to a gradient system with multiplicative noise.
Index Terms—Additive noise, energy function, gradient systems, multiplicative noise, noise effect.
Kevin Ho is with the Department of Computer Science and Communication Engineering, Providence University, Sha-Lu, Taiwan (e-mail: ho@pu.edu.tw).
Hsia-Ching Chang is with the Department of Information Management, Hsiuping University of Science and Technology, Dali Dist., Taichung, Taiwan ROC (e-mail: hcchang@mail.hust.edu.tw).
Wei-Bin Lee is with the Department of Information Engineering and Computer Science, Feng Chia University, Taichung, Taiwan ROC (e-mail: wblee@fcu.edu.tw).
[PDF]
Cite:Kevin Ho, Hsia-Ching Chang, and Wei-Bin Lee, "Effect of Noise on Gradient Systems," International Journal of Computer Theory and Engineering vol. 5, no. 5, pp. 841-845, 2013.