Pseudo Information Divergences Defined on the Family of Specific Probability Distributions

Several information measures have been used as the criteria in information theory, statistics and various fields of engineering. Especially an information divergence has been well used as the measure of the difference between two probability distributions. In this paper, we propose the pseudo information divergence, which functions as usual information divergence, if two measured probability distributions are in some family of specific distributions. We introduce an example of the pseudo information divergence, and apply it to the problem of training multi-layer perceptrons from the data with the gross error noise.