Statistical Learning Theory 的本质
与经典的参数化方法不同，参数化方法假设 x 和 y 的关系服从某个确定性的函数。 (p3) this is a fundamental difference from parametric models, in which the relationship between the inputs x and the outputs y is assumed to follow some unknown function f ∈ F from a known, finite-dimensional set of functions F.
- assuming that the output value y to a given x is stochastically generated by P( · |x) accommodates the fact that in general the information contained in x may not be sufficient to determine a single response in a deterministic manner.
- assuming that the conditional probability P( · |x) is unknown contributes to the fact that we assume that we do not have a reasonable description of the relationship between the input and output values.
SVM 和 GP 的关系
For a brief description of kernel ridge regression and Gaussian processes, see Cristianini and Shawe-Taylor (2000, Section 6.2).
We refer to Wahba (1999) for the relationship between SVMs and Gaussian processes.