SVM Loss Functions, Gradients and Hessians

Two popular loss functions used in the primal SVM formulation are: (a) Squared Loss and (b) Hinge Loss. The following note(s) show how the first and second derivatives of these losses are computed based on the assumptions of the model - a.k.a. linear model (with or without bias). Clearly, these can be extended to the non-linear kernel, however, those are left as an exercise for the reader.

Squared Loss and its first and second derivatives.

Hinge Loss and its first and second derivatives