VLFeat includes a fast SVM solver, called
vl_svmpegasos
. The function implements the Pegasos SVM
algorithm [1], with a few adds such online
Homogeneous kernel map expansion and SVM online statistics.
Pegasos SVM
A simple example on how to use vl_svmpegasos
is presented below. Let's first build the training data
% Set up training data
Np = 200 ;
Nn = 200 ;
Xp = diag([1 3])*randn(2, Np) ;
Xn = diag([1 3])*randn(2, Nn) ;
Xp(1,:) = Xp(1,:) + 2 ;
Xn(1,:) = Xn(1,:) - 2 ;
X = [Xp Xn] ;
y = [ones(1,Np) -ones(1,Nn)] ;
Plotting X and y we have

Learning a linear classifier can be easily done with the following 2 lines of code:
dataset = vl_maketrainingset(X, int8(y)) ;
[w b info] = vl_svmpegasos(dataset, 0.01, ...
'MaxIterations',5000) ;
where we first create a struct containing the training data
using vl_maketraining data
and then we call the the SVM
solver. The output model w
is plotted over the training
data in the following figure.

The output b
is equal to 0
since the
training data admits an SVM model passing from the origins.
The output info
is a struct containing some
statistic on the learned SVM:
info =
dimension: 2
iterations: 5000
maxIterations: 5000
epsilon: -1
lambda: 0.0100
biasMultiplier: 0
biasLearningRate: 1
energyFrequency: 100
elapsedTime: 0.0022
energy: 0.1727
regularizerTerm: 0.0168
lossPos: 0.1003
lossNeg: 0.0556
hardLossPos: 0.1050
hardLossNeg: 0.0750
It is also possible to use under some assumptions[2] an homogeneous kernel map expanded online inside the solver. This can be done with the following command:
dataset = vl_maketrainingset(X, int8(y).'homkermap',2,'KChi2') ;
The above code creates a training set without applying any homogeneous kernel map to the data. When the solver is called it will expand each data point with a Chi Squared kernel of period 2.
Real-time diagnostics
VLFeat allows to get statistics during the training process. It is
sufficient to pass a function handle to the solver. The function
will be then called every energyFrequency
time.
The following function simply plots the SVM energy values for all the past iterations:
function energy = diagnostics(svm,energy)
figure(2) ;
energy = [energy svm.energy] ;
plot(energy) ;
drawnow ;
The energy value for the past iterations are kept in the
row vector energy
. The following code produces a plot of the energy value in real-time during the learning process.
energy = [] ;
dataset = vl_maketrainingset(X, int8(y)) ;
[w b info] = vl_svmpegasos(dataset, lambda, ...
'MaxIterations',5000,...
'DiagnosticFunction',@diagnostics,...
'DiagnosticCallRef',energy) ;

References
- [1] Y. Singer and N. Srebro Pegasos: Primal estimated sub-gradient solver for SVM In Proc. ICML, 2007.
- [2] A. Vedaldi and A. Zisserman. Efficient additive kernels via explicit feature maps. In PAMI, 2011.