MATLAB Central Newsreader - SVM simple question

SVM simple question

Ibraheem — Thu, 11 Mar 2010 02:31:05 -0500

Hi guys,

So far I understand that SVM try to find the parameters (w and b) of the hyperplane described by the equation y(i)=w*x(i)+b
I wonder where svmtrain function save the w vector. I can not find it in the SVMStruct.

Regards,

Re: SVM simple question

Bruno Luong — Thu, 11 Mar 2010 07:30:26 -0500

"Ibraheem " <ibr_ex@hotmail.com> wrote in message <hn9kl9$5nn$1@fred.mathworks.com>...
> Hi guys,
>
> So far I understand that SVM try to find the parameters (w and b) of the hyperplane described by the equation y(i)=w*x(i)+b
> I wonder where svmtrain function save the w vector. I can not find it in the SVMStruct.

svmtrain does no save w and b, because w can only be explicitly computed from *linear* kernel. SVM train stores support vectors and the dual variables times the class binary presentation (+/-1). Everything can be computed from kernel and support vectors.

Bruno

Re: SVM simple question

Ibraheem — Fri, 12 Mar 2010 06:13:02 -0500

Thank you for your concern and quick reply,
Can you, please, tell me how to compute w from *linear* kernel?
Regards,

Re: SVM simple question

Bruno Luong — Fri, 12 Mar 2010 11:09:07 -0500

"Ibraheem " <ibr_ex@hotmail.com> wrote in message <hncm1e$c05$1@fred.mathworks.com>...
> Thank you for your concern and quick reply,
> Can you, please, tell me how to compute w from *linear* kernel?
> Regards,

http://www.mathworks.com/matlabcentral/newsreader/view_thread/260308#679215

replace "w'*x" by kernel operator applied on w and x.

Bruno

Re: SVM simple question

Ibraheem — Tue, 16 Mar 2010 02:33:08 -0400

Hi Bruno,
>http://www.mathworks.com/matlabcentral/newsreader/view_thread/260308#679215
> replace "w'*x" by kernel operator applied on w and x.
Thank you again for your concern and quick reply.
I have read your post but I can not understand the notation.
Do you mean :
    W = sum( lambda*Y*X);
    b = -1/2 ( mean (SV(SV(2)<0)) W'*X + mean(SV(SV(2)>0)) W'*X);
I don't understand two things:
   1. Does the mean for all matrix or for the second column only.
   2. What is the dual variable ? Is it alpha ?
Sorry for bothering and regards,

Re: SVM simple question

Bruno Luong — Tue, 16 Mar 2010 07:52:03 -0400

"Ibraheem " <ibr_ex@hotmail.com> wrote in message <hnmql4$osg$1@fred.mathworks.com>...
> Hi Bruno,
> >http://www.mathworks.com/matlabcentral/newsreader/view_thread/260308#679215
> > replace "w'*x" by kernel operator applied on w and x.
> Thank you again for your concern and quick reply.
> I have read your post but I can not understand the notation.
> Do you mean :
> W = sum( lambda*Y*X);
> b = -1/2 ( mean (SV(SV(2)<0)) W'*X + mean(SV(SV(2)>0)) W'*X);

> I don't understand two things:
> 1. Does the mean for all matrix or for the second column only.

For a data #i, Yi is class (or group +/--1) of the data Xi (a vector observation in R^n, its is one row of the training data your pass to svmclassify), and lambda_i is the dual variable (scalar) returned. Multiply these three term and add them together (sum on i), you get a vector W. Next compute b which is a scalar.

> 2. What is the dual variable ? Is it alpha ?
> Sorry for bothering and regards,

Matlab classify returns in Alpha the product (Yi*lambdai_i) after training.

Bruno

Re: SVM simple question

Ibraheem — Tue, 16 Mar 2010 23:34:05 -0400

I think I get the idea now, thank you very much.

Re: SVM simple question

Daniel — Thu, 03 Jun 2010 14:13:03 -0400

"Bruno Luong" <b.luong@fogale.findmycountry> wrote in message <hnndb3$a6q$1@fred.mathworks.com>...
> "Ibraheem " <ibr_ex@hotmail.com> wrote in message <hnmql4$osg$1@fred.mathworks.com>...
> > Hi Bruno,
> > >http://www.mathworks.com/matlabcentral/newsreader/view_thread/260308#679215
> > > replace "w'*x" by kernel operator applied on w and x.
> > Thank you again for your concern and quick reply.
> > I have read your post but I can not understand the notation.
> > Do you mean :
> > W = sum( lambda*Y*X);
> > b = -1/2 ( mean (SV(SV(2)<0)) W'*X + mean(SV(SV(2)>0)) W'*X);
>
> > I don't understand two things:
> > 1. Does the mean for all matrix or for the second column only.
>
> For a data #i, Yi is class (or group +/--1) of the data Xi (a vector observation in R^n, its is one row of the training data your pass to svmclassify), and lambda_i is the dual variable (scalar) returned. Multiply these three term and add them together (sum on i), you get a vector W. Next compute b which is a scalar.
>
> > 2. What is the dual variable ? Is it alpha ?
> > Sorry for bothering and regards,
>
> Matlab classify returns in Alpha the product (Yi*lambdai_i) after training.
>
> Bruno

where i can find lambda? is alpha=Yi*lambda?
and i'm doing some project using svm and my case is non-linear separable case, in non-linear there is variable C, how can i adjust variable C in svmtrain?

regards