Machine Learning

Consider designing a linear classifier y=sign(f(x;w,b)),f(x;w,b)=wTx+b on a dataset D=(x1,y1),(x2,y2),...,(xN,yN),xid, yi{+1,1}, i=1,2,...,N. Recall that the sign function outputs +1 if the argument is positive, and 1 if the argument is non-positive. The parameters w and b are updated as per the following training algorithm: wnew=wold+ynxn,bnew=bold+yn whenever sign(f(xn;wold,bold))yn. In other words, whenever the classifier wrongly predicts a sample(xn,yn) from the dataset,wold gets updated to wnew, and likewise bold gets updated to bnew. Consider the case (xn,+1), f(xn;wold,bold)0. Then