Consider designing a linear classifier

y=sign(f(x;w, b)),  f(x;w,b)=wTx+b

on a dataset D=(x1, y1),(x2, y2),. . . ,(xN, yN), xi∈ℝd, yi∈ {+1,−1}, i = 1, 2, . . . , N. Recall that the sign function outputs +1 if the argument is positive, and −1 if the argument is non-positive. The parameters w and b are updated as per the following training algorithm:

wnew = wold+ ynxn,       bnew= bold + yn

whenever sign(f(xn; wold, bold)) ≠ yn. In other words, whenever the classifier wrongly predicts a sample (xn, yn) from the dataset, wold gets updated to wnew, and likewise bold gets updated to bnew. Consider the case (xn, +1), f(xn; wold, bold) < 0. Then

A.

f(xn; wnew, bnew) > f(xn; wold, bold)

B.

f(xn; wnew, bnew) < f(xn; wold, bold)

C.

f(xn; wnew, bnew) = f(xn; wold, bold)

D.

ynf(xn; wold, bold) > 1

Solution:

We are given a linear classifier with a prediction function:

f(x;w,b)=wTx+bandy=sign(f(x;w,b))

Update Rule:

When the classifier misclassifies(xn,yn)(x_n, y_n), it updates:

wnew=wold+ynxn,bnew=bold+ynw_{\text{new}} = w_{\text{old}} + y_n x_n, \quad b_{\text{new}} = b_{\text{old}} + y_n

Given:

  • yn=+1y_n = +1

  • f(xn;wold,bold)<0f(x_n; w_{\text{old}}, b_{\text{old}}) < 0

⟹ Classifier predicted negative but the actual class is positive, so it misclassified the point.

Let’s compute:

f(xn;wnew,bnew)=wnewTxn+bnewf(x_n; w_{\text{new}}, b_{\text{new}}) = w_{\text{new}}^T x_n + b_{\text{new}}

Substitute the updated values:

=(wold+ynxn)Txn+(bold+yn)= (w_{\text{old}} + y_n x_n)^T x_n + (b_{\text{old}} + y_n)

Since yn=+1y_n = +1, this becomes:

=woldTxn+xnTxn+bold+1=f(xn;wold,bold)+∥xn∥2+1= w_{\text{old}}^T x_n + x_n^T x_n + b_{\text{old}} + 1 = f(x_n; w_{\text{old}}, b_{\text{old}}) + \|x_n\|^2 + 1

So clearly,

f(xn;wnew,bnew)>f(xn;wold,bold)f(x_n; w_{\text{new}}, b_{\text{new}}) > f(x_n; w_{\text{old}}, b_{\text{old}})

✅ Correct Answer (A)

f(xn;wnew,bnew)>f(xn;wold,bold)f(x_n; w_{\text{new}}, b_{\text{new}}) > f(x_n; w_{\text{old}}, b_{\text{old}})

This means the misclassified sample is now more likely to be classified correctly after the update.