GATE Data Science and Artificial Intelligence Practice Quiz

Question 1

In standard backpropagation for a feedforward neural network, what quantity is propagated backward from the output layer to hidden layers?

Accepted Answer

The gradient of the loss with respect to each layer's pre-activation

Question 2

For a neuron with pre-activation z = w^T x + b and activation a = sigma(z), which expression gives the gradient of the loss L with respect to weight w_i during backpropagation?

Accepted Answer

dL/dw_i = (dL/dz) x_i

Question 3

A network has one hidden layer with tanh activation and a linear output. The loss is mean squared error L = (1/2)(y_hat - y)^2 for one example. If the output neuron computes y_hat = v^T h + c, what is dL/dv?

Accepted Answer

(y_hat - y) h

Question 4

In gradient checking for backpropagation, a parameter theta_i is perturbed by a small epsilon. Which finite-difference approximation is generally preferred for numerically estimating dL/dtheta_i?

Accepted Answer

[L(theta_i + epsilon) - L(theta_i - epsilon)] / (2 epsilon)

Question 5

Consider a hidden layer l with pre-activation z^(l), activation a^(l), and next-layer weights W^(l+1). If delta^(l+1) = dL/dz^(l+1), which expression correctly gives delta^(l)?

Accepted Answer

delta^(l) = ((W^(l+1))^T delta^(l+1)) ⊙ g'(z^(l))