Consider the linear regression setting in which you are given a training set: consisting of inputs where and are conditionally independent given their inputs . Let and . Our goal is to find the parameters for the linear regression model.
One approach for finding these parameters is maximum likelihood estimation in which we maximize the predictive distribution of the data given the parameters. We obtain the MLE parameters as:
To find the parameters we typically perform gradient descent. However, a closed-form solution also exists to find the parameters. Derive the closed-form solution to find
Hint: Instead of maximizing the likelihood directly think about how we can use to the log transformation to simplify this derivation.