Machine Learning - Grid Search

The majority of machine learning models contain parameters that can be adjusted to vary how the model learns. For example, the logistic regression model, from sklearn, has a parameter C that controls regularization,which affects the complexity of the model. How do we pick the best value for C ? The best value is dependent on the data used to train the model.

One method is to try out different values and then pick the value that gives the best score. This technique is known as a grid search. If we had to select the values for two or more parameters, we would evaluate all combinations of the sets of values thus forming a grid of values. Before we get into the example it is good to know what the parameter we are changing does.

tell the model, the training data resembles real world information, place a greater weight on the training data. While lower values of C do the opposite.

To get started we must first load in the dataset we will be working with.

Next in order to create the model we must have a set of independent variables X and a dependant variable y.

Now we will load the logistic model for classifying the iris flowers. from sklearn.linear_model import LogisticRegression Creating the model, setting max_iter to a higher value to ensure that the model finds a result.

C in a logistic regression model is , we will compare this later. In the example below, we look at the iris data set and try to train a model with varying values for C in logistic regression.

After we create the model, we must fit the model to the data.

To evaluate the model we run the score method.

Example from sklearn import datasets from sklearn.linear_model import

0.973. Let's see if we can do any better by implementing a grid search with difference values of 0.973.

We will follow the same steps of before except this time we will set a range of values for C. Knowing which values to set for the searched parameters will take a combination of domain knowledge and practice.

C is , we will set a range of values surrounding it. C = [0.25, 0.5, 0.75, 1, 1.25, 1.5, 1.75, 2] Next we will create a for loop to change out the values of C and evaluate the model with each change. First we will create an empty list to store the score within. scores = []

we must loop over the range of values and update the parameter each time. for choice in C:

logit.fit(X, y) scores.append(logit.score(X, y)) With the scores stored in a list, we can evaluate what the best choice of C is.

Example from sklearn import datasets from sklearn.linear_model import

C = [0.25, 0.5, 0.75, 1, 1.25, 1.5, 1.75, 2] scores = [] for choice in C:

logit.fit(X, y) scores.append(logit.score(X, y))

We can see that the lower values of C performed worse than the base parameter of . However, as we increased the value of C to 1.75 the model experienced increased accuracy.

C beyond this amount does not help increase model accuracy.

We scored our logistic regression model by using the same data that was used to train it. If the model corresponds too closely to that data, it may not be great at predicting unseen data. This statistical error is known as over fitting. To avoid being misled by the scores on the training data, we can put aside a portion of our data and use it specifically for the purpose of testing the model. Refer to the lecture on train/test splitting to avoid being misled and overfitting.