K Nearest Neighbor

• K in KNN is the number of Nearest Neighbors.
• It is a Supervised learning algorithm.
• It classifies data points based on neighbors i.e. similarity measures. So, the data point is classified by five nearest neighbors.
• This algorithm is based on feature similarity.
• Parament tuning is used to choose k.
• If k is too low the algorithm becomes noisy and if it is too big then the algorithm takes forever to process.

How to find K?

• Take sqrt(n) number of points whose distance with the given data point is needed, where n is the no of data points.
• Find sqrt(n) number of nearest neighbors of K (where is the number of data points) by finding the distance between them by the following two methods:
• Also, make sure that the number of nearest neighbors i.e. sqrt(n) should odd. if it even then take sqrt(n)-1 number of nearest neighbors.

1) Euclidean distance:

2) Manhattan Distance:

It is used to find the distance between the real vectors using the sum of their absolute difference.

When to use KNN?

When the data is:

• labeled
• noise-free
• dataset is small

Examples-

• Recommendation system(Netflix, Amazon)
• Concept Search
• Handwriting detection
• Image Recognition

Lazy Learner-

• Memorizes the training data
• No learning phase

Steps-

1. Handle Data(import dataset and split it into training and test set)
2. Similarity(Calculate the distance between the two data instances)
3. Neighbors(Locate k most similar data instances)
4. Response(Generate response from the set of data instances)
5. Accuracy(summarizing the accuracy of the predictions)
6. Main(Tie all together)

Hands-on:

To predict whether a person will be diagnosed with diabetes or not refer to my Github link where both the dataset and the prediction is performed:

pavini11/KNN

So, that’s it friends. I hope you liked my article. Don’t forget to clap if you found my blog useful and if you want to read more of my articles, you can follow me.

Connect with me on:

LinkedIn: https://www.linkedin.com/in/pavini-jain-82851a194/

Twitter: https://twitter.com/PaviniJain?s=08

See you in my next blog!