Less is more | Feature selection in ML

Source: — dataaspirant.com

More the data, better the analysis. Or, is that a myth? Time and again, we keep hearing the term — ‘curse of dimensionality’. Why is having more dimensions (more features in a dataset) a curse? Let’s take a deep dive!

Let’s say, we have 25 observations and only one feature, that is, there is only one attribute that define these observations. We plot these 25 observations as points in a plane. Since there is only one feature, there will be only one dimension. …

It is better to be approximately right rather than precisely wrong.”- Warren Buffet

Source: — giphy.com

Let’s say you are planning an event; you estimate the cost of hosting the event, basis the data from the past few years. After the event, you calculate the actual cost. Did you make an accurate prediction earlier? That’s remote! If that was a yes, you either got very lucky or you are lying ;) But, was your prediction good enough? Sure, that’s a possibility!

How did you know that your prediction was acceptable? The deviation of the prediction from the actual was quite narrow. Wasn’t it…

The ‘why, what & how’ of PCA (Principal Component Analysis)

Source: — megaputer.com

Availability of data needs no introduction. And, its needless to say, data drives businesses, these days. For a given business problem, the more features that we capture, the better the analysis. Isn’t it? Or is it?

What if, …

1. There are more dimensions/features than the number of observations?

2. There are too many dimensions, sometimes running into hundreds or more?

The above scenarios define a specific phenomenon called Curse of Dimensionality, which in simple words mean, too many features could actually be a problem. But, how?

Let’s say, we…

Imagine you are on a game show. There are three closed doors in front of you. One door has a car behind it, and the other two doors have a goat. The objective is to try and choose the door that has the car (of course!). You pick a door. Before revealing what’s behind it, the game show host opens one of the two closed doors and shows you a goat. Now, he asks you a question.

“Switch or Stay”?

You have the option to stay with your original choice or switch to the other closed door. …

Dhivya Karthic

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store