Feature Engineering Made Easy

[duoan]

Six major aspects of feature engineering:

• Feature understanding: learning how to identify data base on its qualities and quantitative state.
• Feature improvement: cleaning and imputing missing data values in order to maximize the dataset's values.
• Feature selection: statistically selecting and subsetting feature sets in order to reduce the noise in our data.
• Feature construction: building new features with the intention of exploiting feature interactions.
• Feature transformation: extracting latent(hidden) structure within datasets in order to mathematically transform our datasets into something new (and usually better).
• Feature learning: harnessing the power of deep learning to view data in a whole new light that will open up new problems to be solved.

[duoan]

Quantitative data are data that are numerical in nature. They should be measuring the quantity of something.

Qualitative data are data that are categorical in nature. They should be describing the quality of something.

[duoan]

The four levels of data.

• The nominal level
• The ordinal level
• The interval level
• The ratio level

[duoan]

The four levels of data.

• The nominal level
• The ordinal level
• The interval level
• The ratio level

The nominal level has the weakest structure. It consists of data that are purely described by name. Basic Examples includes blood type(A,O,AB), species of animal, or names of people. These types of data are all qualitative.

[duoan]

The four levels of data.

• The nominal level
• The ordinal level
• The interval level
• The ratio level

The nominal level has the weakest structure. It consists of data that are purely described by name. Basic Examples includes blood type(A,O,AB), species of animal, or names of people. These types of data are all qualitative.

The ordinal scale inherits all of the properties of the nominal level, but has important additional properties:

• Data at the ordinal level can be naturally ordered.
• The implies that some data values in the column can be considered better than or greater than others.

As with the nominal level, data at the ordinal level is still categorical in nature, even if numbers are used to represent the categories.

[duoan]

Cleaning Datasets

• Identifying missing values in data.
• Removing harmful data.
• Imputing (filling in) these missing values.
• Normalizing / standardizing data
• Constructing brand new features
• Selecting (removing) features manually and automatically
• Using mathematical matrix computations to transform datasets to different dimensions.