Balance "Grid 0" data

[ofithcheallaigh]

This is needed to balance the datasets when looking to make a determination of grid number.

If grids 1 to 9 have 1000 data points in each grid, we will want grid 0 to have 1000 data points to balance

At the minute, my data has a majority of grid 0 data points, so this majority either needs to be down sampled, or, the minority data needs to be up sampled.

To Do

• [x] Investigate methods of down sampling
• [ ] Investigate methods for up sampling (one would be SMOTE)
• [ ] Implement both up- and down-sampling
• [ ] Compare results

[ofithcheallaigh]

I have done a process for allow the addition of Grid 0, and balanced that data.

It is more involved that it maybe needs to be. The process is as follows:

• Get the lengths the the input data
• Find the lengths of the data in each grid position for the data set with an object
• Using the above, figure out how many samples I need from each grid position without an object
• Build the new dataset by taking a portion of samples from each grid point of the data without an object
• Convert this new data set to be Grid 0

As an example, let's say each grid position of the data set with an object has 900 samples, totaling 8100 samples for the full data set. I will need a Grid 0 dataset that has 900 samples. To do this, I will take 100 samples from Grid 1, 100 samples from Grid 2, and so on, for all the grids. This will give me a grid 0 with 900 samples, thereby balancing the data set.

[ofithcheallaigh]

Data sets were balanced, and there isn't the time, or the need to complete the other items here.

Closed.