What should be our focus next?

[IdoRafael]

Dimensionality reduction or feature engineering will probably give the best bang for buck - not sure its allowed.

Also check out Bonus assignments.