host response rate and normalized rating negative coefficient

[wingyiuc]

The coefficient for host response rate and normalized rating seems wrong. I thought they should be the higher the better, but now their coefficents are negative.

[chrystalquek]

Host response rate:

• 18299/74111 are NA columns. They are mean-filled. (tested that it is MCAR)
• The correlation coefficient is almost zero.
• Thus it is expected that the coefficient be very near zero.
• Attempted using StandardScaler/MinMaxScaling, but coefficient still negative.
• Suggestion: Leave out of feature selection.

Normalized rating:

• 16722/74111 are NA columns. They are linear reg-filled. (tested that it is MAR)
• On second thought, multiplication dosen't really make sense, because bad airbnbs with lots of bad reviews might have a higher normalized rating than a good airbnb with only a few reviews.
• Suggestion: A min-maxed scale the review_scores_rating, ignore number_of_reviews.

Progress

• Working on it in the MR (not updated remotely yet)

[wingyiuc]

Ok, let's separate the review scores and number of reviews as two separate features. It's because the number of reviews should have been incorporated into the calculation of review scores, however, we don't have data for each individual review score for the normalization.