Sprint #5 (June 1)

[pagand]

Finalize the preprocessig stage due to time constrain.

[pagand]

Mode: will consider STW in addition to speed to decide. The history is also considered to avoid abrupt mode change

Resistance:Use Thrust / (Torque * Speed) as a measurement to the resistance (double check with Ed). Please provide reference.

clustering: k =5 (discuss why k =5, take a look at the code in the main repo, which I used elbow method to decide k=2 is the most efficient one.)

Consider outlier detection before anything.

feature engineering: use one hot encoding for weather, season, direction to avoid falsify distance obligation.

Based model: LSTM inputs: 'DEPTH', 'ENGINE_1_FLOWTEMPA', 'ENGINE_1_FLOWTEMPB', 'ENGINE_2_FLOWTEMPA', 'ENGINE_2_FLOWTEMPB', 'PITCH_1', 'PITCH_2', 'POWER_1', 'POWER_2', 'RATE_OF_TURN', 'SPEED_1', 'SPEED_2', 'THRUST_1', 'THRUST_2', 'TORQUE_1', 'TORQUE_2', 'TRACK_MADE_GOOD', 'MODE', 'season', 'weekday', 'current', 'direction', 'temperature', 'humidity', 'pressure', 'precipitation', 'rain', 'snowfall', 'weathercode', 'wind_force', 'wind_direc', 'date', 'holiday', 'is_weekday', 'efficiency_ratio1', 'efficiency_ratio1', previous_longitude, previous_latitude, 'previous_heading', 'previous_SOG', 'previous_STW’

note 1: Check with Alison if DEPTH, TRACK_MADE_GOOD data contain useful info. note 2: You can not use Flowtemp, due to it's direct correlation to the FC note 3: Instead of two separate quantities, isn't it better to consider the average (e.g. POWER, ...) note 4: Are the date, temperature, pressure, precipitation really important. Remember if you put so much data into your LSTM model, you just increase the model capacity while not necessary gaining any useful information if the feature have very low correlation; This will result in overfitting. note 5: Do you want to consider both Torque and Thrust as input or a computation of them for resistance

[pagand]

@yim-fan Let me know when you have considered these. Please put to in progress/review if done.

[yim-fan]

• [x] Mode
• [x] Resistance
• [ ] Clustering: tried DBSCAN and K-means, does not have significant improvement on the metrics
• [x] Outlier Detection
• [x] Feature Engineering: one hot
• [x] note1 DEPTH, TRACK_MADE_GOOD: TRACK_MADE_GOOD does not contain any info. DEPTH: values looks good
• [x] note2 flowtemp: dropped because highly correlated with weather code
• [x] note3 now using average for fields like POWER1 and POWER2
• [x] note4 date, temperature, pressure are not used anymore, precipitation is replaced by rain and snowfall
• [x] note5 now using torque and resistance as input features.