Project Plan

[chriskuz]

Start to Finish:

• find dataset
• submit project proposal
• project repo setup and git training
• data cleaning (a few days)
• delegate tasks (plotting + modeling + model error reduction + feature pruning + event detections (look at news per route))
• organize and analyze results
• make final presentation

Detailed task and timeline:

• data cleaning (consistent updating of dataframe)
• yaml file(?)
• address big ticket items
• delegation
• kpi creation/feature engineering
• transformation functions for kpi inclusion/exclusion (removing extra features for master dataset and using functions to add or subtract)
• plot creation
• model exploration
• feature pruning per model

Resources associated with task:

Dependencies:

• data cleaning
• Python packages(sklearn, matplotlib, seaborn, numpy, pandas, os, plotly, math, cmath)
• training and test data splitting
• training data (validation, )

Milestones:

• generalized model prediction for when to buy JetBlue fr lowest price given a route-price history (linear and random forest regression)
• create detailed data visualization addressing model influence

[chriskuz]

KPI exploration: Columns

• (dollar/mile (business oriented) or mile/dollar (consumer + business oriented) (how much money does a customer spend on a per mile basis and is there fluctuation on price and mileage),
• collect weather data (temperature in particular)
• classify most dominant type of weather (rain or no rain, snow or no snow, we would need....route+day-landing pairing to collect weather...not sure if we want to collect weather per search)
• total travel segment time (time wasted for customer on legs of trip summated OR time wasted on general single leg route)...if layover time affects the fare
• average price per route (and then we can say whether the fare is above or below the average/median)...do this for generic route and do this per month/half month
• convert time from seconds to (minutes?)(hours?)

Generic

• how many searches per route

[chriskuz]

Plot idea:

• seats remaining and price tracking over time per flight (timeseries plot)
• longest to shortest route identification (bar chart)
• segmented routes vs non segmented routes (numbers or percentages)
• unique flights searched (number)
• searches per unique flight (ratio)
• most common aircraft (bar chart to show all)
• fare change over time for X routes (timeseires)
• expedia search over time on X routes (timeseries)
• general seasonality analysis/anomaly detection
• see if amount of search correlates to price increase/decrease

[chriskuz]

Data Cleaning:

• address and drop nulls
• look for string errors in equipments and airline names (check everything by doing a distinct count)
• remove empty strings on some columns
• consider removing seconds since datetime objects could generate length of trip
• check for reasonable dates
• dummy some categorical columns (selected features for modeling)
• create functions for handling additional features