Reliability / Increase in month revenue achievement from potential revenue to 75% and Atlas embedding revenue contribution

[Vidiskiu]

Description

We previously measured that revenue management efforts are able to achieve around 64.23% of the in month revenue potential for the month of April 2024.

There is a big difference in decision count between RandomForest and ATLAS Embedding DNN v2, this is because ATLAS Embedding DNN always get restricted due to price change threshold of 15% or 20%, and below minimum rate price. This caused a lot of revenue to be directed to ATLAS RandomForest just because it gets more chances and executions.

This is also applicable to MARS since price prediction beyond treshold is also not logged in pricing_decision table.

Problem

Root Causes:

• If we remove all the restrictions on ATLAS Embedding DNN, we couldn't ensure 100% reliability that the model will always predict price within common sense to the MA ADR, because the MA ADR in the first place is wrong (read below)
• A lot of decisions from ATLAS Embedding DNN and MARS does not go through the send stage, causing prices to be outdated.
• MA90 ADR per Unit sometimes average bookings between high end unit type and low end unit type together causing the MA90 ADR per unit to be most of the time higher or sometimes lower than common price of the room_id.
• Lack of explainability is making it harder to justify the model's predictions
• We notice that some property in potential revenue list such as Bong Hostel has a lot of unnoccupied rooms despite alteration of prices.

Solutions

1. If pricing prediction > treshold, we can adjust final price to be exactly the treshold value, ensuring pricing decisions to be updated and more contribution from respective models.
   • Move validation of pricing such as minimum rate and out of bound treshold to process stage instead of model/send stage.
   • Ensure that data logging in pricing_decision table has been completed in the process stage.
   • Ensure that we are able to log how many pricing decisions ave been pushed through this method.
2. Update feature collection for MA ADR to accomodate the following
   • Recognize the seasonality of the NAB dates and get the ADR from the same seasonality group.
   • SUM the ADR according to the uint type aand quantity to get a better MA ADR value
3. Reduce cases of gateway timeout that causes inference to be empty and no decision made.
4. After the number of pricing prediction > treshold as been significantly reduced to <50/day, we can make escalation status = 1 to avoid it being sent, but it requires manual intervention from pricing ops.

Measurement metrics

• Dashboard Link
  • Daily Decision Count
  • Logs Count
  • Logs Classification Count
• Revenue contribution dashboard

Current measurements:

Period of 2024-04-01 - 2024-04-29

• Achieved Revenue potential: 68.38%
• Revenue contribution of Atlas DNN: 26.5%
• Revenue contribution of MARS: 25.08%
• (Optional) Out of bound decision by Atlas DNN: 50.7%
• (Optional) Out of bound decision by MARS: 28.6%

SLA

RM:

• [ ] Achieved Revenue potential by RM Team: 80% (with adjustment of Bong Hostel, etc)

ATLAS:

• [ ] Revenue contribution of Atlas RandomForest: 25%
• [ ] Revenue contribution of Atlas RandomForest from total potential revenue: 20%
• [ ] Revenue contribution of Atlas DNN: 25%
• [ ] Revenue contribution of Atlas DNN from total potential revenue: 20%
• [ ] (Optional) Supposed of bound decision by Atlas DNN: 30%
• [ ] (Optional) Out of bound decision by Atlas DNN: 5%

MARS:

• [ ] Revenue contribution of MARS: 25%
• [ ] Revenue contribution of MARS from total potential revenue: 20%
• [ ] (Optional) Supposed of bound decision by MARS: 15%
• [ ] (Optional) Out of bound decision by MARS: 5%

---

Updates:

• 2 May: Added modifications to these metrics by agreement & discussion with Bella + Jing:
  • Achieved Revenue potential by RM team: 75% (without Bong Hostel adjustment)-> 80% (with Bong Hostel adjustment)
  • Revenue contribution of Atlas DNN 35% -> 25%
  • Revenue contribution of MARS 30% -> 25%
  • Revenue contribution of Atlas DNN from total potential revenue: 20%
  • Revenue contribution of MARS from total potential revenue: 20%
  • Added ATLAS RandomForest metrics
  • Deadline modified due to additional discussions, findings, requests by RM which led to the tasks that haven't been started: 27 May -> 30 May 2024

[Vidiskiu]

Overall Point: 6.6

Functional Complexity: 1.2

Adjusting revenue management and embedding system involves intricate calculations and decision-making processes, indicating high functional complexity.

Technical Complexity: 1.5

UI/UX Complexity: 0.4

UI/UX changes may be minor, primarily impacting dashboards for monitoring and potentially some internal tooling interfaces.

Data Manipulation: 1

Extensive data manipulation is necessary to adjust and log pricing decisions, as well as restructuring the ADR feature collection.

Testing: 0.5

Thorough testing is crucial due to the financial implications of any errors in the revenue management system.

Dependencies: 0.5

There are likely dependencies on existing data pipelines, models, and possibly external data feeds for seasonality parsing.

Risk and Uncertainty: 0.5

High stakes are involved with potential revenue impacts and model reliability, creating elevated risk and uncertainty.

User Impact: 1

Significant user impact due to direct effect on revenue achievement, decision models' contribution, and operational accuracy.