2021-10-29 meeting overview - information for ABM extension

[MichalisPanayides]

What has been done:

• Introduction ✔️
• Conclusion ✔️
• Addressed several TODOs
• Overview of ciw state dependent distributions
• Began PGR-Talk: How to make awesome package in python

To de discussed:

• Paper
  • [x] Using \mathcal{A}_i for arriving state
  • [x] \ref{} revisited
  • [x] Blocking time closed form
• ABM plan
  • Step 1: State dependent ciw service node
    • For every state have a different service rate
  • Step 2: Server dependent ciw service node
    • For every server AND for every state have a different service rate
    • Server's utility: U = a(#served) + (1-a)(utilisation)
      • Why is this the server's utility?
      • A server would want to maximise that?
  • Step 3: Reinforcement Learning algorithm
    • Use basic RL to self optimise U
    • Service rates now becomes an output (we still not to give some arbitrary policy right?)
    • How the Basic RL would work:
      • Start with an arbitrary state dependent service rate for each server
      • Before every service (of each server) there is a probability p of changing policies
      • If value improves accept new policy otherwise keep the old one
  • Q&A
    • [x] When we change a policy do we change just change μ_ij of a specific server?
    • [x] When we say policy we mean a server's service rate for all states or just for one (μ_ij)?
    • [x] Is the probability p going to be fixed?
    • [x] RL will basically work while the simulation is running?
    • [x] Code: server_behaviour_simulation.py

To be done:

• Game Paper
  • Finish final todos
• Write PGR-Talk
• ABM Plan
  • Step 1
  • Step 2
  • Step 3

[MichalisPanayides]

Two ways to look at the problem:

• Use M/M/c queue and hijack the server node
• Model M/M/c queue as multiple parallel M/M/1 queues

Check how to get server id in ciw

• Each server could have its own id
• If not possibly a PR that just stores server id in Q.data_records()

Utility function

• α (#served) + (1-α) (idle time)
• Maximise the number of people served
• Minimise each server's utilisation (maximise idle time)
• α is different for every server
• The utilisation function can be different for every server

Other

• Write every step mathematically first and then code it
• α will always be an input
• p will always be an input (and will be a given fixed parameter)
• Look in the literature for team make-up in Healthcare
• Changing policy means we only change μ_{ij} for one server (some may never be visited)
• Decide how μ_{ij} will change

Look at picture of whiteboard in PhD folder on phone

[MichalisPanayides]

Reinforcement learning videos: https://deepmind.com/learning-resources/reinforcement-learning-series-2021

[MichalisPanayides]

[MichalisPanayides]