Question about starting values - Lab 2

[Mickiboyhekkens]

Hi Sander,

I had a question regarding setting up the model in lab 2, Question 1:

mu = Beta('mu', -1, None, None, 0) B_tc = Beta('b_tc', -1, None, None, 0) sigma = Beta('sigma', 1, None, None, 0)

How did you arrive at these initial values (-1, -1, 1)? Can we assume that for the exam, we can copy these values from another model, or is there a method to determine these values accurately?

Best regards,

Mick Hekkema

[sandervancranenburgh]

Hi Mick, Excellent question. The initial values determine where the optimisation algorithm starts on the likelihood plane. For MNL models, the LL plane is globally concave. This means that wherever you start, you will always find the maximum of the LL. With MNL models, it is, therefore, common practice to start all betas at zero.

But more complex models, like the panel ML model, often have LL planes which are not globally concave. This means the optimisation might get stuck in local optima. If you suspect it got stuck, or if the optimisation just fails because it is on the 'wrong' path, then you can try a different starting point. In practice, you want to start at a 'good' point. That is, a point where the LL exists (thus is not -inf) and which is not too far from where you expect your maximum likelihood to occur. Thus, starting B_cost at -1 helps the optimisation because you are starting already in the negative domain (which is where you expect it to be). Likewise, the mu of -1 and sigma of 1, mean that your initial model will imply a VTT of exp(-0.5) = 0.6 euro per minute, which is fair. I hope this answers your question. Sander

[allis103]

Hi! a quick follow up on this. What would push the starting values to change ?

For instance, how come in 4.1, the vtt and B_tc are set as:

# Define model parameters
vtt  = Beta('vtt',     1, None, None, 0)
B_tc = Beta('B_tc', -0.1, None, None, 0)

and in 4.3 the parameters are defined as follows :

# Parameters definition enabling the construction of random parameters
vtt = Beta('vtt',    0.1, None, None, 0)
B_tc = Beta('b_tc', -0.1, None, None, 0)    
sigma_vtt = Beta('sigma_vtt', 1, None, None, 0)

The same goes with 5.2:

# Parameters definition enabling the construction of random parameters
vtt       = Beta('vtt',       0.4, None, None, 0)
B_tc      = Beta('b_tc',     -0.4, None, None, 0)    
sigma_vtt = Beta('sigma_vtt ',  2, None, None, 0)

What is the logic behind setting the starting values like that ?

[sandervancranenburgh]

I'm afraid there is not so much higher-order math behind this :)

The model in 4.1 will 'always' converge as long as you do not start the vtt at zero. So, I set vtt to one.

But that was too high (see the estimation results). B_tc of -0.1 was indeed close the final estimate Therefore, in the next model (4.3), I lowered vtt by a factor 10, while keeping b_tc at -0.1.

Finally, for model (5.2), the starting point was again set closer to the expected value. This time vtt is set to 0.4. I can't remember the logic of why I also increased B_tc and sigma. Especially setting sigma to 2 seems not a great choice (seeing the estimation results). Keeping sigma = 1 makes more sense.

Anyway, suboptimally chosen starting points may result in longer than necessary estimation times, but as long as the optimiser does not get stuck in a local solution or fail entirely, it is harmless. Sander

[ftamsma]

Hi Sander,

I also have a follow up question on this topic, how do you know where the optimiser (might) get stuck? Like that you said above, that for VTT = 0 the model will not converge. But do you know this before hand or could you also run it, see that there is no output and then change it?

[sandervancranenburgh]

Ha Femke,

This part of the modelling is a bit of trial and error. There is no way to know upfront what will be good starting points. Based on my experience, I suspect that vtt = 0 will cause issues, but perhaps not. You can simply try it and see what happens. If the model produces very large standard errors or a very poor model fit (e.g. rho_sq = 0.01) then something is wrong. In general, MNL models are very robust towards the modeller’s choice of the starting values. Mixed logit models with normally distributed random parameters are usually also not sensitive to the starting values. Mixed logit models with more exotic/flexible distributions are usually sensitive to the starting values. In that case, the modeller needs to estimate the model numerous times, using different sets of starting values.

I hope this answers your question.

Sander

From: FemkeTamsma @.> Reply to: SEN1221TUD/Q2_2023 @.> Date: Friday, 1 December 2023 at 13:33 To: SEN1221TUD/Q2_2023 @.> Cc: Sander van Cranenburgh - TBM @.>, Comment @.***> Subject: Re: [SEN1221TUD/Q2_2023] Question about starting values - Lab 2 (Issue #10)

Hi Sander,

I also have a follow up question on this topic, how do you know where the optimiser (might) get stuck? Like that you said above, that for VTT = 0 the model will not converge. But do you know this before hand or could you also run it, see that there is no output and then change it?

— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https:/github.com/SEN1221TUD/Q2_2023/issues/10*issuecomment-1836045733__;Iw!!PAKc-5URQlI!5sFvIjICsqzDrHsl2kjfPTAHMgUOPv5ilsYp-FcK8uDtqA8KMJratsJ6m8t5e1CAko-mvERTqrTRcScVUMr8chTkY1uHrfDg$, or unsubscribehttps://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/AV3Z2J4XM2PAZG46WQQDYVDYHHFAZAVCNFSM6AAAAAA77NRAI2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMZWGA2DKNZTGM__;!!PAKc-5URQlI!5sFvIjICsqzDrHsl2kjfPTAHMgUOPv5ilsYp-FcK8uDtqA8KMJratsJ6m8t5e1CAko-mvERTqrTRcScVUMr8chTkY-FPSN0d$. You are receiving this because you commented.Message ID: @.***>

[RoosWentges]

Hi Sander,

I have a question regarding the sigma_vtt = 2 in 5.2. You said that it is better to set the starting value to 1. So i did that. But the answers are then different. The value of sigma_vtt will then become positive and the WTP will change from 18.71 to 18.79. So what is the right answer and when do you know it the 'right' answer?

Kind regards,'

Roos

[sandervancranenburgh]

Hi roos,

Both are correct. The slight difference in the vtt is due to the fact that the integral are approximated.

The normal distribution is symmetric. So a positive or negative sigma is all the same.

I hope this clarifies your questions.

Best

Sander

---

---

From: RoosWentges @.> Sent: Tuesday, December 5, 2023 3:43:53 PM To: SEN1221TUD/Q2_2023 @.> Cc: Sander van Cranenburgh @.>; Comment @.> Subject: Re: [SEN1221TUD/Q2_2023] Question about starting values - Lab 2 (Issue #10)

Hi Sander,

I have a question regarding the sigma_vtt = 2 in 5.2. You said that it is better to set the starting value to 1. So i did that. But the answers are then different. The value of sigma_vtt will then become positive and the WTP will change from 18.71 to 18.79. So what is the right answer and when do you know it the 'right' answer?

Kind regards,'

Roos

— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https://github.com/SEN1221TUD/Q2_2023/issues/10*issuecomment-1840932296__;Iw!!PAKc-5URQlI!7TJPM5da-u881aJk5jNP3CcUAyu2K16ra3swxBfnolo1uh1C8IB3MoC9uireYLFrau_DqW4FJoFvWC6K5oeS7ZA_fEliRKDx$, or unsubscribehttps://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AV3Z2J2OBB2YWBTIN4Y4CHDYH4XKLAVCNFSM6AAAAAA77NRAI2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNBQHEZTEMRZGY__;!!PAKc-5URQlI!7TJPM5da-u881aJk5jNP3CcUAyu2K16ra3swxBfnolo1uh1C8IB3MoC9uireYLFrau_DqW4FJoFvWC6K5oeS7ZA_fFlPs3Gh$. You are receiving this because you commented.Message ID: @.***>