Implementation of Markov polymorphism model with 5 states and 8 paths

[26pan]

Hi Dr. Jackson,

Thank you for all your efforts in making multi-state modeling so accessible for the user.

I encountered a problem when using msm. The results of the Markov polymorphism model of 4 states and 5 paths I calculated in Time Figure 1 were very good and close to those calculated by cox proportional risk model. However, when I added a state (see Figure 2), that is, five states and eight paths, the msm package showed that it could not be fitted, but the results calculated by cox proportional risk model were better, which means that I must have some problems. I would like to consult you. My initial suspicion is that it is the data format or structure, because I just forgot to add the observation cutoff time before Figure 1.

For Figure 2, I named the five states 1, 2, 3, 4, 5, and set eight paths, i.e 1-2; 1-3; 1-4; 1-5; 2-4; 2-5; 3-4; 3-5; 4-5; I have tried to convert probability Q matrix with different values and the result is the same, but it cannot be fitted. According to previous experience, converting probability Q matrix has little effect on the result. Therefore, at present, I have a big doubt about what makes Figure 2 unable to be successfully implemented like Figure 1. Please give me your valuable suggestions.

[chjackson]

What do you mean by "cannot be fitted" - a particular error message, or did it fit the data badly? Also what do you mean by "converting probability Q matrix" - different initial values? If there was an error that indicated convergence failure, then that is usually because the data do not give enough information about the parameters that you are trying to estimate.

[26pan]

What do you mean by "cannot be fitted" - a particular error message, or did it fit the data badly? Also what do you mean by "converting probability Q matrix" - different initial values? If there was an error that indicated convergence failure, then that is usually because the data do not give enough information about the parameters that you are trying to estimate.

Thank you for your reply ."cannot be fitted" means convergence failure, as shown in the figure.I have questions about this statement "the data do not give enough information about the parameters that you are trying to estimate".The weird thing is that if I combine 2 and 3 of the five states 1, 2, 3, 4, 5 into one state, that is, 1, 2, 3, 4, it works without failing to converge. There will be five states when there is no convergence. The above comparison makes me very confused about what part of the information my data does not provide so that "msm" cannot successfully converge.

Thank you again for your positive response！

[26pan]

What do you mean by "cannot be fitted" - a particular error message, or did it fit the data badly? Also what do you mean by "converting probability Q matrix" - different initial values? If there was an error that indicated convergence failure, then that is usually because the data do not give enough information about the parameters that you are trying to estimate.

![Uploading bb720a3255f325b46d301bb22fc7d1b.jpg…]()

[26pan]

What do you mean by "cannot be fitted" - a particular error message, or did it fit the data badly? Also what do you mean by "converting probability Q matrix" - different initial values? If there was an error that indicated convergence failure, then that is usually because the data do not give enough information about the parameters that you are trying to estimate.

![Uploading bb720a3255f325b46d301bb22fc7d1b.jpg…]()

[chjackson]

If you add more states, you are also adding more transition rates. So you are increasing the number of parameters. If you also have covariates, then you are also adding extra parameters representing the effects of covariates on the transition rates. All these new parameters will need new information to be able to estimate them.

With intermittently-observed data and continuous-time models, it is hard to tell where information might be weak. For example, a continuous-time transition rate from state 2 to state 3 may be informed by all observed transitions out of state 2, and transitions into state 3, not just transitions between 2 and 3 over an interval. I can only advise that you explore lots of alternative models to try to find out what is happening.

To diagnose weak information, it might help to use fixedpars to fix particular parameters at their initial values. If it converges with a particular parameter fixed, but fails to converge with that parameter un-fixed, then that is evidence that that parameter is weakly informed by the data.

[26pan]

If you add more states, you are also adding more transition rates. So you are increasing the number of parameters. If you also have covariates, then you are also adding extra parameters representing the effects of covariates on the transition rates. All these new parameters will need new information to be able to estimate them.

With intermittently-observed data and continuous-time models, it is hard to tell where information might be weak. For example, a continuous-time transition rate from state 2 to state 3 may be informed by all observed transitions out of state 2, and transitions into state 3, not just transitions between 2 and 3 over an interval. I can only advise that you explore lots of alternative models to try to find out what is happening.

To diagnose weak information, it might help to use fixedpars to fix particular parameters at their initial values. If it converges with a particular parameter fixed, but fails to converge with that parameter un-fixed, then that is evidence that that parameter is weakly informed by the data.

Thank you for your reply .

[26pan]

If you add more states, you are also adding more transition rates. So you are increasing the number of parameters. If you also have covariates, then you are also adding extra parameters representing the effects of covariates on the transition rates. All these new parameters will need new information to be able to estimate them.

With intermittently-observed data and continuous-time models, it is hard to tell where information might be weak. For example, a continuous-time transition rate from state 2 to state 3 may be informed by all observed transitions out of state 2, and transitions into state 3, not just transitions between 2 and 3 over an interval. I can only advise that you explore lots of alternative models to try to find out what is happening.

To diagnose weak information, it might help to use fixedpars to fix particular parameters at their initial values. If it converges with a particular parameter fixed, but fails to converge with that parameter un-fixed, then that is evidence that that parameter is weakly informed by the data.

I would like to ask one more question, I found this paragraph in your recommended guide (pictured), would the mstate package be more suitable for temporal survival data than the msm package?

[chjackson]

If by "temporal survival data" you mean data where the exact times of all events / transitions are known, so that you know the state of the process at all times, then, yes, I would prefer the packages mentioned here. msm is based on the strong assumption of constant (or piecewise-constant) transition intensities. These other packages allow this assumption to be relaxed so that the transition intensities are arbitrarily-flexible functions of time.

[26pan]

Thank you very much!

---Original--- From: "Chris @.> Date: Thu, Feb 29, 2024 16:47 PM To: @.>; Cc: @.>;"State @.>; Subject: Re: [chjackson/msm] Implementation of Markov polymorphism model with5 states and 8 paths (Issue #96)

If by "temporal survival data" you mean data where the exact times of all events / transitions are known, so that you know the state of the process at all times, then, yes, I would prefer the packages mentioned here. msm is based on the strong assumption of constant (or piecewise-constant) transition intensities. These other packages allow this assumption to be relaxed so that the transition intensities are arbitrarily-flexible functions of time.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you modified the open/close state.Message ID: @.***>