0todd0000
commented
3 years ago Two of the three paired comparisons show differences in this ~80-90% region with one of the paired comparisons showing an additional significant region from ~20-40% of the gait cycle.
As a quick check: have you applied a correction to the post hoc comparisons? For example: critical p = 0.05/n instead of 0.05, where n is the number of post hoc tests? If you have not applied a correction like this, please apply one and check whether the post hoc results still yield a result over the 20-40% region.
My main question to the forum is if anyone has had any feedback or reviewer comments about this type of reporting?
I personally haven't received any reviewer comments about post hoc reporting.
Translating from more traditional discrete (0D) analysis, a lack of a significant main effect in an ANOVA would usually lead to not looking at paired comparisons, so I could see an argument that I can not look further than the region flagged in the ANOVA. So am I justified in looking at the whole gait cycle in paired comparisons when there is only a significant result for the ANOVA in a proportion of it?
Yes, I think it's fine to look at the whole gait cycle provided the results do not contradict the original ANOVA results. If they do contradict the ANOVA results, then an explanation would be required. This explanation could involve the consideration of various factors including local vs. global characteristics like smoothness, variance, normality and outliers.
Perhaps the most general approach would be: (a) regard the ANOVA result as the main result, and (b) regard post hoc results simply as qualifications of the main ANOVA result. From this perspective, you can do anything in post hoc analysis, provided you do not contradict the main ANOVA results. In this case I think it would be sufficient to state that the observed effect was relatively large in the 20-40% region, but it was insufficiently large to yield significance at the ANOVA level.
kane-middleton
Hi all,
I have a similar issue to #6. However in my particular case, I have a one-way RM ANOVA with three levels that shows significance at ~80-90% of the gait cycle. In the same manner as 0D analysis, I have made an a priori decision to use any significance in the ANOVA as justification to then conduct post hoc tests to look at paired comparisons between the three levels. Two of the three paired comparisons show differences in this ~80-90% region with one of the paired comparisons showing an additional significant region from ~20-40% of the gait cycle. The F statistic for the ANOVA clearly goes towards the critical F in this region but does not breach it. From #6 I can see how this might be statistically, given Todd's responses.
My main question to the forum is if anyone has had any feedback or reviewer comments about this type of reporting? Translating from more traditional discrete (0D) analysis, a lack of a significant main effect in an ANOVA would usually lead to not looking at paired comparisons, so I could see an argument that I can not look further than the region flagged in the ANOVA. So am I justified in looking at the whole gait cycle in paired comparisons when there is only a significant result for the ANOVA in a proportion of it?
Thanks,
Kane.