Double check correct back-transformation process for yi

[egouldo]

I would like to double-check that the transformation of the analyst's submitted values onto the response scale has been calculated correctly.

I have put together a file with the following variables:

• "estimate_type" y25, y50 or y75 for 25th, 50th and 75th percentile scenarios respectively

• "response_id" as per Combined Master

• "submission_id" as per Combined Master

• "analysis_id" as per Combined Master

• "split_id" as per Combined Master

• "TeamIdentifier" as per Combined Master

• "id_col" - merges the TeamIdentifier with response, submission, analysis and split ID's

• "response_transformation_description" - As per Combined Master: If 'transformed' indicated under 'response_transformed', this column indicates the mathematical transformation

• "response_transformation_status" - As per Combined Master: If 'transformed' indicated under 'response_transformed', this column indicates the mathematical transformation unless the reported estimates are back-transformed, in which case, that is indicated by 'back.transformed'

• "response_variable_type" constructed vs. original, As per Combined Master

• "response_construction_description"

• "response_variable_name"

• "transformation" of the response variable

• "construction" of the response variable if not original

• "link_function_reported" link function reported by the analyst

• "adjusted_df" - As per Combined Master

• "transformation_type" - this is a constructed variable that

• original values supplied by the analyst (variables ending with _original)

• the transformed values onto the response scale (variables ending with _response)

  • fit: The yi estimate
  • se.fit Standard error of the yi estimate
  • ci.low / ci.high 95% CI's

Some information on how the response-scale estimates are calculated

1. We assign a transformation_type to be applied to the estimate using this code
2. Depending on the transformation_type different *_back() transformation functions are applied the estimates. The code that assigns which function to use is here
3. All *_back() transformation functions take the value and the se to then simulate a distribution of length sim from which the mean, se, and 95%CIs are returned. You can see the code for each function

Tasks for @parkerth :

• [ ] Check that the value of transformation_type is correct. This determines what back-transformation function is applied to the analysts estimates (ending with _original), the logic is:

Where $link_fun and $response use the value for that analysis in those columns for transformation_type, i.e. the analyst submitted value.

• [ ] Check that the code assigning the back-transformation function is correct, i.e. this code
• [ ] Check that the back-transformation calculation code is correct, i.e. we're correctly back-transforming to the response-scale? Functions are here

There could be two explanations for getting extreme values:

1. The analyst has told us the wrong information about the link function, response construction/transformation, or the wrong adjusted degrees of freedom.
2. Our _back transformations are wrong.
3. Something else? The analysts' estimate uncertainty is just huge and therefore when we transform we're getting crazy distributions -> crazy back-transformed estimates?

Here's the file: checking_back_transformation.csv

Originally posted by @egouldo in https://github.com/egouldo/ManyAnalysts/issues/272#issuecomment-1679839449

[egouldo]

Just adding a note to remind that we agreed this wasn't necessary to fix before distributing the draft manuscript to analysts.

[parkerth]

I will try to get to this later today.

[egouldo]

Thanks! And sorry I just re-found this issue in the old repo!

[parkerth]

The links in this issue don't work, so I can't check what's going on. That said, is this in reference to the Eucalyptus data that we ultimately decided to delete the many extreme outliers? I think we may already have dealt with this.

[hannahsfraser]

ping @egouldo This one looks resolved?

[egouldo]

Agreed, suspect this was in relation to the extreme yi values, closing.