mattwigway / DiscreteChoiceModels.jl

Discrete choice/random utility models in Julia
MIT License
13 stars 3 forks source link

Precalculate and autostandardize variables #26

Open mattwigway opened 10 months ago

mattwigway commented 10 months ago

Sometime we run into weirdness with scaling—if one variable has a very different scale from another, the optimizer struggles, the exponentiated utilities may become infinite, &c., &c. We should consider auto-standardizing all variables as part of data preparation.

Relatedly, you can put just about any Julia code you want in the model specification. It would be useful to precalculate all parts of the utility function that don't depend on the coefficients (e.g. a square term, or a fixed effect with ==) as part of the standardization process, so that doesn't have to happen every time the utility function is evaluated. This could give a significant performance boost, particularly if there are allocations involved in calculating these things. Potentially could even dispense with dataframes entirely and put everything in a Float64 matrix.