search

tk3369 / YeoJohnsonTrans.jl

Yeo-Johnson Transform
MIT License
5 stars 2 forks source link

The Yeo-Johnson Transformation for rescaling datasets is causing my optimization process to slow down. #9

Open Mehdilotfi7 opened 4 months ago

Mehdilotfi7 commented 4 months ago

Hello, I'm working on optimizing an objective function that involves rescaling my data. While log transformation is common, I've opted for the Yeo-Johnson transformation due to its ability to handle values between zero and one effectively.

However, I've encountered a challenge: the Yeo-Johnson transformation is computationally expensive, especially when iterating over multiple parameter tests during optimization.

I'm wondering if you have any suggestions for potentially reducing the running time associated with the Yeo-Johnson transformation within the optimization process.

Perhaps there are vectorized implementations available, or pre-computation strategies that could be employed?

Any insights you could share would be greatly appreciated.

Thanks.

tk3369 commented 2 months ago

Hi, I have just updated the dependency for Optim to the most recent version. I don't know if that helps but worth trying.

The code is fairly straightforward but I don't really have time to work on this. You can take a look and do some benchmarking and see if you can improve its performance. Or you can ask the community for help -- Discourse would be a good option. A lot of people there are willing to help especially anything related to performance.

tk3369 commented 2 months ago

Are you using the general transform function or would you be able to provide your own lambda parameter?