python-attrs / cattrs

Composable custom class converters for attrs, dataclasses and friends.
https://catt.rs
MIT License
781 stars 108 forks source link

Avoid linecache unique file generation in case of linecache disabled #461

Closed sahu-sunil closed 7 months ago

sahu-sunil commented 7 months ago

When cattrs' converter/hooks or same classes are created too frequent or in large amount, generate_unique_filename (generation of unique file with uuid) becomes slow/inefficient even if linecache flag is disabled. This PR executes this function only when cache is enabled. More optimisation can be done for compilation of same classes repeatably, as mentioned in below issue. Issue discussion: https://github.com/python-attrs/cattrs/issues/445

Tinche commented 7 months ago

@sahu-sunil cool, I can merge this in. Does it fix your actual problem though? You can install from this branch and test.

If not I can try applying more optimizations.

sahu-sunil commented 7 months ago

Yes it was bit helpful but in long duration CPU is still gradually increasing. I am confident that this is due to linecache left enabled somewhere hidden place. I am going to try cattrs with linecache False by default instead of True (current behaviour).

Yes, please check if you see some scope of optimisation on repeated compilation

Tinche commented 7 months ago

@sahu-sunil Ok, let's merge this in and I can take a look at a bigger optimization.

Can you just add a short line to HISTORY.md?

Tinche commented 7 months ago

Thanks, I'll ping you when I have something more.

sahu-sunil commented 7 months ago

Yes it was bit helpful but in long duration CPU is still gradually increasing. I am confident that this is due to linecache left enabled somewhere hidden place. I am going to try cattrs with linecache False by default instead of True (current behaviour).

I tried with this locally and the cpu is stable now. Will see if I can find all the places where linecache can be disabled.

I know it won't be feasible to reverse on cattrs level, like I did locally