search

snake-charmer-devs / snake-charmer

A self-contained Python workbench for scientific programming, data mining, maths, stats and visualization
66 stars 10 forks source link

Multiple issues with scikit-learn #12

Closed andrewclegg closed 10 years ago

andrewclegg commented 10 years ago

https://gist.github.com/andrewclegg/e3060f4e49e04ee66dac

Planning to upgrade to a newer build and see where that gets me.

Raise scikit-learn ticket if this doesn't help.

andrewclegg commented 10 years ago

They are almost all due to sys.modules changing size while clean_warning_registry iterates over it.

And we previously had no failures, with the same release of sklearn.

I think this is a race condition which only shows itself under heavy load. We can probably get around it by patching clean_warning_registry to iterate over a shallow copy of sys.modules, if it rears up again.

andrewclegg commented 10 years ago

See also http://blog.labix.org/2008/06/27/watch-out-for-listdictkeys-in-python-3

andrewclegg commented 10 years ago

All tests pass when running on its own, so I haven't introduced any nasty regressions. About to test on fully-loaded server.

andrewclegg commented 10 years ago

Works fine under load too. Pull request sent (https://github.com/scikit-learn/scikit-learn/pull/3196). Will keep this open for tracking, til the pull's accepted. Snake Charmer will continue to install off my fork til then.