CPU usage (OSX)

[3rd3]

Is it normal that Python runs at 75% CPU when I move the mouse?

[ljos]

75 sounds a bit high, but depending on what hardware you have that doesn't sound impossible. The problem is probably that we check which window you are in for every event that comes in and mouse moving sends a lot of events. I have been looking for a better solution to this, but have down prioritised it because it hasn't been that much of a problem for me.

Thanks for bringing this back to my attention.

[ljos]

I also notice that it goes pretty high if you type really fast as well.

[3rd3]

One solution would be to check only every x movement for the window on mouse move events. I wouldn’t mind a bit fuzziness in the mouse position records. Also it is more likely that the front window changes on mouse-down and keyboard events.

[ljos]

I am trying to get a profiler running, but it doesn't look like cProfiler and profiler doesn't play nice with the objC api, or it is just me doing things wrong...

I just realised though that it cannot just be the sniff_cocoa handler that is the result of the high cpu usage. There must be something else because I changed some code to be a bit more efficient and it didn't do that much. I would like to profile the code to see if it is this part that is the real problem or if we have something else that should be optimised first.

[gurgeh]

The same thing happens on Linux, so it is not a problem with sniff_cocoa. I am sure that we can lower CPU usage. Perhaps not by only checking every x movement, but by only checking every second or so.

[knarf-se]

Hello, I wanted to chime in here too. I've noticed that selfspy hogs my CPU from time to time, and it does not seem to correlate to mouse movement only (I did a test whizzing my mouse all over the screen, and indeed the CPU usage of selfspy skyrocketed!).

Currently it only irritates me when it steals CPU cycles from Noise so my music stutters other than that, I beleive that this is really cool! ;-)

[ljos]

Thanks for reminding me of this.

I managed to do some initial profiling a little while ago. (Not the easiest task in OS X because of the ObjC bridge.) The problem I am seeing is that the serialisation to sqlite is pretty slow. I don't believe this is a problem with sqlite, but with the ORM stuff and sqlalchemy. We should look into how we can use sqlalchemy more efficiently or lose the ORM stuff and use the low level sqlite API's.

I don't think we need to limit the logging before we have checked how we can remove the overhead in serialising to the database.

[knarf-se]

Hey, That was quick! (^_^ )

I also guessed that it has to do with the data serialization, I will try to run selfspy with selfspy -d /run/shm/spy to see if I get any improvements that way (It won't write any data to the harddrive).

[rdslw]

Just to report: problem still persists on newest OSX 10.9.4 and newest selfspy taken from github. Mouse movement spikes usage to up to 40% here, while typing on kbd to about 20%. After a day, in terms of total CPU time consumption, Python process on which selfspy runs, is second biggest CPU hog on my mac :-(

[ljos]

I am not actively working on this anymore, and it will be some time before I will; if ever. I have other projects and this one is very low priority. I think there are a lot of people who will be grateful if someone would work on this.

[ranman]

I'm looking into this over the next few weeks. Upgrading to the latest sqlite package and driver was a slight performance gain. I think there are some options for improving speed still and I'll update this if I find anything.

[rdslw]

Maybe we should just write it to the sqlite in batches, every 60 seconds. If sqlite is the problem here which I dont know.

[leafo]

Not entirely sure if this is the same issue on OSX, but I was having 100% CPU issues on linux. I have a 1.9 GB selfpsy database if that's related at all. I noticed that got_screen_change was being called for every event that happened, and it ran a decent amount of queries. Events were being created faster than selfspy could process them so it would get stuck processing all the events at 100% CPU. If I typed/moved mouse a bunch then the processing could take a whole minute after I stopped using the computer :/

I made got_screen_change return immediately if it's called with the same arguments as last time. It seems to be a safe change as it looks like the body of the method only does anything important if values are being updated.

This solves my 100% CPU problem, but it still has pretty high CPU when actively doing things, around ~15%, so I'm sure there are more optimizations to be done.