search

jasonjmcghee / rem

An open source approach to locally record and enable searching everything you view on your Mac.
https://rem.ing
MIT License
2.18k stars 60 forks source link

When rem is Remembering, mediaanalysisd spins up taking 30%-50% CPU #59

Closed Complexity closed 5 months ago

Complexity commented 5 months ago

Issue:

When rem is running, and actively Remembering a session, Apple's mediaanalysisd also starts up. It will run constantly with CPU between 30% - 50% of total usage for the entire session. (rem itself consumes around 5% CPU.) After selecting Stop Remembering in the rem menu, the process peaks briefly and then drops away.

What is expected:

Not sure...

Why is this an issue?

For an app which derives its functionality from running all day, triggering another process that consumes considerable resources might lead to unnecessary battery consumption.

Discussion

In comparison, using Quicktime to record a (full screen) capture session does not trigger mediaanalysisd. Once a Quicktime screen capture movie is recorded and available in a Quicktime window, its text is also selectable, with copy to clipboard available.

Quicktime's screencapture process hovers between 15% -20% CPU.

rem, when running, shows an Energy Impact of between 100 and 300 on my M1 MBP 14", on battery. In comparison, with Quicktime running and full-screen recording, Energy Impact for the screencapture process is around 10 - 15, an order of magnitute less.

Is mediaanalysisd doing something specific and of value for rem? Is it actively spawned, or spawned by MacOS due to the way the video is being stored by rem?

Is there anything to be learned from the way Quicktime does its screen capture that would make rem more efficient for long-term active sessions?

jasonjmcghee commented 5 months ago

This is the process that takes a screenshot and recognizes the text. It would be great to find creative ways to reduce the impact of this.

Enabling "only OCR the active window" should help, which will be released in the next release.

50 would also help reduce this. If we compared to previous screenshot and if it was identical within a threshold, don't save a new frame / re-ocr.

At the end of the day, this isn't a bug, it's core to the app being able to provide search, but unfortunate how inefficient it is.

It would be fantastic to find creative ways to reduce energy usage.

jasonjmcghee commented 5 months ago

64 helps fix this. no new release yet.

Complexity commented 5 months ago

Is it possible, then, that Quicktime's OCR only runs on the specific still paused in the Quicktime player. In that way, it offers in-the-moment copying of OCR text, which suits that use case but of course not rem's functions.

One thought that occurs would be to defer OCR for laptops until on plug-in power, then batch it—but that might create needless bottlenecks, and wouldn't serve the longer term plan to incorporate high availability RAG search.

50 would certainly reduce invoking mediaanalysisd (and an additional benefit—it would also help in reducing duplicate text fed to the RAG search).

Agree it's not a bug but a discussion. Will close with thanks and keep an eye on development. A great project!

jasonjmcghee commented 5 months ago

Haven’t tried ocr on the video after the fact. I would expect a big drop in quality. But worth trying.