[Core] High CPU usage on Raspberry Pi Zero W after the switch to async-std – needs profiling

[eigenein]

1. Exactly one of my-iot named processes: ~75%
2. async-std/runtime thread: ~25%

[eigenein]

It's not the web server.

[eigenein]

Could be futures::channel::mpsc, see also #28.

[eigenein]

Try async_std::sync::channel.

[eigenein]

Still uses 100% CPU 👎

[eigenein]

Alright, this seems to fix the 100% CPU load:

diff --git a/src/main.rs b/src/main.rs
index ab8b7fa..a123a8f 100644
--- a/src/main.rs
+++ b/src/main.rs
@@ -38,7 +38,7 @@ async fn main() -> Result {
     info!("Starting services…");
     let mut bus = Bus::new();
     core::db::tasks::spawn(db.clone(), &mut bus);
-    services::db::Db.spawn("system::db".into(), &mut bus, db.clone());
+    // services::db::Db.spawn("system::db".into(), &mut bus, db.clone());
     services::spawn_all(&settings, &opts.service_ids, &mut bus, &db).await?;

     if !opts.no_web_server {

[eigenein]

Okay, surprisingly this is it:

    /// Selects the database size.
    pub async fn select_size(&self) -> Result<i64> {
        // language=sql
        const QUERY: &str = r#"
            -- noinspection SqlResolve
            SELECT page_count * page_size as size FROM pragma_page_count(), pragma_page_size()
        "#;
        Ok(query_scalar(QUERY)
            .fetch_one(&mut *self.inner_connection.lock().await)
            .await?)
    }

[eigenein]

Seems to be the fetch_one issue again, this is a fix:

diff --git a/src/core/db.rs b/src/core/db.rs
index 2ae3731..a4ffb27 100644
--- a/src/core/db.rs
+++ b/src/core/db.rs
@@ -155,9 +155,11 @@ impl Connection {
             -- noinspection SqlResolve
             SELECT page_count * page_size as size FROM pragma_page_count(), pragma_page_size()
         "#;
-        Ok(query_scalar(QUERY)
-            .fetch_one(&mut *self.inner_connection.lock().await)
-            .await?)
+        Ok(*query_scalar(QUERY)
+            .fetch_all(&mut *self.inner_connection.lock().await)
+            .await?
+            .first()
+            .unwrap())
     }

     /// Selects the specified sensor.

[eigenein]

The issue is still there on the Raspberry, but on the MacBook it's only <1% CPU.

[eigenein]

May be related to #146, check the database sensors again.

[eigenein]

The current average is ~50% CPU load, which is too much for I/O-bound tasks. Also, a lot of these warnings:

[WARN] my_iot::core::db::tasks: Upserted in 2.4s > 1.0s.

[eigenein]

Indeed, upsert_message_to consumes this much CPU, but I don't know why.

[eigenein]

Upserts are much faster without the INSERT INTO sensors … ON CONFLICT (pk) DO UPDATE SET part.

[eigenein]

REPLACE INTO is much faster than ON CONFLICT (pk) DO UPDATE SET 🤔

[eigenein]

The weird thing is: once restarted it takes around 30-50% CPU – still to much, but it increases permanently up to 100% once My IoT serves the very first HTTP request, e.g. /sensors/youless::power/json. And this is because of the async-std/runtime thread, not the web server one. 🤷‍♂️

[eigenein]

High chance this is because of https://github.com/launchbadge/sqlx/issues/616

[eigenein]

Looks fine at the moment.