Performance issues with "many" custom keyboard elements

[xzilja]

I'm working on an app that requires to display substantial amount of TextField inputs to user on a single page (think 30+ in a grid that can be visible all at once). While utalizing default keyboard and quickly entering / deleting values in each of the fields, everything works as expected.

Once I switched to using my custom keyboard via attaching .customKeyboard to each of the text fields, I started noticing big performance issues.

Opening custom keyboard on first textfield works as expected, then things start to degrade rapidly when opening custom keyboards and entering values in more and more fields. Typing becomes laggy, animations suffer.

Is there any chance that every time a custom keyboard is shown, it never gets offloaded from memory / rendering tree when another TextField is focused, thus accumulating something over time that could be causing these rendering issues?

I know swift introspect library also adds some views on top of this to perform it's functionality, maybe that might play a role here as well?

[xzilja]

Small update on this. I've made a very simple repro locally, in essence just a bunch of

TextFields(...)
.customKeyboard { _, _, _ in
    EmptyView()
}

And I'm seeing substantial performance issues that seem to correlate with amount of TextFields. With Empty view approach above, there is a big lag when navigating to the page with all these fields.

Alternatively you could probably replace empty view with a button that adds random letter to textfield and you should see some lag as well I think.

[paescebu]

@xzilja thanks for the feedback! Interesting finding. Can you provide me with an example project where you can reproduce the issue? I tried putting 30 TextFields into a VStack + ScrollView but I cant reproduce the issue you're facing.

Kind regards Pascal

[xzilja]

@paescebu Sure thing!

First of all, I might be doing something weird here with looping over state array like this, but same code works fine when removing .customKeyboard modifier.

Having trouble initializing GitHub project right now, so here are sample files after creating default project in XCode and Repro steps:

1. Basically when you open sheet for the first time (and never touch textfields) everything works as expected
2. Type something into a text field or few of them. You should notice typing lag already. Close the sheet
3. Re-open the sheet and you should see a laggy delay (I see one on simulator using m3 max)
4. Remove .customKeyboard bit and repeat same with default keyboard, there is no lag

ContentView

import SwiftUI

struct ContentView: View {
  @State private var showSheet = false

  var body: some View {
    VStack {
      Button("Show Keyboard Sheet") {
        showSheet.toggle()
      }
    }
    .sheet(isPresented: $showSheet) {
      KeyboardView()
    }
  }
}

KeyboardView

import SwiftUI
import CustomKeyboardKit

struct KeyboardView: View {
  @State private var strings = [
    "1", "2", "3", "4", "5",
    "4", "5", "6", "7", "8",
    "9", "10", "11", "12", "13",
    "14", "15", "16", "17", "18",
    "19", "20", "21", "22", "23",
    "24", "25", "26", "27", "28",
    "29", "30", "31", "32", "33",
    "34", "35", "36", "37", "38",
    "39", "40", "41", "42", "43"
  ]

  var body: some View {
    ScrollView {
      ForEach($strings, id: \.self) { $val in
        TextField("", text: $val)
          .customKeyboard {textDocumentProxy,_,_ in
            VStack {
              Button("A") {
                textDocumentProxy.insertText("A")
              }
              Button("B") {
                textDocumentProxy.insertText("B")
              }
              Button("C") {
                textDocumentProxy.insertText("C")
              }
              Button("D") {
                textDocumentProxy.insertText("D")
              }
            }
          }
          .padding()
          .background {
            RoundedRectangle(cornerRadius: 8)
              .fill(.gray)
          }
      }
    }
    .contentMargins(.all, 16)
  }
}

[paescebu]

@xzilja I think you need to guide me a bit through the issue.

I cant observe any lag or performance degradation in your example. See following video: https://github.com/paescebu/CustomKeyboardKit/assets/59558722/99cdf2be-5803-4d3f-8717-ed003de21da4

I tried to reproduce it on Simulators as well as on a physical device

[paescebu]

@xzilja A bit of a blind shot, but could you try out this branch? https://github.com/paescebu/CustomKeyboardKit/tree/bugfix/potential_memory_leak

Is this improving things for you?

[xzilja]

@paescebu

Sure, I'll also showcase by video, so it's easier to explain 😅

First things first, to make things more drastic I changed that strings array to @State private var strings = Array(1...100).map { "\($0)" } so it renders 100 inputs

Below are videos with and without custom keyboard modifier. For me, there is a drastic difference. I wonder if this issue is related to latest XCode / Swift maybe? I'm on XCode 15.3

Video with .customKeyboard { ... bit

https://github.com/paescebu/CustomKeyboardKit/assets/3154053/6161146e-1b61-4799-a9b4-0a928b229502

Video without .customKeyboard { ... bit, i.e. just native keyboard

https://github.com/paescebu/CustomKeyboardKit/assets/3154053/ded6b141-320a-428e-8152-fee330d541ba

[xzilja]

Just tried with bugfix/potential_memory_leak branch, cleaned build folder and derived data to ensure, but lag still persists.

[paescebu]

Tried many things. I doubt that theres a low hanging fruit of a fix for this one. I have to spend some time on this to figure out what I can do to improve the performance. Surely the overhead of introspecting textfields is not helping

[xzilja]

Gotcha, perhaps there is a way to somehow perform all of this compute only when keyboard needs to be / is shown? Based on input's focus state perhaps? I'll try to conditionally apply customKeyboard modifier like this today, but will likely need to be some kind of hacky way to do this.

[paescebu]

Yeah I think we're just running into a limit as introspection of SwiftUI has some overhead.

Thinking about making this a known limitation. But I'll surely investigate it a bit more tonight. But atm i'm a bit out of ideas.

[paescebu]

I don't think theres much I can do on my side, cause even if i simply add a simple UIView with a red background as a custom Keyboard, it will slow everything down. Also if i don't do anything in the introspection, and simply add the modifier to all of them it will slow down. I think the performance limitation is not something I can do much. Unless I find a more efficient way of introspecting TextFields and TextAreas without swiftui introspect.

Try following example to see what I mean:

import SwiftUI
import SwiftUIIntrospect
import UIKit

struct ContentView: View {
  @State private var showSheet = false

  var body: some View {
    VStack {
      Button("Show Keyboard Sheet") {
        showSheet.toggle()
      }
      .sheet(isPresented: $showSheet) {
          KeyboardView()
      }
    }
  }
}

struct KeyboardView: View {
    @State private var strings = [
        "1", "2", "3", "4", "5", "6", "7", "8", "9", "10",
        "11", "12", "13", "14", "15", "16", "17", "18", "19", "20",
        "21", "22", "23", "24", "25", "26", "27", "28", "29", "30",
        "31", "32", "33", "34", "35", "36", "37", "38", "39", "40",
        "41", "42", "43", "44", "45", "46", "47", "48", "49", "50",
        "51", "52", "53", "54", "55", "56", "57", "58", "59", "60",
        "61", "62", "63", "64", "65", "66", "67", "68", "69", "70",
        "71", "72", "73", "74", "75", "76", "77", "78", "79", "80",
        "81", "82", "83", "84", "85", "86", "87", "88", "89", "90",
        "91", "92", "93", "94", "95", "96", "97", "98", "99", "100",
        "101", "102", "103", "104", "105", "106", "107", "108", "109", "110",
        "111", "112", "113", "114", "115", "116", "117", "118", "119", "120",
        "121", "122", "123", "124", "125", "126", "127", "128", "129", "130",
        "131", "132", "133", "134", "135", "136", "137", "138", "139", "140",
        "141", "142", "143", "144", "145", "146", "147", "148", "149", "150",
        "151", "152", "153", "154", "155", "156", "157", "158", "159", "160",
        "161", "162", "163", "164", "165", "166", "167", "168", "169", "170",
        "171", "172", "173", "174", "175", "176", "177", "178", "179", "180",
        "181", "182", "183", "184", "185", "186", "187", "188", "189", "190",
        "191", "192", "193", "194", "195", "196", "197", "198", "199", "200"
    ]

  var body: some View {
    ScrollView {
      ForEach($strings, id: \.self) { $val in
        TextField("", text: $val)
          .introspect(.textField, on: .iOS(.v17)) { textField in
              //do nothing
          }
          .padding()
          .background {
            RoundedRectangle(cornerRadius: 8)
              .fill(.gray)
          }
      }
    }
    .contentMargins(.all, 16)
  }
}
#Preview {
    KeyboardView()
}

My only chance would be to write/develop some introspection, tailored solely for TextFields and TextAreas that are only in focus, but I doubt that theres a way to do this elegantly.

I can try to open a ticket/issue on their side. But I doubt that theres something that can be done there too

[xzilja]

I also spent a bit of time on this today and started tinkering with alternative solution for my use case, i.e a simple UIViewRepresentable TextField that takes in and adds custom keyboard to itself. Even this seems to have some performance implications compared to default TextField usage in SwiftUI unless I comment out some code. It does perform quiet a bit better i.e. I need like 3x the views to start seeing lag compared to what we tested.

But at the end it seems like anything related to setting up custom keyboards will need to happen on some kind of "on focus" hook and then deallocate everything when it is lost. Maybe below sheds some inspiration

import SwiftUI

struct CustomKeyboardTextField<Content: View>: UIViewRepresentable {
  let keyboard: Content

  func makeUIView(context: Context) -> UITextField {
    let textField = UITextField()
//    let hostingController = UIHostingController(rootView: keyboard)
//    hostingController.view.frame = .init(origin: .zero, size: hostingController.view.intrinsicContentSize)
//    textField.inputView = hostingController.view
    return textField
  }

  func updateUIView(_ uiView: UITextField, context: Context) { }
}

[paescebu]

Hey!

Yeah it's definitely a solution. Reminds me a bit of KavSofts solution 😅. And happy to hear it helps the performance a bit. But I guess you just lose lots of the convenience you get with the default TextField/TextEditor provided by SwiftUI. I think in this case CustomKeyboardKit brings you lots of that convenience. But if you want to build something even more tailored for your use case with an extensive amount of TextFields you have to fallback to something custom, especially if it also has to have a good performance.

Let me know what you came up with! Really curious. Maybe there's always a learning behind it :)

Have a great weekend Pascal

[xzilja]

Will do, might ping you on X 😅 For now, I guess feel free to close this issue, if there is not much we can follow up on 👍 I'll likely need to continue on a custom solution for my use case as well

[paescebu]

Definitely! You can and should always ping me :)