knuddelsgmbh / jtokkit

JTokkit is a Java tokenizer library designed for use with OpenAI models.
https://jtokkit.knuddels.de/
MIT License
553 stars 42 forks source link

Produces GC and Hangs for a second or two. #60

Closed Shvet closed 9 months ago

Shvet commented 1 year ago

After adding this library in jetpack compose, Whenever i open composable it will produce GC,

I have Registry and encoding code in ViewModel.

 var inputTokens by mutableIntStateOf(0)
    var outPutTokens by mutableIntStateOf(0)

    private val registry = Encodings.newDefaultEncodingRegistry()
    private val encoding = registry.getEncoding(EncodingType.CL100K_BASE)

    fun checkTokens(isInput: Boolean, text: String) {
        if (isInput) {
            inputTokens = encoding.countTokens(text)
        } else {
            outPutTokens = encoding.countTokens(text)
        }
    }
tox-p commented 1 year ago

I'm not familiar enough with the android view model lifecycle, but as stated in the documentation, you need to make sure that the encoding registry is only instatiated once and then reused afterwards.

See my comment here: https://github.com/knuddelsgmbh/jtokkit/issues/50

tox-p commented 9 months ago

closing due to inactivity