Parser is very slow

[Valve]

We have it in production and it consistently takes ~30-40ms on a t3.micro AWS instance. That's the slowest component in a request. Is there a way to speed it up?

[johnbalvin]

have you tried other package? like this: https://github.com/mssola/user_agent I'm looking for the best one but I have not decided yet and that one is the one I will try

[Valve]

I haven't tried, just used this one (probably because it was a part of a larger project). Will need to try the one you mentioned @johnbalvin thanks

[agend07]

make a cache - for example map of 50k most used user agents strings to the results of parsing them. U might use this for: https://github.com/hashicorp/golang-lru

[Valve]

@agend07 making a cache to cache the results of parsing a 50 character string? In Golang? You must be joking. It should complete in milliseconds making the cache unnecessary. The quality of implementation is so bad that the first instinct should be to fix that implementation, not design a hack to cache this slowness.

[agend07]

Not sure why I must be joking. This module is about checking user agent string against long list of regexes, which is slow. And golang regex implementation isn't the fastest one. So its better to cache the results. 'Fix the implementation' would mean writing better regex package which is a huge task. And I didn't mean 50 character string but using cache with 50 000 elements. There are implementations in other languages: https://github.com/ua-parser. You might check if they are that much faster.

[Valve]

@agend07 to give you an idea of a performance level expected from a UA parser, please read this: https://yauaa.basjes.nl/README-Performance.html

This product uses a Java parser generator (ANTLR) to build a parser. No regexes necessary. Also please see this: http://www.otsukare.info/2013/10/02/ua-parsing

Overall I can tell that this project is ~50-200 times slower than it should (and can!) be.

[Valve]

ok, for the record, the parser instantiation with:

    parser := uaparser.NewFromSaved()

Is what's slow. Avoid creating a new parser each time (e.g. for each request) and cache it somewhere and reuse if possible.

Actual parsing is 2ms

[Valve]

@johnbalvin looks like the other package that you mentioned is not maintained properly (not that I blame them, it's hard to spend time on a project you no longer use), just fyi.