Poor regex performance

[JamieMagee]

Describe the bug When running tern against postgres:latest approximately 85% of the runtime is spent in update_master_list and approximately 60% of that function (and 50% of total runtime) is spent compiling and running regexes in prop_names.

I've attached profiler logs for more information

To Reproduce Steps to reproduce the behavior:

1. tern --driver fuse --clear-cache report --docker-image postgres:latest
2. Wait...

Profiler logs generated using

export PYTHONPATH=/workspaces/tern/
pyinstrument -r html -o index.html tern/__main__.py --driver fuse --clear-cache report --docker-image postgres:latest

Expected behavior Better performance

Environment you are running Tern on Enter all that apply

• Output of 'tern --version'
• Operating System (Linux Distro and version or Mac or Windows)
• Vagrant file
• Container OS
• Python version (3.6 or higher)
• Cloud environment (AWS, Azure, GCP)

Please attach files if they exist

• Profiler logs
• 

[nishakm]

@JamieMagee are you working on a fix?

[JamieMagee]

@nishakm Unfortunately I haven't got time right now to properly investigate and test a fix, but I was looking at this as a starting point:

prop_decorators = r'^__|^_'
for key in obj.__dict__.keys():
    # remove private and protected decorator characters if any
    priv_name = '_' + obj.__class__.__name__
    prop_name = re.sub(priv_name, '', key)
    prop_name = re.sub(prop_decorators, '', prop_name, 1)
    yield key, prop_name

to

for key in obj.__dict__.keys():
    prop_name = key.split('__')[-1]
    yield key, prop_name

[nishakm]

@JamieMagee no worries. I can take a look.

[nishakm]

Looks to me like to_dict() is called way too many times with update_master_list. Let me try looking at that first.