Fix posts lookup Twitter map performance - Githubissues

superfaceai / station

Comlink metadata for self-integrating applications.

https://superface.ai/catalog

MIT License

22 stars 5 forks source link

Fix posts lookup Twitter map performance #207

Closed janhalama closed 2 years ago

janhalama commented 2 years ago

Description

This PR fixes social-media/posts-lookup performance by:

mapping posts in single Jessie script
using hashmap for authors and media lookup

Motivation and Context

We want to fix poor performance of social-media/posts-lookup Twitter map.

The performance improved significantly. When running in AWS Lambda environment single perform duration improved from 77s to 3.5s

Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] I have read the CONTRIBUTING document.
[x] I haven't repeated the code. (DRY)
[ ] I have added tests to cover my changes.
[x] All new and existing tests passed.

jnv commented 2 years ago

Did you talk about escape hatch, @zdne? :joy: