Closed crankycoder closed 6 years ago
I just fetched the latest S3 files:
In [15]: coinstall = json.load(open('guid_coinstallation.json'))
In [16]: ranking = json.load(open('guid_install_ranking.json'))
In [17]: coinstall_guids = set(coinstall.keys())
In [18]: ranking_guids = set(ranking.keys())
In [19]: coinstall_guids - ranking_guids
Out[19]: set()
If I obtain the previous co-installation JSON model:
In [21]: old_coinstal_guids = set(old_coinstall.keys())
In [22]: old_coinstal_guids - ranking_guids
Out[22]:
{'breeze-minimize-window-button@andreyorst.com',
'instarss@turmel.info',
'jid1-q3sH8pJhc4KGUp@jetpack',
'jid1-zYaXnuzvDbGcEg@jetpack',
'killit@jetpack',
'octopatcher_wowmotty@gmail.com',
'rym2spotify@magdolen.sk',
'selection-popup@foreverglory.me',
'tb-clone-tab-single@codefisher.org',
'{04331ca8-933b-4f14-bc84-806e7227be94}',
'{24e9b60f-0b13-43ce-aa07-ceea8ab747a8}',
'{d400d670-e438-49e6-9788-c9ae0d37158b}',
'{d6fc5642-b9b3-4ba2-b5cd-70116a22f0fa}'}
In [23]:
This error happens when the ETL jobs are updated - so there is something out of sync with self._guid_maps
.
self._guid_maps
is only updated on https://github.com/mozilla/taar-lite/blob/238385f76cb7dade1c2801f363cd357e4bf45f6e/taar_lite/recommenders/guid_based_recommender.py#L74. I'm putting in a patch so that whenever the JSON loader is refreshed, it triggers a recomputation of the normalization tables.
The error in production should only happen on ETL job completion when the longitudinal DAG is run.
cc/ @jasonthomas
newrelic is showing a crash w.r.t GUID lookups:
URL path is : /taarlite/api/v1/addon_recommendations/%7Bad0d925d-88f8-47f1-85ea-8463569e756e%7D/