search

singer-io / tap-appsflyer

A Singer.io tap for extracting data from the AppsFlyer API
GNU Affero General Public License v3.0
11 stars 40 forks source link

NoneType Exception when replicating #8

Closed helloimowen closed 6 years ago

helloimowen commented 6 years ago

Encountering an AttributeError when the StitchData service attempts to transform a certain row from our AppsFlyer in-app events.

Here are the logs:

2018-08-14 05:03:59,908Z main - INFO Running tap-appsflyer version 0.0.7 and target-stitch version 1.7.4 2018-08-14 05:04:00,061Z main - DEBUG Getting initial state 2018-08-14 05:04:00,121Z main - INFO Starting tap: tap-env/bin/tap-appsflyer --config /tmp/tap_config.json --state /tmp/tap_state.json 2018-08-14 05:04:00,124Z main - INFO Starting target: target-env/bin/target-stitch --config /tmp/target_config.json 2018-08-14 05:04:00,404Z tap - INFO do_sync() 2018-08-14 05:04:00,404Z tap - INFO Starting sync. Will sync these streams: ['in_app_events'] 2018-08-14 05:04:00,404Z tap - INFO Syncing in_app_events 2018-08-14 05:04:00,408Z tap - INFO Syncing data from 2018-06-04 17:17:22 to 2018-06-14 17:17:22 2018-08-14 05:04:00,413Z tap - INFO GET https://hq.appsflyer.com/export/id1293721589/in_app_events_report/v5?from=2018-06-04+17%3A17&api_token=....................................&to=2018-06-14+17%3A17 2018-08-14 05:04:00,504Z target - INFO Using Stitch import URL https://api.stitchdata.com/v2/import/batch 2018-08-14 05:04:18,813Z tap - INFO STATS: {"http_status_code": 200, "source": "in_app_events", "duration": 18.399158000946045, "status": "succeeded"} 2018-08-14 05:04:18,814Z tap - Traceback (most recent call last): 2018-08-14 05:04:18,814Z tap - File "tap-env/bin/tap-appsflyer", line 11, in <module> 2018-08-14 05:04:18,814Z tap - sys.exit(main()) 2018-08-14 05:04:18,814Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_appsflyer/__init__.py", line 458, in main 2018-08-14 05:04:18,814Z tap - do_sync() 2018-08-14 05:04:18,814Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_appsflyer/__init__.py", line 440, in do_sync 2018-08-14 05:04:18,814Z tap - stream.sync() # pylint: disable=not-callable 2018-08-14 05:04:18,814Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_appsflyer/__init__.py", line 402, in sync_in_app_events 2018-08-14 05:04:18,814Z tap - record = xform(row, schema) 2018-08-14 05:04:18,815Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_appsflyer/__init__.py", line 89, in xform 2018-08-14 05:04:18,815Z tap - xform_boolean_field(record, "wifi") 2018-08-14 05:04:18,815Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_appsflyer/__init__.py", line 75, in xform_boolean_field 2018-08-14 05:04:18,815Z tap - if record[field_name].lower() == "TRUE".lower(): 2018-08-14 05:04:18,815Z tap - AttributeError: 'NoneType' object has no attribute 'lower' 2018-08-14 05:04:18,839Z target - INFO Exiting normally 2018-08-14 05:04:18,875Z main - INFO Target exited normally with status 0 2018-08-14 05:04:18,876Z main - INFO Exit status is: Tap failed with code 1. Target succeeded. Our data probably has an event with a blank boolean field. Is this something that can be accounted for and handled in the tap, or should we try to remove this event from appsflyer entirely?

The code in question: https://github.com/singer-io/tap-appsflyer/blob/3acfdbdf722ad5775262fd8b99d0aa8ac29baf94/tap_appsflyer/__init__.py#L75

dmosorast commented 6 years ago

I didn't realize that merging the PR would auto-close this issue, but this should be fixed in version 0.0.8.