search

singer-io / tap-fullstory

A Singer tap for extracting data from the FullStory API
GNU Affero General Public License v3.0
7 stars 9 forks source link

Encode zipped file string as UTF-8 #1

Closed JamesPaden closed 6 years ago

JamesPaden commented 6 years ago

I believe this fixes an issue I'm having in production in Stitch. You can see my support requests and error logs in Intercom under james@expectedbehavior.com.

I verified locally that this is the issue by setting the encoding to ascii. I then received the error noted in production. When I set it to utf, the error went away and the code worked as expected.

This appears to be related to some sort of system setting, that sets the default encoding, and I gues it's currently set to ascii.

dsprayberry commented 6 years ago

Full traceback from extraction logs:

2017-12-18 17:25:51,366Z tap - File "tap-env/bin/tap-fullstory", line 11, in 2017-12-18 17:25:51,366Z tap - sys.exit(main()) 2017-12-18 17:25:51,366Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_fullstory/init.py", line 163, in main 2017-12-18 17:25:51,366Z tap - do_sync() 2017-12-18 17:25:51,366Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_fullstory/init.py", line 155, in do_sync 2017-12-18 17:25:51,366Z tap - sync_events() 2017-12-18 17:25:51,366Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_fullstory/init.py", line 144, in sync_events 2017-12-18 17:25:51,366Z tap - for event in download_events(export_bundle['Id']): 2017-12-18 17:25:51,366Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_fullstory/init.py", line 134, in download_events 2017-12-18 17:25:51,366Z tap - body = request("get", params) 2017-12-18 17:25:51,366Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/backoff.py", line 286, in retry 2017-12-18 17:25:51,367Z tap - ret = target(*args, *kwargs) 2017-12-18 17:25:51,367Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/singer/utils.py", line 33, in wrapper 2017-12-18 17:25:51,367Z tap - return func(args, **kwargs) 2017-12-18 17:25:51,367Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_fullstory/init.py", line 106, in request 2017-12-18 17:25:51,367Z tap - json_body = unzip_to_json(resp.content) 2017-12-18 17:25:51,367Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_fullstory/init.py", line 63, in unzip_to_json 2017-12-18 17:25:51,367Z tap - decoded = io.TextIOWrapper(content_gz).read() 2017-12-18 17:25:51,367Z tap - File "/code/orchestrator/tap-env/lib/python3.5/encodings/ascii.py", line 26, in decode 2017-12-18 17:25:51,367Z tap - return codecs.ascii_decode(input, self.errors)[0] 2017-12-18 17:25:51,367Z tap - UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 290089: ordinal not in range(128)

KAllan357 commented 6 years ago

@JamesPaden This looks great thanks!

JamesPaden commented 6 years ago

@KAllan357 Thanks, and you're welcome! When is this likely to get deployed to Stitch production?

KAllan357 commented 6 years ago

@JamesPaden Once I get access to the project on PyPi, I should be able to push it out relatively quickly. I'd say within the next hour or two.

KAllan357 commented 6 years ago

Released 0.1.2 and deployed to Stitch prod.