singer-io / tap-fullstory

A Singer tap for extracting data from the FullStory API
GNU Affero General Public License v3.0
7 stars 9 forks source link

Bump singer-python to v1.9.1 #7

Closed migstopheles closed 4 years ago

migstopheles commented 4 years ago

Description of change

~3 years ago, singer-python was patched to fix issues with the standard JSON encoder failing to encode Decimal types.

Bumping the version of singer-python to its max minor version (1.9.1) should fix this issue.

Example error

2020-03-13 01:00:54,987Z    tap - Traceback (most recent call last):
2020-03-13 01:00:54,987Z    tap -   File "tap-env/bin/tap-fullstory", line 9, in <module>
2020-03-13 01:00:54,987Z    tap -     load_entry_point('tap-fullstory==1.0.3', 'console_scripts', 'tap-fullstory')()
2020-03-13 01:00:54,988Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_fullstory/__init__.py", line 162, in main
2020-03-13 01:00:54,988Z    tap -     do_sync()
2020-03-13 01:00:54,988Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_fullstory/__init__.py", line 154, in do_sync
2020-03-13 01:00:54,988Z    tap -     sync_events()
2020-03-13 01:00:54,988Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_fullstory/__init__.py", line 146, in sync_events
2020-03-13 01:00:54,988Z    tap -     singer.write_record("events", event)
2020-03-13 01:00:54,988Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/singer/__init__.py", line 61, in write_record
2020-03-13 01:00:54,988Z    tap -     write_message(RecordMessage(stream=stream_name, record=record))
2020-03-13 01:00:54,988Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/singer/__init__.py", line 52, in write_message
2020-03-13 01:00:54,988Z    tap -     sys.stdout.write(message.tojson() + '\n')
2020-03-13 01:00:54,989Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/singer/__init__.py", line 33, in tojson
2020-03-13 01:00:54,989Z    tap -     return json.dumps(self.asdict())
2020-03-13 01:00:54,989Z    tap -   File "/root/.pyenv/versions/3.5.2/lib/python3.5/json/__init__.py", line 230, in dumps
2020-03-13 01:00:54,989Z    tap -     return _default_encoder.encode(obj)
2020-03-13 01:00:54,989Z    tap -   File "/root/.pyenv/versions/3.5.2/lib/python3.5/json/encoder.py", line 198, in encode
2020-03-13 01:00:54,989Z    tap -     chunks = self.iterencode(o, _one_shot=True)
2020-03-13 01:00:54,989Z    tap -   File "/root/.pyenv/versions/3.5.2/lib/python3.5/json/encoder.py", line 256, in iterencode
2020-03-13 01:00:54,989Z    tap -     return _iterencode(o, 0)
2020-03-13 01:00:54,989Z    tap -   File "/root/.pyenv/versions/3.5.2/lib/python3.5/json/encoder.py", line 179, in default
2020-03-13 01:00:54,989Z    tap -     raise TypeError(repr(o) + " is not JSON serializable")
2020-03-13 01:00:54,989Z    tap - TypeError: Decimal('24.1444') is not JSON serializable

Manual QA steps

Risks

Rollback steps

cmerrick commented 4 years ago

Hi @migstopheles, thanks for your contribution!

In order for us to evaluate and accept your PR, we ask that you sign a contribution license agreement. It's all electronic and will take just minutes.

cmerrick commented 4 years ago

You did it @migstopheles!

Thank you for signing the Singer Contribution License Agreement.

migstopheles commented 4 years ago

@cmerrick is this OK to be merged or are there more steps required?

lbillingham commented 4 years ago

Hello folks 👋 , I'm getting similar problems with Decimal conversions. Anything I can do to help with this PR?

markshilton commented 4 years ago

We could really, really do with getting this merged so we can use this tap properly. It's currently unusable for us without this fix. Is anyone able to help?

migstopheles commented 4 years ago

@dmosorast @KAllan357 @cmerrick sorry for the :postal_horn: but does anyone know how i can get this PR approved and merged? thanks

markshilton commented 4 years ago

Thanks for merging this @KAllan357. You've made a bunch of engineers and data scientists at TravelNest very happy! 😄