search

singer-io / tap-fullstory

A Singer tap for extracting data from the FullStory API
GNU Affero General Public License v3.0
7 stars 9 forks source link

TypeError: Decimal is not JSON serializable #8

Open zacarybrown opened 4 years ago

zacarybrown commented 4 years ago

I'm trying to use this integration from within Stitch data. It fails because of a TypeError. The error log is listed below. Did FullStory change their response format at some point?

Thanks in advance,

Zac

2020-06-02 17:35:21,493Z   main - INFO Running tap-fullstory version 1.0.3 and target-stitch version 3.0.2
2020-06-02 17:35:21,670Z   main - INFO [smart-services] building smart service producer: topic( com.stitchdata.extractionJobStarted ), use_ssl( true )
2020-06-02 17:35:21,673Z   main - INFO [smart-services] building smart service producer: topic( com.stitchdata.extractionJobFinished ), use_ssl( true )
2020-06-02 17:35:22,674Z   main - INFO [smart-services] event successfully sent to kafka: com.stitchdata.extractionJobStarted [44] at offset None
2020-06-02 17:35:22,747Z   main - INFO Current state: {}
2020-06-02 17:35:22,748Z   main - INFO Starting tap: tap-env/bin/tap-fullstory --config /tmp/tap_config.json --state /tmp/tap_state.json
2020-06-02 17:35:22,751Z   main - INFO Starting target: target-env/bin/target-stitch --config /tmp/target_config.json
2020-06-02 17:35:23,183Z    tap - INFO Starting sync
2020-06-02 17:35:23,184Z    tap - INFO GET https://export.fullstory.com/api/v1/export/list?start=1558915200
2020-06-02 17:35:23,329Z    tap - INFO HTTP request to "https://export.fullstory.com/api/v1/export/list" endpoint took 0.144s, returned status code 200
2020-06-02 17:35:23,330Z    tap - INFO GET https://export.fullstory.com/api/v1/export/get?id=158310720048
2020-06-02 17:35:23,617Z target - INFO Using batch_size_prefernces of {'full_table_streams': [], 'batch_size_preference': None, 'user_batch_size_preference': None}
2020-06-02 17:35:23,618Z target - INFO Using turbo_boost_factor of 5
2020-06-02 17:36:16,170Z    tap - INFO HTTP request to "https://export.fullstory.com/api/v1/export/get" endpoint took 52.841s, returned status code 200
2020-06-02 17:36:16,190Z    tap - INFO replicated 28 records from "events" endpoint
2020-06-02 17:36:16,191Z    tap - Traceback (most recent call last):
2020-06-02 17:36:16,191Z    tap -   File "tap-env/bin/tap-fullstory", line 11, in <module>
2020-06-02 17:36:16,191Z    tap -     load_entry_point('tap-fullstory==1.0.3', 'console_scripts', 'tap-fullstory')()
2020-06-02 17:36:16,192Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_fullstory/__init__.py", line 162, in main
2020-06-02 17:36:16,192Z    tap -     do_sync()
2020-06-02 17:36:16,192Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_fullstory/__init__.py", line 154, in do_sync
2020-06-02 17:36:16,192Z    tap -     sync_events()
2020-06-02 17:36:16,192Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_fullstory/__init__.py", line 146, in sync_events
2020-06-02 17:36:16,192Z    tap -     singer.write_record("events", event)
2020-06-02 17:36:16,192Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/singer/__init__.py", line 61, in write_record
2020-06-02 17:36:16,192Z    tap -     write_message(RecordMessage(stream=stream_name, record=record))
2020-06-02 17:36:16,192Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/singer/__init__.py", line 52, in write_message
2020-06-02 17:36:16,192Z    tap -     sys.stdout.write(message.tojson() + '\n')
2020-06-02 17:36:16,192Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/singer/__init__.py", line 33, in tojson
2020-06-02 17:36:16,192Z    tap -     return json.dumps(self.asdict())
2020-06-02 17:36:16,193Z    tap -   File "/root/.pyenv/versions/3.5.2/lib/python3.5/json/__init__.py", line 230, in dumps
2020-06-02 17:36:16,193Z    tap -     return _default_encoder.encode(obj)
2020-06-02 17:36:16,193Z    tap -   File "/root/.pyenv/versions/3.5.2/lib/python3.5/json/encoder.py", line 198, in encode
2020-06-02 17:36:16,193Z    tap -     chunks = self.iterencode(o, _one_shot=True)
2020-06-02 17:36:16,193Z    tap -   File "/root/.pyenv/versions/3.5.2/lib/python3.5/json/encoder.py", line 256, in iterencode
2020-06-02 17:36:16,193Z    tap -     return _iterencode(o, 0)
2020-06-02 17:36:16,193Z    tap -   File "/root/.pyenv/versions/3.5.2/lib/python3.5/json/encoder.py", line 179, in default
2020-06-02 17:36:16,193Z    tap -     raise TypeError(repr(o) + " is not JSON serializable")
2020-06-02 17:36:16,193Z    tap - TypeError: Decimal('32.95022') is not JSON serializable
2020-06-02 17:36:16,221Z target - INFO Serializing batch with 27 messages for table events
2020-06-02 17:36:16,222Z target - INFO Sending batch of 55534 bytes to https://api.stitchdata.com/v2/import/batch
2020-06-02 17:36:16,421Z target - INFO Requests complete, stopping loop
2020-06-02 17:36:16,472Z   main - INFO Target exited normally with status 0
2020-06-02 17:36:16,684Z   main - INFO [smart-services] event successfully sent to kafka: com.stitchdata.extractionJobFinished [2] at offset None
2020-06-02 17:36:16,687Z   main - INFO No tunnel subprocess to tear down
2020-06-02 17:36:16,687Z   main - INFO Exit status is: Tap failed with code 1. Target succeeded.
migstopheles commented 4 years ago

my PR is open to fix this but nobody is looking :crying_cat_face: