singer-io / tap-typeform

Singer.io tap for extracting TypeForm data
GNU Affero General Public License v3.0
11 stars 20 forks source link

Answers not syncing #9

Open pcheek opened 5 years ago

pcheek commented 5 years ago

I had the Typeform "answers" table syncing successfully for a few weeks and then something happened and the answers table is no longer syncing even though the questions are continuing to successfully sync. I have tried re-running the data extraction and using a new Typeform API key with full permissions and that did not resolve the problem. Maybe there's something obvious I'm missing?

Here are the logs from the most recent extraction where you will see that zero answers were extracted:


2019-06-04 19:02:16,072Z   main - INFO Starting tap to discover schemas: tap-env/bin/tap-typeform --config /tmp/tap_discover_config.json --discover
2019-06-04 19:02:16,262Z   main - INFO Tap exited normally.
2019-06-04 19:02:16,262Z   main - INFO Saving list of discovered streams
2019-06-04 19:02:16,300Z   main - INFO Saving structure of stream landings
2019-06-04 19:02:16,743Z   main - INFO Saving structure of stream answers
2019-06-04 19:02:16,842Z   main - INFO Saving structure of stream questions
2019-06-04 19:02:16,947Z   main - INFO Getting list of selected properties
2019-06-04 19:02:16,968Z   main - INFO Starting tap: tap-env/bin/tap-typeform --config /tmp/tap_config.json --properties /tmp/properties.json --catalog /tmp/catalog.json --state /tmp/tap_state.json
2019-06-04 19:02:16,970Z   main - INFO Starting target: target-env/bin/target-stitch --config /tmp/target_config.json
2019-06-04 19:02:17,153Z    tap - INFO form: C6B6FW 
2019-06-04 19:02:17,203Z target - INFO Using Stitch import URL https://api.stitchdata.com/v2/import/batch
2019-06-04 19:02:17,317Z    tap - INFO "form definition C6B6FW" job took 0.2s, ended with status "succeeded"
2019-06-04 19:02:17,318Z    tap - INFO replicated 4 records from "questions" endpoint
2019-06-04 19:02:17,318Z    tap - INFO start_date: 2015-01-01T05:00:00+00:00 
2019-06-04 19:02:17,318Z    tap - INFO end_date: 2019-06-04T19:00:00+00:00 
2019-06-04 19:02:17,318Z    tap - INFO last_date: 2016-05-04T09:00:00+00:00 
2019-06-04 19:02:17,318Z    tap - INFO ut_current_date: 1462352400 
2019-06-04 19:02:17,318Z    tap - INFO ut_next_date: 1462356000 
2019-06-04 19:02:17,319Z    tap - INFO Forms query - form: C6B6FW start_date: 2016-05-04 09:00 end_date: 2016-05-04 10:00 
2019-06-04 19:02:17,472Z    tap - INFO raw data items= 0
2019-06-04 19:02:17,473Z    tap - INFO "form C6B6FW" job took 0.2s, ended with status "succeeded"
2019-06-04 19:02:17,474Z    tap - INFO replicated 0 records from "answers" endpoint
2019-06-04 19:02:17,474Z    tap - INFO ut_current_date: 1462356000 
2019-06-04 19:02:17,474Z    tap - INFO ut_next_date: 1462359600 
2019-06-04 19:02:23,154Z    tap - INFO Forms query - form: C6B6FW start_date: 2016-05-04 10:00 end_date: 2016-05-04 11:00 
2019-06-04 19:02:23,266Z    tap - INFO raw data items= 0
2019-06-04 19:02:23,266Z    tap - INFO "form C6B6FW" job took 5.8s, ended with status "succeeded"
2019-06-04 19:02:23,267Z    tap - INFO replicated 0 records from "answers" endpoint
2019-06-04 19:02:23,267Z    tap - INFO ut_current_date: 1462359600 
2019-06-04 19:02:23,267Z    tap - INFO ut_next_date: 1462363200 
2019-06-04 19:02:29,154Z    tap - INFO Forms query - form: C6B6FW start_date: 2016-05-04 11:00 end_date: 2016-05-04 12:00 
2019-06-04 19:02:29,315Z    tap - INFO raw data items= 0```
countershane commented 4 years ago

I'm seeing this also. It's strange actually that the questions keep syncing, because the questions actually haven't changed. I would expect to see the Landings and Answers tables updating. My initial sync went fine and all the historical data is loaded in correctly. There's no bug reports to note.

pjatx commented 4 years ago

Also noticing this - just tried to leverage this tap via Stitch Data - questions are syncing fine, but answers are failing.

countershane commented 4 years ago

So I switched to daily syncing from the default of hourly, and it seems like answers data is coming through now... Could you try that and see if it works for you too?

pjatx commented 4 years ago

@countershane thanks man! that did the trick for me. 🎉

vaishnavi3005 commented 4 years ago

Having the same issue as @countershane . Switching to daily syncing also did not work.