singer-io / tap-typeform

Singer.io tap for extracting TypeForm data
GNU Affero General Public License v3.0
11 stars 20 forks source link

Add tap tester #30

Closed 0xpetersatoshi closed 3 years ago

0xpetersatoshi commented 3 years ago

Description of change

https://jira.talendforge.org/browse/TDL-12531

Added full suite of tests. This required some major refactoring of the bookmarking, how metadata and the catalog file was produced, adding a way to incorporate INCREMENTAL streams, as well as the corresponding helper functions for integrating this all into the tap.

Note: I get almost all the tests to pass but encountering this error and not sure what is missing:

======================================================================
ERROR: test_run (test_typeform_bookmarks.TypeformBookmarks)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/code/tap-typeform/tests/test_typeform_bookmarks.py", line 76, in test_run
    self.bookmarks_test(expected_streams)
  File "/opt/code/tap-typeform/tests/test_typeform_bookmarks.py", line 124, in bookmarks_test
    second_sync_record_count = self.run_and_verify_sync(conn_id)
  File "/opt/code/tap-typeform/tests/base.py", line 182, in run_and_verify_sync
    sync_job_name = runner.run_sync_mode(self, conn_id)
  File "/opt/code/tap-tester/tap_tester/runner.py", line 26, in run_sync_mode
    return BACKEND.run_sync_mode(scenario, conn_id)
  File "/opt/code/tap-tester/tap_tester/backends/in_memory.py", line 161, in run_sync_mode
    raise Exception("sync invocation failed")
Exception: sync invocation failed

----------------------------------------------------------------------

Manual QA steps

Risks

Rollback steps

0xpetersatoshi commented 3 years ago

@kspeer825 @KAllan357 I submitted the changes suggested.