This fixes a problem in which this tap would fail when at least 1 submitted landing had no answers. This was resulting in a problem in which the process would exit and a bunch of data would not be extracted.
This problem happens for forms in which every field is not required, thus the user is able to submit the response without answering any questions. The following error then would show up when running this tap if at least a single user response fell in such scenario:
CRITICAL 'NoneType' object is not iterable
Traceback (most recent call last):
File "/Users/murilo.kakazu/.virtualenvs/tap-typeform/bin/tap-typeform", line 33, in <module>
sys.exit(load_entry_point('tap-typeform==2.0.0', 'console_scripts', 'tap-typeform')())
File "/Users/murilo.kakazu/.virtualenvs/tap-typeform/lib/python3.10/site-packages/singer/utils.py", line 229, in wrapped
return fnc(*args, **kwargs)
File "/Users/murilo.kakazu/.virtualenvs/tap-typeform/lib/python3.10/site-packages/tap_typeform-2.0.0-py3.10.egg/tap_typeform/__init__.py", line 49, in main
_sync(client, config, args.state, catalog.to_dict())
File "/Users/murilo.kakazu/.virtualenvs/tap-typeform/lib/python3.10/site-packages/tap_typeform-2.0.0-py3.10.egg/tap_typeform/sync.py", line 83, in sync
stream_obj.sync_obj(client, state, catalog['streams'], form, config["start_date"],
File "/Users/murilo.kakazu/.virtualenvs/tap-typeform/lib/python3.10/site-packages/tap_typeform-2.0.0-py3.10.egg/tap_typeform/streams.py", line 151, in sync_obj
max_bookmark = self.write_records(records, catalogs, selected_stream_ids,
File "/Users/murilo.kakazu/.virtualenvs/tap-typeform/lib/python3.10/site-packages/tap_typeform-2.0.0-py3.10.egg/tap_typeform/streams.py", line 127, in write_records
max_bookmark = self.sync_child_stream(record, catalogs, state, selected_stream_ids,form_id, start_date, max_bookmark)
File "/Users/murilo.kakazu/.virtualenvs/tap-typeform/lib/python3.10/site-packages/tap_typeform-2.0.0-py3.10.egg/tap_typeform/streams.py", line 95, in sync_child_stream
for rec in record[self.child_data_key]:
TypeError: 'NoneType' object is not iterable
This is bad, as even if the admin updates the form now making the questions required, the existing responses will still fail to be processed by this tap, as old responses without answers will still exist.
I fixed this by simply checking if the stream's child key is blank on the record, then it won't try to process the children data. e.g: when running the submitted_landing stream, if the record's answers attribute is blank, then it will register a submitted_landing, but no answers. As expected 😄
Description of change
This fixes a problem in which this tap would fail when at least 1 submitted landing had no answers. This was resulting in a problem in which the process would exit and a bunch of data would not be extracted.
This problem happens for forms in which every field is not required, thus the user is able to submit the response without answering any questions. The following error then would show up when running this tap if at least a single user response fell in such scenario:
This is bad, as even if the admin updates the form now making the questions required, the existing responses will still fail to be processed by this tap, as old responses without answers will still exist.
I fixed this by simply checking if the stream's child key is blank on the record, then it won't try to process the children data. e.g: when running the
submitted_landingstream, if the record'sanswersattribute is blank, then it will register asubmitted_landing, but noanswers. As expected 😄Also, this fixes both reported issues: https://github.com/singer-io/tap-typeform/issues/22 and https://github.com/singer-io/tap-typeform/issues/24
Manual QA steps
1 - Create a form, with 1 or more questions, in which none of them have the "required" validation. e.g:
2 - Open the form and submit it without answering the questions 3 - Run this tap for that form_id 4 - 💥!
CRITICAL 'NoneType' object is not iterableRisks
Rollback steps