RecursionError: maximum recursion depth exceeded for a put operation

[erosons]

Hello Folks,

I have used dremio cloner in the past successful for source migration, I try to implement again to migrating an s3 source, but this time ran into a couple of error. I am using dremio version 22.1.7 integrated with Active Directory Log INFO:2023-05-16 14:48:25,032:Executing command 'put'. WARNING:2023-05-16 14:48:25,651:_process_acl: Source User 30489f5d-678a-4129-a7ad-6becbbc425ca not found in the target Dremio Environment. User is removed from ACL definition as per ignore_missing_acl_user configuration. space:Samson Error from console Traceback (most recent call last): File "/Users/s.eromonsei/dremio-cloner/src/dremio_cloner.py", line 159, in <module> main() File "/Users/s.eromonsei/dremio-cloner/src/dremio_cloner.py", line 49, in main put_dremio_environment(config) File "/Users/s.eromonsei/dremio-cloner/src/dremio_cloner.py", line 96, in put_dremio_environment writer.write_dremio_environment() File "/Users/s.eromonsei/dremio-cloner/src/DremioWriter.py", line 90, in write_dremio_environment self._write_space(space, self._config.space_process_mode, self._config.space_ignore_missing_acl_user, self._config.space_ignore_missing_acl_group) File "/Users/s.eromonsei/dremio-cloner/src/DremioWriter.py", line 126, in _write_space return self._write_entity(entity, process_mode, ignore_missing_acl_user_flag, ignore_missing_acl_group_flag) File "/Users/s.eromonsei/dremio-cloner/src/DremioWriter.py", line 312, in _write_entity updated_entity = self._dremio_env.update_catalog_entity(entity['id'], entity, self._config.dry_run, report_error) File "/Users/s.eromonsei/dremio-cloner/src/Dremio.py", line 250, in update_catalog_entity return self._api_put_json(self._catalog_url + entity_id, entity, source="update_catalog_entity", report_error = report_error) File "/Users/s.eromonsei/dremio-cloner/src/Dremio.py", line 443, in _api_put_json return self._api_put_json(url, json_data, source, report_error, False) File "/Users/s.eromonsei/dremio-cloner/src/Dremio.py", line 443, in _api_put_json return self._api_put_json(url, json_data, source, report_error, False) File "/Users/s.eromonsei/dremio-cloner/src/Dremio.py", line 443, in _api_put_json return self._api_put_json(url, json_data, source, report_error, False) [Previous line repeated 967 more times] File "/Users/s.eromonsei/dremio-cloner/src/Dremio.py", line 430, in _api_put_json response = requests.request("PUT", self._endpoint + url, json=json_data, headers=self._headers, timeout=self._api_timeout, verify=self._verify_ssl) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/requests/api.py", line 59, in request return session.request(method=method, url=url, **kwargs) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/requests/sessions.py", line 587, in request resp = self.send(prep, **send_kwargs) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/requests/sessions.py", line 701, in send r = adapter.send(request, **kwargs) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/requests/adapters.py", line 486, in send resp = conn.urlopen( File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/urllib3/connectionpool.py", line 790, in urlopen response = self._make_request( File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/urllib3/connectionpool.py", line 536, in _make_request response = conn.getresponse() File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/urllib3/connection.py", line 454, in getresponse httplib_response = super().getresponse() File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/http/client.py", line 1322, in getresponse response.begin() File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/http/client.py", line 327, in begin self.headers = self.msg = parse_headers(self.fp) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/http/client.py", line 221, in parse_headers return email.parser.Parser(_class=_class).parsestr(hstring) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/email/parser.py", line 67, in parsestr return self.parse(StringIO(text), headersonly=headersonly) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/email/parser.py", line 56, in parse feedparser.feed(data) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/email/feedparser.py", line 176, in feed self._call_parse() File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/email/feedparser.py", line 180, in _call_parse self._parse() File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/email/feedparser.py", line 295, in _parsegen if self._cur.get_content_maintype() == 'message': File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/email/message.py", line 594, in get_content_maintype ctype = self.get_content_type() File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/email/message.py", line 578, in get_content_type value = self.get('content-type', missing) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/email/message.py", line 471, in get return self.policy.header_fetch_parse(k, v) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/email/_policybase.py", line 316, in header_fetch_parse return self._sanitize_header(name, value) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/email/_policybase.py", line 287, in _sanitize_header if _has_surrogates(value): File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/email/utils.py", line 57, in _has_surrogates s.encode() RecursionError: maximum recursion depth exceeded while calling a Python object

[mxmarg]

After a quick initial look at the stack trace, it seems like the infinite recursion occurs here: File "/Users/s.eromonsei/dremio-cloner/src/Dremio.py", line 443, in _api_put_json return self._api_put_json(url, json_data, source, report_error, False) [Previous line repeated 967 more times]

Can you please try to modify line 476 to say if reauthenticate: instead of if not reauthenticate: and rerun? https://github.com/deane-dremio/dremio-cloner/blob/master/src/Dremio.py#L476

[erosons]

Thank you that works, I was able to perform put operations