search

dremio-professional-services / dremio-cloner

27 stars 21 forks source link

Dremio Cloner with Cloud: ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None) #45

Closed cmillerja closed 9 months ago

cmillerja commented 9 months ago

I am trying to test Dremio Cloner with Cloud. I have followed the readme at https://github.com/deane-dremio/dremio-cloner/blob/master/README.md.

However, when attempting to perform a PUT to Cloud I encounter the following error:

File "C:\Python\Lib\site-packages\requests\adapters.py", line 501, in send raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))

Here is an excerpt from the config_write_dir.json file:

{"dremio_cloner": [ {"command":"put"}, {"target": [ {"endpoint": "http://api.eu.dremio.cloud/"}, {"username": ""}, {"password": ""}, {"verify_ssl": "True"}, {"is_community_edition": "False"}, {"is_dremio_cloud": "True"}, {"dremio_cloud_org_id": "#########################"}, {"dremio_cloud_project_id": "###############"} ] },

mxmarg commented 9 months ago

Hi @cmillerja, I hope you are doing well :)

I just tested a Cloner put operation against Dremio Cloud EU, which created multiple folders and views, without issue.

The error An existing connection was forcibly closed by the remote host is unlikely to be caused by Cloner directly and more likely a general network or configuration issue, e.g. on the level of your Python requests library or possibly the network you were using to connect to DC.

tolbrooker commented 9 months ago

Thanks, @mxmarg! Can you confirm we are using the correct url for a EU based cloud? http://api.eu.dremio.cloud/

tolbrooker commented 9 months ago

Also, for your reference, I dont get this error when using a dry run.

tolbrooker commented 9 months ago

Also, for your reference, i don't get the error when configured for on prem connections.

cmillerja commented 9 months ago

Also, for your reference, a get/read works fine.

acrylJonny commented 9 months ago

@cmillerja - the issue is your endpoint (http://api.eu.dremio.cloud/). This should be https, i.e. https://api.eu.dremio.cloud/.

tolbrooker commented 9 months ago

Thanks @dremioJonny.