search

babelfish-for-postgresql / babelfish_extensions

Babelfish for PostgreSQL provides the capability for PostgreSQL to work with applications written for Microsoft SQL Server. Babelfish understands the SQL Server wire-protocol and T-SQL, the Microsoft SQL Server query and procedural language, so you don’t have to switch database drivers or rewrite all of your application queries.
https://babelfishpg.org/
Apache License 2.0
276 stars 93 forks source link

[Bug]: bcp giving "An existing connection was forcibly closed by the remote host." error #1647

Open calmitchell617 opened 1 year ago

calmitchell617 commented 1 year ago

What happened?

I am loading a large amount of tables in a for loop with bcp. It is working well most of the time but sometimes errors out (on the same tables).

My powershell commands, on a Windows server, look like this:

-- data export command

bcp "<SELECT * FROM MYDB.MYSCHEMA.MYTABLE>" <FILEPATH> -c -t"<CUSTOM DELIM>" -r"<CUSTOM NEWLINE>" -S <RDS ENDPOINT> -U <USERNAME> -P <PASSWORD>

--data import command

bcp <MYDB.MYSCHEMA.MYTABLE> in <FILEPATH> -c -t"<CUSTOM DELIM>" -r"<CUSTOM NEWLINE>" -S <BABELFISH ENDPOINT> -U <USERNAME> -P <PASSWORD>

I can't see anything in those tables that seem special, but it consistently errors on the same ones. Does anyone have any knowledge of what might be causing this?

Version

BABEL_3_X_DEV (Default)

Extension

None

Which flavor of Linux are you using when you see the bug?

No response

Relevant log output

Starting copy...
1000 rows sent to SQL Server. Total sent: 1000
SQLState = 08S01, NativeError = 10054
Error = [Microsoft][ODBC Driver 17 for SQL Server]TCP Provider: An existing connection was forcibly closed by the remote host.

SQLState = 08S01, NativeError = 10054
Error = [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure
SQLState = 08S01, NativeError = 0
Error = [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure
SQLState = 08S01, NativeError = 0
Error = [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure
SQLState = 08S01, NativeError = 0
Error = [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure
SQLState = 08S01, NativeError = 0
Error = [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure
SQLState = 08S01, NativeError = 0
Error = [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure
SQLState = 08S01, NativeError = 0
Error = [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure
SQLState = 08S01, NativeError = 0
Error = [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure
SQLState = 08S01, NativeError = 0
Error = [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure
SQLState = 08S01, NativeError = 0
Error = [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure
SQLState = S1000, NativeError = 0
Error = [Microsoft][ODBC Driver 17 for SQL Server]Text column data incomplete
SQLState = 08S01, NativeError = 0
Error = [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure
SQLState = 08S01, NativeError = -2147467259
Error = [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure

BCP copy in failed


### Code of Conduct

- [X] I agree to follow this project's Code of Conduct.
calmitchell617 commented 1 year ago

Side note, is it possible to load the data via psql's \copy command, then move the data to the Babelfish DB?

KushaalShroff commented 1 year ago

@calmitchell617 can you please provide more details like the table DDL and also check if there was a crash and provide it's stack trace along with any logs you can capture.

calmitchell617 commented 1 year ago

Yes, I will provide as much as possible.

This work is for a 3rd party client - I just asked them what is OK to share. They will probably want certain things shared privately, can you please let me know how that can be accomplished? Email?

Once the problem is diagnosed, I would be happy to post a generalized example for the benefit of the viewers of this issue, but like I said, I'm not sure what the issue is, so cannot generalize it at this time.

calmitchell617 commented 1 year ago

@KushaalShroff they said it is OK to share DDL privately, as well as the stack trace.

How can I get that to you? If you want to send me an email, the contact form at https://www.sqlpipe.com/contact goes to my inbox.

suprio-amzn commented 1 year ago

@calmitchell617: Are you using Babelfish on Aurora PostgreSQL? If so, you can open a support ticket with AWS and share the materials there.

calmitchell617 commented 1 year ago

@suprio-amzn, thank you for the response. Yes, this is on Aurora PostgreSQL. Should I open the ticket on the customer's account, or in mine?

My account does not have paid AWS support, but I think the customer does.

KushaalShroff commented 1 year ago

You can create with customer's account id

bachpro900 commented 9 months ago

Hello @calmitchell617

Did you manage the issue? I am getting exactly the same error when copying large data. I am thinking about exceeding destination memory, but not sure.