airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
16.32k stars 4.16k forks source link

Improve error messages for MySQL source #27219

Open nataliekwong opened 1 year ago

nataliekwong commented 1 year ago

Improve error messages for the MySQL source! Some baselines:

Error type: Request timed out due to HikariPool (24% of errors)

Message: HikariPool-1 - Connection is not available, request timed out after 60001ms.

Suggestion: show the same guidance implemented here: https://github.com/airbytehq/airbyte/issues/25889

Error type: Network error (15% of errrors)

State code: 08S01; Message: Communications link failure. The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.

Open question - which could this relate to? Option 1: show the same guidance implemented here: https://github.com/airbytehq/airbyte/issues/25889 Option 2: Too many sockets open (is this an Airbyte-controlled error?) Option 3: Incorrect Unix socket file name or TCP/IP port number when trying to connect to the server. Also check that the TCP/IP port you are using has not been blocked by a firewall or port blocking service.

Error type: Security configuration invalid (9%)

Unsecured connection not allowed. If no SSH Tunnel set up, please use one of the following SSL modes: required, verify-ca, verify-identity

Suggestion: Default the SSL mode to "required" on the form.

Error type: Malformed database URL (9%)

State code: 08001; Message: Cannot load connection class because of underlying exception: com.mysql.cj.exceptions.WrongArgumentException: Malformed database URL, failed to parse the main URL sections. State code: 08001; Message: Cannot load connection class because of underlying exception: com.mysql.cj.exceptions.WrongArgumentException: Malformed database URL, failed to parse the connection string near ':mysql://{x}/{x}?user={x}&password={x}&sslMode=REQUIRED'.

Open question - which could this relate to? Option 1: Database name is entered incorrectly / no such database exists. User needs to fix the entered configuration. Option 2: MySQL JDBC jar file version does not match the MySQL database server version

Error type: Access denied (8%)

State code: 28000; Error code: 1045; Message: Access denied for user '{x}'@'{x}' (using password: YES) State code: 28000; Error code: 1045; Message: Access denied for user '{x}'@'{x} (using password: NO)

Open question - which could this relate to? Option 1: Don't use a no-password, anonymous user for authentication Option 2: Check your password (if using password). Otherwise, check your user has access to the database.

Error type: SSH configuration doesn't work (5%)

Could not connect with provided SSH configuration. Error: Timed out while opening a SSH Tunnel. Please double check the given SSH configurations and try again.

Open question: is there any recommendation to offer here other than to check the configuration? Is there anything we can log differently to gather more information?

Error Type: SSL connection required: (5%)

State code: 08001; Message: SSL Connection required, but not provided by server.

Suggestion: Failed to find a valid SSL configuration. Ensure your database is set up with SSL.

Error Type: SshException: No more authentication methods available (3%)

Could not connect with provided SSH configuration. Error: org.apache.sshd.common.SshException: No more authentication methods available

Open question: What does this mean?

Error Type: 42000 Unknown Database 1049 (2%)

State code: 42000; Error code: 1049; Message: Unknown database '{X}'

Suggestion: Failed to find database {x}. Verify the database you entered exists and the user has permission to access.

Anything with <=5 errors I skipped and did not include here, but can be seen here: https://docs.google.com/spreadsheets/d/1q9cybppuoWl7rgAdMOi3oxyazcnEkvcZZ4RiftDAbaY/edit?pli=1#gid=767914303

cc @prateekmukhedkar

andrewtheis commented 10 months ago

User in that 51% here. Trying to connect local Airbyte instance to local MySQL 8.0 database, receiving 08S01. Will search for another ETL solution.

nataliekwong commented 10 months ago

@andrewtheis We've seen that creating a dedicated user can sometimes resolve the issue. @prateekmukhedkar do you have any other suggestions for helping to resolve the error?

andrewtheis commented 10 months ago

Figured it out, issue was I had to use the Docker bridge hostname. I always forget about that 😅

nataliekwong commented 10 months ago

Glad to hear!