search

apache / doris

Apache Doris is an easy-to-use, high performance and unified analytics database.
https://doris.apache.org
Apache License 2.0
11.86k stars 3.12k forks source link

[Bug] Hive Catalog read broker file failed #34066

Open Jrebel-i opened 2 months ago

Jrebel-i commented 2 months ago

Search before asking

Version

2.0.7

What's Wrong?

mysql>  select  * from    hive.app.app_hotel_supplier_shopping_records_hotel_d 
where   day_num='2024-04-22' 
and     source = 'xxxxxxxxxxx' 
and     hotel_country_code = 'US' 
and     supplier_code in ('yyyyyyyyyyyyyy') limit 5;
+-------------+----------------+-------------------+------+----------+--------------------+-----------------+------------------+--------------------+-----------------+---------------+------------------------+----------+------------+------------+----------------------+------------------------------+----------------------------+----------------------------------+-------------------------------------+---------------+---------------+------------+
| source      | supplier_code  | advance_days_type | stay | hotel_id | hotel_country_code | number_of_rooms | number_of_adults | number_of_children | shopping_type   | is_batch_rate | is_supplier_batch_rate | is_cache | error_code | is_success | hotel_shopping_count | hotel_shopping_success_count | hotel_shopping_cache_count | hotel_shopping_buyer_batch_count | hotel_shopping_supplier_batch_count | pricing_count | booking_count | day_num    |
+-------------+----------------+-------------------+------+----------+--------------------+-----------------+------------------+--------------------+-----------------+---------------+------------------------+----------+------------+------------+----------------------+------------------------------+----------------------------+----------------------------------+-------------------------------------+---------------+---------------+------------+
| xxxxxxxxxxx | yyyyyyyyyyyyyy | 2                 |    1 | 11111111 | US                 |               1 |                1 |                  0 | PURCHASE_SOURCE | N             | N                      | N        | 0          | Y          |                    1 |                            1 |                          0 |                                0 |                                   0 |            10 |             1 | 2024-04-22 |
| xxxxxxxxxxx | yyyyyyyyyyyyyy | 3                 |    1 | 11111111 | US                 |               1 |                1 |                  0 | PURCHASE_SOURCE | Y             | Y                      | N        | 0          | Y          |                    1 |                            1 |                          0 |                                1 |                                   1 |            10 |             1 | 2024-04-22 |
| xxxxxxxxxxx | yyyyyyyyyyyyyy | -1~0              |    1 | 11111111 | US                 |               1 |                1 |                  0 | PURCHASE_SOURCE | Y             | UNKNOWN                | Y        | 0          | Y          |                    5 |                            5 |                          5 |                                5 |                                   0 |            10 |             1 | 2024-04-22 |
| xxxxxxxxxxx | yyyyyyyyyyyyyy | 15-30             |    1 | 11111111 | US                 |               1 |                1 |                  0 | PURCHASE_SOURCE | Y             | Y                      | N        | 0          | Y          |                    2 |                            2 |                          0 |                                2 |                                   2 |            10 |             1 | 2024-04-22 |
| xxxxxxxxxxx | yyyyyyyyyyyyyy | 15-30             |    1 | 11111111 | US                 |               1 |                1 |                  0 | PURCHASE_SOURCE | Y             | Y                      | N        | 0          | Y          |                    1 |                            1 |                          0 |                                1 |                                   1 |            10 |             1 | 2024-04-22 |
+-------------+----------------+-------------------+------+----------+--------------------+-----------------+------------------+--------------------+-----------------+---------------+------------------------+----------+------------+------------+----------------------+------------------------------+----------------------------+----------------------------------+-------------------------------------+---------------+---------------+------------+
5 rows in set (2.48 sec)

mysql>  select  * from    hive.app.app_hotel_supplier_shopping_records_hotel_d 
where   day_num='2024-04-22' 
and     source = 'xxxxxxxxxxx' 
and     hotel_country_code = 'US' 
and     supplier_code in ('yyyyyyyyyyyyyy') limit 5;
ERROR 1105 (HY000): errCode = 2, detailMessage = (10.150.30.42)[CANCELLED][INTERNAL_ERROR]Orc row reader nextBatch failed. reason = Failed to read jfs://hdfs/user/hive/warehouse/app.db/app_hotel_supplier_shopping_records_hotel_d/day_num=2024-04-22/part-00113-ffb5e688-5907-4f97-98e7-e57f2ea823bf-c000: [THRIFT_RPC_ERROR]read broker file failed, broker:TNetworkAddress(

What You Expected?

Data read normal

How to Reproduce?

No response

Anything Else?

No response

Are you willing to submit PR?

Code of Conduct

RookieRascal commented 2 months ago

Check HDFS Status: Ensure that the HDFS is operational and that there are no issues with reading from or writing to the HDFS. Verify File Existence: Check if the file part-00113-ffb5e688-5907-4f97-98e7-e57f2ea823bf-c000 exists in the specified directory jfs://hdfs/user/hive/warehouse/app.db/app_hotel_supplier_shopping_records_hotel_d/day_num=2024-04-22/. File Permissions: Ensure that the user running the query has the necessary permissions to read the file. Network Connection: Verify that there are no network issues between the client and the HDFS cluster. The error message mentions a potential Thrift RPC error, which could indicate a communication issue. Check for Data Corruption: There might be corruption in the file itself. You can attempt to read the file using other tools or inspect it directly in HDFS to see if it's intact. Consult Logs: Check the logs on the HDFS cluster, as well as any relevant logs on the MySQL server, for more detailed error messages or stack traces that might provide additional clues about the root cause of the issue. If these steps don't resolve the problem, you may need to involve your Hadoop administrator or database administrator for further investigation and assistance.