apache / doris

Apache Doris is an easy-to-use, high performance and unified analytics database.
https://doris.apache.org
Apache License 2.0
12.37k stars 3.22k forks source link

my Issue: no valid Basic authorization #6241

Open boboyunz opened 3 years ago

boboyunz commented 3 years ago

i want to add data with steam load ,but get this error no valid Basic authorization this is be log I0715 05:07:46.350615 17205 stream_load.cpp:186] new income streaming load request.id=7045db7eb458d83a-7180372f6e4996a3, job_id=-1, txn_id=-1, label=39c25a5c-7000-496e-a98e-348a264c81dd, db=QHYGJ, tbl=cargis2 W0715 05:07:46.350641 17205 stream_load.cpp:212] parse basic authorization failed.id=7045db7eb458d83a-7180372f6e4996a3, job_id=-1, txn_id=-1, label=39c25a5c-7000-496e-a98e-348a264c81dd

my request header: {Content-Type: ;charset=UTF-8 User-Agent: Mozilla/4.0 Content-Length: 175 expect: 100-continue format: json strip_outer_array: true jsonpaths: ["$.cph","$.cpys","$.dt","$.ptbm","$.alarm","$.sta","$.xslc","$.gc","$.sbsd","$.xssd","$.xslc","$.dr","$.lat","$.lon"] label: 39c25a5c-7000-496e-a98e-348a264c81dd Authorization: Basic *****

}

boboyunz commented 3 years ago

最后我的解决方式如下,供大家参考: 根据官方文档,fe收到stream load 请求后,只是做了负载均衡,随机选择一个BE,转发http请求,问题就出在转发请求会把Basic authorization丢掉,导致访问失败。所以我直接跳过FE,自己维护一个BE列表,随机选择一个来发送请求,这样就把问题解决了。

xy720 commented 3 years ago

你可以试试把fe.conf里加上enable_http_server_v2=false看看,可能是因为读取header里的Authorization信息时读错了,另外不要在这里暴露Authorization信息

hf200012 commented 3 years ago
boboyunz commented 3 years ago

你可以试试把fe.conf里加上enable_http_server_v2=false看看,可能是因为读取header里的Authorization信息时读错了,另外不要在这里暴露Authorization信息

这个配置貌似是把 httpv2 禁用了 如何能解决问题呢:)请赐教 Authorization信息 已经隐藏

hf200012 commented 3 years ago

This is not a bug, you can send out the entire operation process of your stream load, if you are using curl, send out the entire command

hf200012 commented 3 years ago

你可以试试把fe.conf里加上enable_http_server_v2=false看看,可能是因为读取header里的Authorization信息时读错了,另外不要在这里暴露Authorization信息

这个配置貌似是把 httpv2 禁用了 如何能解决问题呢:)请赐教 Authorization信息 已经隐藏

You can add me on WeChat and I will help you see your problem, 35926237

arjenzhou commented 2 years ago

https://github.com/apache/incubator-doris/blob/51e210869acfb6ba31971b13f4ebae287bae1c44/extension/DataX/doriswriter/src/main/java/com/alibaba/datax/plugin/writer/doriswriter/DorisWriterEmitter.java#L173

https://github.com/arjenzhou/porter/blob/9889ec2507850f199f7bcf28923268592e1a51ef/common/src/main/java/de/xab/porter/common/util/Https.java#L96-L107

It is because not handling 307 redirect correctly. You can refer the code above.