Srpc无法正确生成官方hbase.thrift的srpc客户端

archliu321 commented 8 months ago

详细描述:

srpc工具版本: srpc_generator version 0.10.0
hbase版本及下载地址 https://archive.apache.org/dist/hbase/2.4.5/
hbase.thrift的路径: hbase-2.4.5/hbase-thrift/src/main/resources/org/apache/hadoop/hbase/thrift/Hbase.thrift
生成命令: srpc_generator thrift ./Hbase.thrift ./
问题描述:
- 能成功生成脚手架文件, 但是脚手架文件事实上无法编译通过
- 生成的Hbase.thrift.h里面有字符串错误, 会编译不过
- 手动解决上一步的问题后, 发现Hbase.thrift.h文件存在变量重复定义
- 手动解决上一步问题后, 发现Hbase.thrift.h文件, 似乎是不支持map?

archliu321 commented 8 months ago

手动解决上一步问题后, 发现Hbase.thrift.h文件问题较大, Request的参数部分或全部丢失, 不符合Hbase.thrift中的定义如下是Hbase.thrift中对getRows方法的定义如下是实际生成的Hbase.thrift.h中getRowsRequest的类

archliu321 commented 8 months ago

另外, 2.4.5版本的Hbase.thrift文件中的语法, 似乎与srpc官方示例中的thrift语法并不完全相同

Barenboim commented 8 months ago

你好，麻烦把你的.thrift文件贴一下！我们的这个解析是手工搞的，我看看这里是哪个数据类型不支持。

Barenboim commented 8 months ago

另外，可能先把注释去了再试一下。不知道是不是我们解析注释不太正确。

Barenboim commented 8 months ago

map我们好像是有解析的啊，之前有测试过map。

archliu321 commented 8 months ago

@Barenboim 我用的Hbase.thrift文件就是官方的原始文件, 从下面这个地址下载 https://archive.apache.org/dist/hbase/2.4.5/hbase-2.4.5-src.tar.gz 解压后Hbase.thrift的文件路径是 hbase-2.4.5/hbase-thrift/src/main/resources/org/apache/hadoop/hbase/thrift/Hbase.thrift

另外我尝试过注释全去掉, 并且删除了Hbase.thrift中定义的大部分方法, 保留一个最小版本的thrift文件, 重新生成后, 发现仍然会有方法请求的定义丢失参数的问题.

holmes1412 commented 8 months ago

hi，我们也复现了，主要是一些函数的参数换行解析不正确。我们再进一步定位看看还有什么地方解析有问题的，然后一起修复～感谢报出这个问题～

Barenboim commented 8 months ago

我们也确认了对map的支持。我们当时测试是用的thrift官方提供的一个文件，去掉内嵌python。以下的thrift文件我们可以完全支持：

/*
 * Licensed to the Apache Software Foundation (ASF) under one
 * or more contributor license agreements. See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership. The ASF licenses this file
 * to you under the Apache License, Version 2.0 (the
 * "License"); you may not use this file except in compliance
 * with the License. You may obtain a copy of the License at
 *
 *   http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing,
 * software distributed under the License is distributed on an
 * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 * KIND, either express or implied. See the License for the
 * specific language governing permissions and limitations
 * under the License.
 *
 * Contains some contributions under the Thrift Software License.
 * Please see doc/old-thrift-license.txt in the Thrift distribution for
 * details.
 */

namespace c_glib TTest
namespace cpp thrift.test
namespace delphi Thrift.Test
namespace go thrifttest
namespace java thrift.test
namespace js ThriftTest
namespace lua ThriftTest
namespace netstd ThriftTest
namespace perl ThriftTest
namespace php ThriftTest
namespace py ThriftTest
namespace py.twisted ThriftTest
namespace rb Thrift.Test
namespace st ThriftTest
namespace xsd test (uri = 'http://thrift.apache.org/ns/ThriftTest')

// Presence of namespaces and sub-namespaces for which there is
// no generator should compile with warnings only
namespace noexist ThriftTest
namespace cpp.noexist ThriftTest

namespace * thrift.test

/**
 * Docstring!
 */
enum Numberz
{
  ONE = 1,
  TWO,
  THREE,
  FIVE = 5,
  SIX,
  EIGHT = 8
}

const Numberz myNumberz = Numberz.ONE;
// the following is expected to fail:
// const Numberz urNumberz = ONE;

typedef i64 UserId

struct Bonk
{
  1: string message,
  2: i32 type
}

typedef map<string,Bonk> MapType

struct Bools {
  1: bool im_true,
  2: bool im_false,
}

struct Xtruct
{
  1:  string string_thing,
  4:  i8     byte_thing,
  9:  i32    i32_thing,
  11: i64    i64_thing
}

struct Xtruct2
{
  1: i8     byte_thing,  // used to be byte, hence the name
  2: Xtruct struct_thing,
  3: i32    i32_thing
}

struct Xtruct3
{
  1:  string string_thing,
  4:  i32    changed,
  9:  i32    i32_thing,
  11: i64    i64_thing
}

struct Insanity
{
  1: map<Numberz, UserId> userMap,
  2: list<Xtruct> xtructs
}

struct CrazyNesting {
  1: string string_field,
  2: optional set<Insanity> set_field,
  // Do not insert line break as test/go/Makefile.am is removing this line with pattern match
  3: required list<map<set<i32>, map<i32,set<list<map<Insanity,string>>>>>> list_field,
  4: binary binary_field
}

union SomeUnion {
  1: map<Numberz, UserId> map_thing,
  2: string string_thing,
  3: i32 i32_thing,
  4: Xtruct3 xtruct_thing,
  5: Insanity insanity_thing
}

exception Xception {
  1: i32 errorCode,
  2: string message
}

exception Xception2 {
  1: i32 errorCode,
  2: Xtruct struct_thing
}

struct EmptyStruct {}

struct OneField {
  1: EmptyStruct field
}

service ThriftTest
{
  /**
   * Prints "testVoid()" and returns nothing.
   */
  void         testVoid(),

  /**
   * Prints 'testString("%s")' with thing as '%s'
   * @param string thing - the string to print
   * @return string - returns the string 'thing'
   */
  string       testString(1: string thing),

  /**
   * Prints 'testBool("%s")' where '%s' with thing as 'true' or 'false'
   * @param bool  thing - the bool data to print
   * @return bool  - returns the bool 'thing'
   */
  bool         testBool(1: bool thing),

  /**
   * Prints 'testByte("%d")' with thing as '%d'
   * The types i8 and byte are synonyms, use of i8 is encouraged, byte still exists for the sake of compatibility.
   * @param byte thing - the i8/byte to print
   * @return i8 - returns the i8/byte 'thing'
   */
  i8           testByte(1: i8 thing),

  /**
   * Prints 'testI32("%d")' with thing as '%d'
   * @param i32 thing - the i32 to print
   * @return i32 - returns the i32 'thing'
   */
  i32          testI32(1: i32 thing),

  /**
   * Prints 'testI64("%d")' with thing as '%d'
   * @param i64 thing - the i64 to print
   * @return i64 - returns the i64 'thing'
   */
  i64          testI64(1: i64 thing),

  /**
   * Prints 'testDouble("%f")' with thing as '%f'
   * @param double thing - the double to print
   * @return double - returns the double 'thing'
   */
  double       testDouble(1: double thing),

  /**
   * Prints 'testBinary("%s")' where '%s' is a hex-formatted string of thing's data
   * @param binary  thing - the binary data to print
   * @return binary  - returns the binary 'thing'
   */
  binary       testBinary(1: binary thing),

  /**
   * Prints 'testStruct("{%s}")' where thing has been formatted into a string of comma separated values
   * @param Xtruct thing - the Xtruct to print
   * @return Xtruct - returns the Xtruct 'thing'
   */
  Xtruct       testStruct(1: Xtruct thing),

  /**
   * Prints 'testNest("{%s}")' where thing has been formatted into a string of the nested struct
   * @param Xtruct2 thing - the Xtruct2 to print
   * @return Xtruct2 - returns the Xtruct2 'thing'
   */
  Xtruct2      testNest(1: Xtruct2 thing),

  /**
   * Prints 'testMap("{%s")' where thing has been formatted into a string of 'key => value' pairs
   *  separated by commas and new lines
   * @param map<i32,i32> thing - the map<i32,i32> to print
   * @return map<i32,i32> - returns the map<i32,i32> 'thing'
   */
  map<i32,i32> testMap(1: map<i32,i32> thing),

  /**
   * Prints 'testStringMap("{%s}")' where thing has been formatted into a string of 'key => value' pairs
   *  separated by commas and new lines
   * @param map<string,string> thing - the map<string,string> to print
   * @return map<string,string> - returns the map<string,string> 'thing'
   */
  map<string,string> testStringMap(1: map<string,string> thing),

  /**
   * Prints 'testSet("{%s}")' where thing has been formatted into a string of values
   *  separated by commas and new lines
   * @param set<i32> thing - the set<i32> to print
   * @return set<i32> - returns the set<i32> 'thing'
   */
  set<i32>     testSet(1: set<i32> thing),

  /**
   * Prints 'testList("{%s}")' where thing has been formatted into a string of values
   *  separated by commas and new lines
   * @param list<i32> thing - the list<i32> to print
   * @return list<i32> - returns the list<i32> 'thing'
   */
  list<i32>    testList(1: list<i32> thing),

  /**
   * Prints 'testEnum("%d")' where thing has been formatted into its numeric value
   * @param Numberz thing - the Numberz to print
   * @return Numberz - returns the Numberz 'thing'
   */
  Numberz      testEnum(1: Numberz thing),

  /**
   * Prints 'testTypedef("%d")' with thing as '%d'
   * @param UserId thing - the UserId to print
   * @return UserId - returns the UserId 'thing'
   */
  UserId       testTypedef(1: UserId thing),

  /**
   * Prints 'testMapMap("%d")' with hello as '%d'
   * @param i32 hello - the i32 to print
   * @return map<i32,map<i32,i32>> - returns a dictionary with these values:
   *   {-4 => {-4 => -4, -3 => -3, -2 => -2, -1 => -1, }, 4 => {1 => 1, 2 => 2, 3 => 3, 4 => 4, }, }
   */
  map<i32,map<i32,i32>> testMapMap(1: i32 hello),

  /**
   * So you think you've got this all worked out, eh?
   *
   * Creates a map with these values and prints it out:
   *   { 1 => { 2 => argument,
   *            3 => argument,
   *          },
   *     2 => { 6 => <empty Insanity struct>, },
   *   }
   * @return map<UserId, map<Numberz,Insanity>> - a map with the above values
   */
  map<UserId, map<Numberz,Insanity>> testInsanity(1: Insanity argument),

  /**
   * Prints 'testMulti()'
   * @param i8 arg0 -
   * @param i32 arg1 -
   * @param i64 arg2 -
   * @param map<i16, string> arg3 -
   * @param Numberz arg4 -
   * @param UserId arg5 -
   * @return Xtruct - returns an Xtruct with string_thing = "Hello2, byte_thing = arg0, i32_thing = arg1
   *    and i64_thing = arg2
   */
  Xtruct testMulti(1: i8 arg0, 2: i32 arg1, 3: i64 arg2, 4: map<i16, string> arg3, 5: Numberz arg4, 6: UserId arg5),

  /**
   * Print 'testException(%s)' with arg as '%s'
   * @param string arg - a string indication what type of exception to throw
   * if arg == "Xception" throw Xception with errorCode = 1001 and message = arg
   * else if arg == "TException" throw TException
   * else do not throw anything
   */
  void testException(1: string arg) throws(1: Xception err1),

  /**
   * Print 'testMultiException(%s, %s)' with arg0 as '%s' and arg1 as '%s'
   * @param string arg - a string indicating what type of exception to throw
   * if arg0 == "Xception" throw Xception with errorCode = 1001 and message = "This is an Xception"
   * else if arg0 == "Xception2" throw Xception2 with errorCode = 2002 and struct_thing.string_thing = "This is an Xception2"
   * else do not throw anything
   * @return Xtruct - an Xtruct with string_thing = arg1
   */
  Xtruct testMultiException(1: string arg0, 2: string arg1) throws(1: Xception err1, 2: Xception2 err2)

  /**
   * Print 'testOneway(%d): Sleeping...' with secondsToSleep as '%d'
   * sleep 'secondsToSleep'
   * Print 'testOneway(%d): done sleeping!' with secondsToSleep as '%d'
   * @param i32 secondsToSleep - the number of seconds to sleep
   */
  oneway void testOneway(1:i32 secondsToSleep)
}

service SecondService
{
  /**
   * Prints 'testString("%s")' with thing as '%s'
   * @param string thing - the string to print
   * @return string - returns the string 'thing'
   */
  string secondtestString(1: string thing)
}

struct VersioningTestV1 {
       1: i32 begin_in_both,
       3: string old_string,
       12: i32 end_in_both
}

struct VersioningTestV2 {
       1: i32 begin_in_both,

       2: i32 newint,
       3: i8 newbyte,
       4: i16 newshort,
       5: i64 newlong,
       6: double newdouble
       7: Bonk newstruct,
       8: list<i32> newlist,
       9: set<i32> newset,
       10: map<i32, i32> newmap,
       11: string newstring,
       12: i32 end_in_both
}

struct ListTypeVersioningV1 {
       1: list<i32> myints;
       2: string hello;
}

struct ListTypeVersioningV2 {
       1: list<string> strings;
       2: string hello;
}

struct GuessProtocolStruct {
  7: map<string,string> map_field,
}

struct LargeDeltas {
  1: Bools b1,
  10: Bools b10,
  100: Bools b100,
  500: bool check_true,
  1000: Bools b1000,
  1500: bool check_false,
  2000: VersioningTestV2 vertwo2000,
  2500: set<string> a_set2500,
  3000: VersioningTestV2 vertwo3000,
  4000: list<i32> big_numbers
}

struct NestedListsI32x2 {
  1: list<list<i32>> integerlist
}
struct NestedListsI32x3 {
  1: list<list<list<i32>>> integerlist
}
struct NestedMixedx2 {
  1: list<set<i32>> int_set_list
  2: map<i32,set<string>> map_int_strset
  3: list<map<i32,set<string>>> map_int_strset_list
}
struct ListBonks {
  1: list<Bonk> bonk
}
struct NestedListsBonk {
  1: list<list<list<Bonk>>> bonk
}

struct BoolTest {
  1: optional bool b = true;
  2: optional string s = "true";
}

struct StructA {
  1: required string s;
}

struct StructB {
  1: optional StructA aa;
  2: required StructA ab;
}

struct OptionalSetDefaultTest {
  1: optional set<string> with_default = [ "test" ]
}

struct OptionalBinary {
  1: optional set<binary> bin_set = {}
  2: optional map<binary,i32> bin_map = {}
}

holmes1412 commented 8 months ago

hi，这个问题已经修复了，麻烦试试：https://github.com/sogou/srpc/pull/352

archliu321 commented 8 months ago

@Barenboim @holmes1412 非常感谢两位大佬的高效支持, 修复后的代码确实可以正常生成thrift客户端脚手架、且编译不报错. 我基于修改后的srpc_generator, 将2.4.5版本的Hbase.thrift生成srpc客户端, 然而接下来仍然遇到了2个问题:

1.srpc_generator生成的Hbase.thrift.h, 似乎存在接口方法函数丢失参数的情形?

可以看到Hbase.thrift文件中, 定义的getRow方法有3个入参数, 第3个是"3:map<Text, Text> attributes"
但是生成的Hbase.thrift.h文件中, class getRowRequest类中只存在2个参数, 缺少了一个attributes
我对thrift知之甚少, 不太确定上述缺参数的现象是正常情况, 还是确实存在问题. (我之前提到的似乎不支持map, 也正是因为看到了attributes参数缺失的现象)

2.我用srpc生成的Hbase.srpc.h客户端, 尝试去调用不缺参数的方法访问hbase, 被hbase服务端拒绝, 似乎协议没有对齐? 我不太清楚srpc的thrift客户端背后会做什么, 以及这个问题是不是srpc的thrift客户端导致的.

确认前提1: 确认本地下载的Hbase.thrift和hbase服务端都是2.4.5版本的, 且通过thrift生成的Hbase同步客户端, 可以正常读取hbase的存储内容信息
确认前提2: 调用Hbase.thrift中定义的一个简单方法getColumnDescriptors, 确认Hbase.thrift.h中生成的getColumnDescriptors方法对应的class getColumnDescriptorsRequest和class getColumnDescriptorsResponse肉眼看上去不存在缺成员变量的情况
- 确认前提3: 测试代码下
报错现场: -- srpc错误码: 100 -- errorno: 104 (errno104 : Connection reset by peer) -- srpc errmsg: System Error. Use get_error() to get errno
srpc生成的thrift客户端与hbase服务端的tcp连接现场, 其中10.88.0.2是客户端, 172.16.96.126是hbase的服务端, 可以看到在客户端发送请求给hbase服务端后, 连接建立成功, 然后被hbase服务端关闭

Barenboim commented 8 months ago

接口不一致问题我们看一下。无法访问的问题，需要确认一下，我们会产生thrift协议的框架和srpc thrift框架。只有thrift协议框架才是你要的，你确认一下没用错。

Barenboim commented 8 months ago

你可以看一下tutorial里的例子，07和08才是thrift协议的框架，用的thrift framed binary通信。03和04并不是，那个是srpc自己的协议加上thrift idl。

archliu321 commented 8 months ago

好的, 我再确认一下用法

holmes1412 commented 8 months ago

第一个少了参数的问题也已经定位到了，确实还是解析的问题。马上修复一下～感谢～

第二个问题里的用法可以参考生成出来的client.thrift_skeleton.cc代码～

holmes1412 commented 8 months ago

@archliu321 第一个参数的问题也修复了，是map解析的问题。我把其中一个带了attribute的结构单独提取出来，检查了下它生成的代码，包括xxx.thrift.h和xxx.srpc.h里的结构定义与接口定义，并且分别修改了skeleton.cc中的代码，填了attribute中的内容，运行起来互相通信成功，所以看起来是ok了。麻烦再试一下现在的PR里的代码，如果发现其他问题麻烦再给我们反馈哈～感谢～

archliu321 commented 8 months ago

@holmes1412 非常感谢如此迅速的修复. 我切换为最新的srpc代码并生成了hbase_thrift_client, 发hbase_thrift_client可以连接上hbase服务端, 但是在 hbase_thrift_client发送完请求后, hbase服务端不会回包, hbase_thrift_client在等待超时后关闭, 不符合预期.

为了解决上述问题, 我做了一些测试, 目前发现用‘srpc生成的hbase_thrift客户端’与‘thrift生成的hbase_thrift客户端’相比, 客户端与服务端在tcp连接建立完成后, 正式的数据时tcp的载荷在头部存在几个字节的微小差异, srpc生成的客户端多发了“00 00 00 b1”. 抓包现场如下:

其他测试信息:

srpc生成的hbase_thrift客户端查询服务端的代码, 这次确保调用的是‘Hbase::ThriftClient’ 真正的thrifit客户端, 如下 2.用srpc生成的客户端和用thrift生成的客户端, 查询库的row_key、hbase_column_names_等确保是完全一致的, 这个从截图的tcp载荷上可以得到验证, 两次测试除了首部的“00 00 00 b1”, 剩下的内容完全一致.

我不太了解thrift的协议, 所以不清楚这种差异是否是协议版本差异导致的, 或者是其他原因另附上抓包现场文本:

srpc生成的hbase_thrift客户端发送载荷 0000 be 4b 3f e5 bf d2 f6 42 a3 fe ff 7d 08 00 45 00 .K?....B...}..E. 0010 00 e9 dc e5 40 00 40 06 46 41 0a 58 00 02 ac 10 ....@.@.FA.X.... 0020 60 7e a3 e6 17 74 50 89 65 dc 51 80 39 9a 80 18 `~...tP.e.Q.9... 0030 00 e5 17 c4 00 00 01 01 08 0a 3f 08 8e dc b9 b2 ..........?..... 0040 a5 e9 00 00 00 b1 80 01 00 01 00 00 00 12 67 65 ..............ge 0050 74 52 6f 77 73 57 69 74 68 43 6f 6c 75 6d 6e 73 tRowsWithColumns 0060 00 00 00 00 0b 00 01 00 00 00 14 7a 68 69 68 75 ...........zhihu 0070 5f 69 6e 76 65 72 74 5f 64 65 74 61 69 6c 73 0f _invert_details. 0080 00 02 0b 00 00 00 01 00 00 00 20 38 35 31 32 61 .......... 8512a 0090 37 30 33 61 34 33 38 37 66 65 38 39 30 64 66 62 703a4387fe890dfb 00a0 39 39 33 30 38 35 38 37 34 31 65 0f 00 03 0b 00 9930858741e..... 00b0 00 00 04 00 00 00 09 69 6e 76 5f 64 3a 75 69 64 .......inv_d:uid 00c0 00 00 00 09 69 6e 76 5f 64 3a 75 72 6c 00 00 00 ....inv_d:url... 00d0 0b 69 6e 76 5f 64 3a 74 69 74 6c 65 00 00 00 0d .inv_d:title.... 00e0 69 6e 76 5f 64 3a 63 6f 6e 74 65 6e 74 0d 00 04 inv_d:content... 00f0 0b 0b 00 00 00 00 00 .......
thrift生成的hbase_thrift客户端发送载荷 0000 be 4b 3f e5 bf d2 f6 42 a3 fe ff 7d 08 00 45 00 .K?....B...}..E. 0010 00 e5 14 44 40 00 40 06 0e e7 0a 58 00 02 ac 10 ...D@.@....X.... 0020 60 7e 9c 14 17 74 db 57 27 cd 6e 96 08 22 80 18 `~...t.W'.n..".. 0030 00 e5 17 c0 00 00 01 01 08 0a 3e e9 dc f8 b9 93 ..........>..... 0040 f4 06 80 01 00 01 00 00 00 12 67 65 74 52 6f 77 ..........getRow 0050 73 57 69 74 68 43 6f 6c 75 6d 6e 73 00 00 00 00 sWithColumns.... 0060 0b 00 01 00 00 00 14 7a 68 69 68 75 5f 69 6e 76 .......zhihu_inv 0070 65 72 74 5f 64 65 74 61 69 6c 73 0f 00 02 0b 00 ert_details..... 0080 00 00 01 00 00 00 20 38 35 31 32 61 37 30 33 61 ...... 8512a703a 0090 34 33 38 37 66 65 38 39 30 64 66 62 39 39 33 30 4387fe890dfb9930 00a0 38 35 38 37 34 31 65 0f 00 03 0b 00 00 00 04 00 858741e......... 00b0 00 00 09 69 6e 76 5f 64 3a 75 69 64 00 00 00 09 ...inv_d:uid.... 00c0 69 6e 76 5f 64 3a 75 72 6c 00 00 00 0b 69 6e 76 inv_d:url....inv 00d0 5f 64 3a 74 69 74 6c 65 00 00 00 0d 69 6e 76 5f d:title....inv 00e0 64 3a 63 6f 6e 74 65 6e 74 0d 00 04 0b 0b 00 00 d:content....... 00f0 00 00 00 ...

holmes1412 commented 8 months ago

你好，感谢这么详细的debug信息~

srpc框架中支持的thrift协议是Framed协议，thrift的RPC协议有很多种，Framed相比于传统的 Buffered(Unframed) 协议，编码是先写size，再写payload，所以前面会有4个字节的数据用于存储Protocol层的字节串长度，你上面的b1就是后面177的payload长度。

关于thrift协议的一个参考：https://anthony-dong.github.io/2022/03/20/1fbc1901406195cf47c58e7436468f2e/

我可以查一下hbase如何支持framed的thrift协议，看看有什么解决办法～

Barenboim commented 8 months ago

是的，异步只能采用Framed的协议。如果hbase使用的是标准的thrift server应该可以配置成这种协议。

archliu321 commented 8 months ago

@Barenboim @holmes1412 感谢两位大佬的回答, 很好的解决了我对thrift协议的疑惑, 看来是我用到的hbase自身的局限, 非常感谢. 希望srpc和workflow影响力越来越大, 希望更多的小伙伴能用上srpc和workflow.

Barenboim commented 8 months ago

你先试一下改配置把程序跑通啊，我查了一下，是可以改的。

---原始邮件--- 发件人: @.> 发送时间: 2023年12月25日(周一) 中午11:42 收件人: @.>; 抄送: @.**@.>; 主题: Re: [sogou/srpc] Srpc无法正确生成官方hbase.thrift的srpc客户端 (Issue #351)

@Barenboim @holmes1412 感谢两位大佬的回答, 很好的解决了我对thrift协议的疑惑, 看来是我用到的hbase自身的局限, 非常感谢. 希望srpc和workflow影响力越来越大, 希望更多的小伙伴能用上srpc和workflow.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

holmes1412 commented 8 months ago

@archliu321 不客气的哈，随时交流的～这个问题也有解决办法：可以通过修改hbase的配置，让hbase支持framed的thrift协议，配置项hbase.regionserver.thrift.framed配置为true即可。比如

<property> 
  <name>hbase.regionserver.thrift.framed</name> 
  <value>true</value> 
</property> 
<property> 
  <name>hbase.regionserver.thrift.framed.max_frame_size_in_mb</name> 
  <value>1024</value> 
</property> 
<property> 
  <name>hbase.regionserver.thrift.compact</name> 
  <value>true</value> 
</property>

参考：https://docs.cloudera.com/runtime/7.2.0/troubleshooting-hbase/topics/hbase-thrift-server-crashes.html?

archliu321 commented 8 months ago

@Barenboim @holmes1412 我这边已经成功通过srpc生成的hbase_thrift客户端访问hbase server获取到了期望的数据, 非常感谢两位的帮助和支持.

另外需要额外注意一点, 因为srpc支持的是thrift framed binary protocol, hbase server的配置项不仅要 framed=true, 而且一定要 compact=false, 这样方能请求成功.

Barenboim commented 8 months ago

@Barenboim @holmes1412 我这边已经成功通过srpc生成的hbase_thrift客户端访问hbase server获取到了期望的数据, 非常感谢两位的帮助和支持.

另外需要额外注意一点, 因为srpc支持的是thrift framed binary protocol, hbase server的配置项不仅要 framed=true, 而且一定要 compact=false, 这样方能请求成功.

哈哈，太好了，帮了我们大忙。也麻烦 star一下workflow或srpc项目呀。

archliu321 commented 8 months ago

必须全部一键三连[呲牙][呲牙][呲牙]

sogou / srpc

Srpc无法正确生成官方hbase.thrift的srpc客户端 #351