search

MyCATApache / Mycat-Server

GNU General Public License v2.0
9.51k stars 3.85k forks source link

load data infile 问题 #1546

Open weconquered opened 7 years ago

weconquered commented 7 years ago

版本： 1.6.5-beta 问题描述： 使用库内分表，配置了3张表 subTables="match$1-3"。 使用语句： insert into match(ModelKey,Name) values(1,3); insert into match(ModelKey,Name) values(2,3); insert into match(ModelKey,Name) values(3,3); 可以正常插入数据 然后验证使用： load data local infile "/Users/boloomo/Desktop/query_result.csv" replace into table match fields terminated by "," OPTIONALLY ENCLOSED BY '"' lines terminated by '\n' (ModelKey,Name) 

    文本数据：

1,3 3,3 2,3

    发现没有load进去， 去看日志发现：

2017-06-22 18:26:26.680 INFO [$_NIOREACTOR-3-RW] (io.mycat.net.AbstractConnection.close(AbstractConnection.java:510)) - close connection,reason:stream closed ,ServerConnection [id=1, schema=TESTDB, host=127.0.0.1, user=user,txIsolation=3, autocommit=true, schema=TESTDB] 2017-06-22 18:26:26.681 INFO [$_NIOREACTOR-1-RW] (io.mycat.net.handler.FrontendAuthenticator.success(FrontendAuthenticator.java:226)) - ServerConnection [id=3, schema=null, host=127.0.0.1, user=user,txIsolation=3, autocommit=true, schema=null]'user' login success

详细配置如下：

shema配置：

<dataNode name="dn1" dataHost="localhost1" database="testMycat" />

<dataHost name="localhost1" maxCon="1000" minCon="10" balance="0"
          writeType="0" dbType="mysql" dbDriver="native" switchType="1"  slaveThreshold="100">
    <heartbeat>select user()</heartbeat>
    <writeHost host="hostS1" url="192.168.129.88:3306" user="root"
               password="123456" />
</dataHost>

rule配置： mod-long

weconquered commented 7 years ago

经验证、分库情况下没有问题。

yanjunli commented 7 years ago

嗯.我也验证了一下,在分表情况下会有这个问题.

weconquered commented 7 years ago

在分库的情况下， load一个超过4G的文件会错. 2017-06-26 12:57:08.479 ERROR [$_NIOREACTOR-0-RW] (io.mycat.net.NIOReactor$RW.run(NIOReactor.java:132)) - caught err: java.lang.OutOfMemoryError at java.io.FileOutputStream.writeBytes(Native Method) ~[?:1.8.0_11] at java.io.FileOutputStream.write(FileOutputStream.java:307) ~[?:1.8.0_11] at java.io.ByteArrayOutputStream.writeTo(ByteArrayOutputStream.java:154) ~[?:1.8.0_11] at io.mycat.server.handler.ServerLoadDataInfileHandler.saveByteOrToFile(ServerLoadDataInfileHandler.java:289) ~[Mycat-server-1.6.5-BETA.jar:?] at io.mycat.server.handler.ServerLoadDataInfileHandler.handle(ServerLoadDataInfileHandler.java:254) ~[Mycat-server-1.6.5-BETA.jar:?] at io.mycat.net.FrontendConnection.loadDataInfileData(FrontendConnection.java:256) ~[Mycat-server-1.6.5-BETA.jar:?] at io.mycat.net.handler.FrontendCommandHandler.handle(FrontendCommandHandler.java:59) ~[Mycat-server-1.6.5-BETA.jar:?] at io.mycat.net.FrontendConnection.rawHandle(FrontendConnection.java:500) ~[Mycat-server-1.6.5-BETA.jar:?] at io.mycat.net.FrontendConnection.handle(FrontendConnection.java:482) ~[Mycat-server-1.6.5-BETA.jar:?] at io.mycat.net.AbstractConnection.onReadData(AbstractConnection.java:323) ~[Mycat-server-1.6.5-BETA.jar:?] at io.mycat.net.NIOSocketWR.asynRead(NIOSocketWR.java:196) ~[Mycat-server-1.6.5-BETA.jar:?] at io.mycat.net.AbstractConnection.asynRead(AbstractConnection.java:275) ~[Mycat-server-1.6.5-BETA.jar:?] at io.mycat.net.NIOReactor$RW.run(NIOReactor.java:102) ~[Mycat-server-1.6.5-BETA.jar:?] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_11] 2017-06-26 12:57:09.649 INFO [Timer0] (io.mycat.backend.datasource.PhysicalDatasource.closeByIdleMany(PhysicalDatasource.java:280)) - too many ilde cons ,close some for datasouce hostM1

drink-du commented 5 years ago

请问现在这个单库分表的load问题修复了没

  • © Githubissues.
  • Githubissues is a development platform for aggregating issues.