com.alibaba.otter.node.etl.select.exceptions.SelectException: java.lang.NullPointerException

luyee commented 6 years ago

有两个pipeline(单向，mysql5.7.16-log)跑着跑着突然就挂起，报了这个错，没找到其他相关日志。。

pid:15 nid:1 exception:setl:com.alibaba.otter.node.etl.select.exceptions.SelectException: java.lang.NullPointerException
at com.alibaba.otter.node.etl.select.selector.MessageParser.parse(MessageParser.java:209)
at com.alibaba.otter.node.etl.select.selector.canal.CanalEmbedSelector.selector(CanalEmbedSelector.java:258)
at com.alibaba.otter.node.etl.select.SelectTask.processSelect(SelectTask.java:236)
at com.alibaba.otter.node.etl.select.SelectTask.access$300(SelectTask.java:94)
at com.alibaba.otter.node.etl.select.SelectTask$1.run(SelectTask.java:208)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at com.alibaba.otter.node.etl.select.selector.MessageParser.internParse(MessageParser.java:378)
at com.alibaba.otter.node.etl.select.selector.MessageParser.parse(MessageParser.java:181)

luyee commented 6 years ago

加了log,这个DataMedia为null了。看到是ddl相关的(源/目标库名，表名均相同)，关闭ddl与跳过ddl异常，正常后，再开启 ddl 恢复了

DataMedia dataMedia = ConfigHelper.findSourceDataMedia(pipeline,
    schemaName,
    tableName,
    notExistReturnNull);
logger.warn("#####     schemaName: "+schemaName+" tableName: "+tableName+" notExistReturnNull "+notExistReturnNull+" dataMedia:"+ JSON.toJSONString(dataMedia));
// 如果EventType是CREATE/ALTER，需要reload
// DataMediaInfo;并且把CREATE/ALTER类型的事件丢弃掉.
if (dataMedia != null && (eventType.isCreate() || eventType.isAlter() || eventType.isRename())) {
    DbDialect dbDialect = dbDialectFactory.getDbDialect(pipeline.getId(),
        (DbMediaSource) dataMedia.getSource());
    dbDialect.reloadTable(schemaName, tableName);// 更新下meta信息
}

boolean ddlSync = pipeline.getParameters().getDdlSync();
if (ddlSync) {
    // 处理下ddl操作
    EventData eventData = new EventData();
    eventData.setSchemaName(schemaName);
    eventData.setTableName(tableName);
    eventData.setEventType(eventType);
    eventData.setExecuteTime(entry.getHeader().getExecuteTime());
    eventData.setSql(rowChange.getSql());
    eventData.setDdlSchemaName(rowChange.getDdlSchemaName());
    eventData.setTableId(dataMedia.getId());
    return Arrays.asList(eventData);

iushas commented 6 years ago

解决了吗兄弟，什么原因，遇到了类似的错误

alibaba / otter

com.alibaba.otter.node.etl.select.exceptions.SelectException: java.lang.NullPointerException #393