abel533 / Mapper

Mybatis Common Mapper - Easy to use
https://mybatis.io
MIT License
7.29k stars 1.62k forks source link

[insertList] [sql server] 不允许从数据类型 varbinary 到 datetime2 的隐式转换。请使用 CONVERT 函数来运行此查询。 #904

Open WangJincheng4869 opened 5 months ago

WangJincheng4869 commented 5 months ago

在 SQL server 2012 数据库中 insertList 方法会报错 不允许从数据类型 varbinary 到 datetime2 的隐式转换。请使用 CONVERT 函数来运行此查询。

触发条件

当多条数据中同时存在 datetime 类型,且其中有一个为 null 时就会触发。

以下情况不报错

@Test
public void  test(){
    DynamicDataSourceContextHolder.push("219bac8af4b940668e932798cdd8f2d9");
    MdCologIbTest mdCologIbTest = new MdCologIbTest(new Date(), null);
    MdCologIbTest mdCologIbTest1 = new MdCologIbTest(new Date(), null);
    List<MdCologIbTest> list = Arrays.asList(mdCologIbTest1, mdCologIbTest);
    mdCologIbTestMapper.insertList(list);
}
@Test
public void  test(){
    DynamicDataSourceContextHolder.push("219bac8af4b940668e932798cdd8f2d9");
    MdCologIbTest mdCologIbTest = new MdCologIbTest(new Date(), new Date());
    MdCologIbTest mdCologIbTest1 = new MdCologIbTest(new Date(), new Date());
    List<MdCologIbTest> list = Arrays.asList(mdCologIbTest1, mdCologIbTest);
    mdCologIbTestMapper.insertList(list);
}

以下情况报错

@Test
public void  test(){
    DynamicDataSourceContextHolder.push("219bac8af4b940668e932798cdd8f2d9");
    MdCologIbTest mdCologIbTest = new MdCologIbTest(new Date(), null);
    MdCologIbTest mdCologIbTest1 = new MdCologIbTest(new Date(), new Date());
    List<MdCologIbTest> list = Arrays.asList(mdCologIbTest1, mdCologIbTest);
    mdCologIbTestMapper.insertList(list);
}

依赖环境

JDK 17、spring boot 3.0.9、mapper 4.2.3、mybatis 3.5.13、mssql-jdbc 11.2.3.jre17

堆栈信息

org.springframework.jdbc.UncategorizedSQLException: 
### Error updating database.  Cause: com.microsoft.sqlserver.jdbc.SQLServerException: 不允许从数据类型 varbinary 到 datetime2 的隐式转换。请使用 CONVERT 函数来运行此查询。
### The error may exist in com/simperfect/bp/stat/dao/MdCologIbTestMapper.java (best guess)
### The error may involve com.simperfect.bp.stat.dao.MdCologIbTestMapper.insertList-Inline
### The error occurred while setting parameters
### SQL: INSERT INTO md_colog_ib_test  ( id,create_time,release_time )  VALUES   ( ?,?,? ) , ( ?,?,? )
### Cause: com.microsoft.sqlserver.jdbc.SQLServerException: 不允许从数据类型 varbinary 到 datetime2 的隐式转换。请使用 CONVERT 函数来运行此查询。
; uncategorized SQLException; SQL state [S0003]; error code [257]; 不允许从数据类型 varbinary 到 datetime2 的隐式转换。请使用 CONVERT 函数来运行此查询。

    at org.mybatis.spring.MyBatisExceptionTranslator.translateExceptionIfPossible(MyBatisExceptionTranslator.java:93)
    at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:439)
    at jdk.proxy2/jdk.proxy2.$Proxy115.insert(Unknown Source)
    at org.mybatis.spring.SqlSessionTemplate.insert(SqlSessionTemplate.java:272)
    at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:62)
    at org.apache.ibatis.binding.MapperProxy$PlainMethodInvoker.invoke(MapperProxy.java:142)
    at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:86)
    at jdk.proxy2/jdk.proxy2.$Proxy208.insertList(Unknown Source)
    at com.simperfect.bp.stat.MsSqlTest.test(MsSqlTest.java:33)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
    at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
    at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
    at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
    at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
    at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
    at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
    at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
    at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
    at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
    at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
    at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
    at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
    at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:217)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:213)
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:138)
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:68)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
    at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
    at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
    at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:147)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:127)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:90)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:55)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:102)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:54)
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
    at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
    at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53)
    at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:57)
    at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
    at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
    at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
    at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:232)
    at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55)
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: 不允许从数据类型 varbinary 到 datetime2 的隐式转换。请使用 CONVERT 函数来运行此查询。
    at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:265)
    at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1676)
    at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(SQLServerPreparedStatement.java:620)
    at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtExecCmd.doExecute(SQLServerPreparedStatement.java:540)
    at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7620)
    at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:3916)
    at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:268)
    at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:242)
    at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.execute(SQLServerPreparedStatement.java:518)
    at com.alibaba.druid.filter.FilterChainImpl.preparedStatement_execute(FilterChainImpl.java:3446)
    at com.alibaba.druid.filter.FilterEventAdapter.preparedStatement_execute(FilterEventAdapter.java:434)
    at com.alibaba.druid.filter.FilterChainImpl.preparedStatement_execute(FilterChainImpl.java:3444)
    at com.alibaba.druid.proxy.jdbc.PreparedStatementProxyImpl.execute(PreparedStatementProxyImpl.java:158)
    at com.alibaba.druid.pool.DruidPooledPreparedStatement.execute(DruidPooledPreparedStatement.java:483)
    at org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:48)
    at org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:75)
    at org.apache.ibatis.executor.SimpleExecutor.doUpdate(SimpleExecutor.java:50)
    at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:117)
    at org.apache.ibatis.executor.CachingExecutor.update(CachingExecutor.java:76)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:61)
    at jdk.proxy2/jdk.proxy2.$Proxy207.update(Unknown Source)
    at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:197)
    at org.apache.ibatis.session.defaults.DefaultSqlSession.insert(DefaultSqlSession.java:184)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:425)
    ... 77 more
abel533 commented 5 months ago

如何解决呢?

WangJincheng4869 commented 5 months ago

如何解决呢?

很抱歉,能力有限,研究一下午了,没有头绪,还在跟......

abel533 commented 5 months ago

这种语法没法 if null,用batch也不适合。

WangJincheng4869 commented 5 months ago

这种语法没法 if null,用batch也不适合。

现在还不清楚问题是出在 mybatis 上还是,驱动上面。我在想在参数上添加 jdbcType 会不会解决这个问题。

我记得应该是可以在#{record.createTime} 里面指定类型,正在想办法测。

<bind name="listNotEmptyCheck"
  value="@tk.mybatis.mapper.util.OGNL@notEmptyCollectionCheck(list, 'com.simperfect.bp.stat.dao.MdCologIbTestMapper.insertList 方法参数为空')" />
INSERT INTO md_colog_ib_test <trim prefix="(" suffix=")" suffixOverrides=",">id,create_time,release_time,</trim> VALUES
<foreach collection="list" item="record" separator=",">
  <trim prefix="(" suffix=")" suffixOverrides=",">#{record.id},#{record.createTime},#{record.releaseTime},</trim>
</foreach>
abel533 commented 5 months ago

GitHub Copilot

在批量插入数据时,如果其中一个日期为null,可能会导致错误。这是因为SQL Server不允许将null插入到datetime2类型的列中,除非该列已经被设置为允许null值。

解决这个问题的一种方法是在插入数据之前检查日期是否为null。如果日期为null,你可以选择跳过这条数据,或者将日期设置为一个默认值。

以下是如何在Java代码中实现这个逻辑的示例:

List<YourObject> objects = ...; // your list of objects to insert
objects = objects.stream()
    .filter(obj -> obj.getCreateTime() != null && obj.getReleaseTime() != null)
    .collect(Collectors.toList());

在这个示例中,我们使用Java 8的流(Stream)API来过滤掉那些createTimereleaseTime为null的对象。然后,你可以将过滤后的列表objects用于批量插入。

如果你想为null的日期设置一个默认值,你可以使用Optional类的orElse方法:

for (YourObject obj : objects) {
    LocalDateTime defaultTime = LocalDateTime.of(2000, 1, 1, 0, 0); // default date
    obj.setCreateTime(Optional.ofNullable(obj.getCreateTime()).orElse(defaultTime));
    obj.setReleaseTime(Optional.ofNullable(obj.getReleaseTime()).orElse(defaultTime));
}

在这个示例中,我们为createTimereleaseTime设置了一个默认的日期时间(2000年1月1日)。如果这些字段为null,它们将被设置为这个默认值。

WangJincheng4869 commented 5 months ago

设置默认值和排除null,都不太好,默认值肯定是不符合业务场景的。 我还有个想法就是,如果添加类型不行就拆分批量插入sql,手写个工具类,把为null的时间字段进行分类,然后再分别调用 insertList。当然,这样做非常恶心,因为不一定会有多少个时间字段,组合起来很恐怖.....

WangJincheng4869 commented 5 months ago

添加类型是可以的,但是不能存在主键字段,否则会报错不允许插入主键:当 IDENTITY_INSERT 设置为 OFF 时,不能为表 'md_colog_ib_test' 中的标识列插入显式值。

@Insert("<script>" +
        "INSERT INTO md_colog_ib_test <trim prefix=\"(\" suffix=\")\" suffixOverrides=\",\">create_time,release_time,</trim> VALUES" +
        "<foreach collection=\"list\" item=\"record\" separator=\",\">" +
        "<trim prefix=\"(\" suffix=\")\" suffixOverrides=\",\">#{record.createTime,jdbcType=TIMESTAMP},#{record.releaseTime,jdbcType=TIMESTAMP},</trim>" +
        "</foreach>" +
        "</script>")
void saveBatch(List<MdCologIbTest> list);
WangJincheng4869 commented 5 months ago

临时解决方案:

基于我之前自定义的 JakartaEntityResolve 完成强制适配,这个方案不适合全部人,也不适合提交 pr,因为它并不通用,简单粘贴一部分代码:

自定义的 JakartaEntityResolve

修改了 processField 中的部分代码,在 if (field.isAnnotationPresent(ColumnType.class)) 添加 else 分支,强制为 date 设置默认值

if (field.isAnnotationPresent(ColumnType.class)) {
    ColumnType columnType = field.getAnnotation(ColumnType.class);
    // 是否为 blob 字段
    entityColumn.setBlob(columnType.isBlob());
    // column可以起到别名的作用
    if (StringUtil.isEmpty(columnName) && StringUtil.isNotEmpty(columnType.column())) {
        columnName = columnType.column();
    }
    if (columnType.jdbcType() != JdbcType.UNDEFINED) {
        entityColumn.setJdbcType(columnType.jdbcType());
    }
    if (columnType.typeHandler() != UnknownTypeHandler.class) {
        entityColumn.setTypeHandler(columnType.typeHandler());
    }
} else {
    // 如果是时间类型则设置类型,防止 SQL server 下报错 https://github.com/abel533/Mapper/issues/904
    if (field.getJavaType() == Date.class || field.getJavaType() == java.sql.Date.class) {
        entityColumn.setJdbcType(JdbcType.TIMESTAMP);
    }
}

完全重写了 insertList 方法,强制不写入主键(仅适合我的场景)

public String insertList(MappedStatement ms) {
    final Class<?> entityClass = getEntityClass(ms);
    //开始拼sql
    StringBuilder sql = new StringBuilder();
    sql.append("<bind name=\"listNotEmptyCheck\" value=\"@tk.mybatis.mapper.util.OGNL@notEmptyCollectionCheck(list, '").append(ms.getId()).append(" 方法参数为空')\"/>");
    sql.append(SqlHelper.insertIntoTable(entityClass, tableName(entityClass), "list[0]"));
    if (DataSourceUtils.isDm()) {
        sql.append(DmSqlHelper.insertColumns(entityClass, true, false, false));
        sql.append(" VALUES ");
        sql.append("<foreach collection=\"list\" item=\"record\" separator=\",\" >");
        sql.append("<trim prefix=\"(\" suffix=\")\" suffixOverrides=\",\">");
        //获取全部列
        Set<EntityColumn> columnList = EntityHelper.getColumns(entityClass);
        //当某个列有主键策略时,不需要考虑他的属性是否为空,因为如果为空,一定会根据主键策略给他生成一个值
        for (EntityColumn column : columnList) {
            if (!column.isIdentity() && column.isInsertable()) {
                sql.append(column.getColumnHolder("record")).append(",");
            }
        }
    } else {
        sql.append(SqlHelper.insertColumns(entityClass, true, false, false));
        sql.append(" VALUES ");
        sql.append("<foreach collection=\"list\" item=\"record\" separator=\",\" >");
        sql.append("<trim prefix=\"(\" suffix=\")\" suffixOverrides=\",\">");
        //获取全部列
        Set<EntityColumn> columnList = EntityHelper.getColumns(entityClass);
        //当某个列有主键策略时,不需要考虑他的属性是否为空,因为如果为空,一定会根据主键策略给他生成一个值
        for (EntityColumn column : columnList) {
            if (!column.isId() && column.isInsertable()) {
                sql.append(column.getColumnHolder("record")).append(",");
            }
        }
    }
    sql.append("</trim>");
    sql.append("</foreach>");

    // 反射把MappedStatement中的设置主键名
    EntityHelper.setKeyProperties(EntityHelper.getPKColumns(entityClass), ms);

    return sql.toString();
}
LSL1618 commented 3 months ago

关于“SQLServerException: 不允许从数据类型 varbinary 到 datetime2 的隐式转换”异常的解决办法,可以在实体类属性上添加字段注解@ColumnType(jdbcType = JdbcType.TIMESTAMP)或者@ColumnType(jdbcType = JdbcType.DATE),也可以在实体类的XML映射文件上指明jdbcType类似这样<result column="created_at" jdbcType="TIMESTAMP" property="createdAt" />,能解决sqlserver数据库表中多个时间类型字段值为null在插入或更新报错的情况。