Open guofengzh opened 1 year ago
I get the following:
snikolaev@dev2:~/manticore-demo$ mvn test
[INFO] Scanning for projects...
[INFO]
[INFO] ---------------------< com.example:manticore-demo >---------------------
[INFO] Building manticore-demo 1.0.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-resources-plugin:3.3.1:resources (default-resources) @ manticore-demo ---
[INFO] Copying 1 resource from src/main/resources to target/classes
[INFO] Copying 1 resource from src/main/resources to target/classes
[INFO]
[INFO] --- maven-compiler-plugin:3.11.0:compile (default-compile) @ manticore-demo ---
[INFO] Changes detected - recompiling the module! :source
[INFO] Compiling 11 source files with javac [debug release 17] to target/classes
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] error: release version 17 not supported
[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.068 s
[INFO] Finished at: 2023-11-23T11:14:01Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.11.0:compile (default-compile) on project manticore-demo: Compilation failure
[ERROR] error: release version 17 not supported
[ERROR]
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
Is this expected? If not, is there a docker image I can git clone
and run the test in to make sure we are on the same page?
It seems that the version of your JDK is less than 17.
This is a Spring boot 3 project that requires at least Java 17.
Please also verify, according to the doc, that mysql_version_string is set to 5.0.37.
I just verify again that the project runs well on Java 17 and 21, on WIndows 10 here.
With openjdk 21 I get this:
# cat Dockerfile
FROM openjdk:21-bookworm
RUN cd / && \
wget https://repo.manticoresearch.com/manticore-repo.noarch.deb && \
dpkg -i manticore-repo.noarch.deb && \
apt -y update && \
apt -y install manticore manticore-extra vim maven mariadb-client
RUN mkdir -p /var/run/manticore
RUN sed -i 's/searchd {/searchd {\nmysql_version_string = 5.0.37\n/' /etc/manticoresearch/manticore.conf
RUN git clone https://github.com/guofengzh/manticore-demo.git /manticore-demo
docker build --load . -t issue_1618
docker run -it --rm issue_1618 bash
In the container:
searchd
mysql -P9306 -h0 -e "CREATE TABLE testrt (title TEXT, content TEXT, gid INT);"
cd /manticore-demo
mvn test
Result:
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v3.1.3)
2023-11-23T15:14:13.147Z INFO 128 --- [ main] c.e.m.m.TestRTMybatisTest : Starting TestRTMybatisTest using Java 21 with PID 128 (started by root in /manticore-demo)
2023-11-23T15:14:13.151Z INFO 128 --- [ main] c.e.m.m.TestRTMybatisTest : No active profile set, falling back to 1 default profile: "default"
2023-11-23T15:14:13.452Z INFO 128 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFAULT mode.
2023-11-23T15:14:13.495Z INFO 128 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 38 ms. Found 1 JPA repository interfaces.
2023-11-23T15:14:13.713Z INFO 128 --- [ main] o.h.j.i.u.LogHelper : HHH000204: Processing PersistenceUnitInfo [name: default]
2023-11-23T15:14:13.735Z INFO 128 --- [ main] o.h.Version : HHH000412: Hibernate ORM core version 6.2.7.Final
2023-11-23T15:14:13.736Z INFO 128 --- [ main] o.h.c.Environment : HHH000406: Using bytecode reflection optimizer
2023-11-23T15:14:13.822Z INFO 128 --- [ main] o.h.b.i.BytecodeProviderInitiator : HHH000021: Bytecode provider name : bytebuddy
2023-11-23T15:14:13.882Z INFO 128 --- [ main] o.s.o.j.p.SpringPersistenceUnitInfo : No LoadTimeWeaver setup: ignoring JPA class transformer
2023-11-23T15:14:13.995Z WARN 128 --- [ main] o.h.d.Dialect : HHH000511: The 5.0.37 version for [org.hibernate.dialect.MySQLDialect] is no longer supported, hence certain features may not work properly. The minimum supported version is 5.7.0. Check the community dialects project for available legacy versions.
2023-11-23T15:14:14.078Z INFO 128 --- [ main] o.h.b.i.BytecodeProviderInitiator : HHH000021: Bytecode provider name : bytebuddy
2023-11-23T15:14:14.364Z INFO 128 --- [ main] o.h.e.t.j.p.i.JtaPlatformInitiator : HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
2023-11-23T15:14:14.365Z INFO 128 --- [ main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2023-11-23T15:14:14.525Z INFO 128 --- [ main] o.s.d.j.r.q.QueryEnhancerFactory : Hibernate is in classpath; If applicable, HQL parser will be used.
2023-11-23T15:14:14.856Z INFO 128 --- [ main] c.e.m.m.TestRTMybatisTest : Started TestRTMybatisTest in 1.926 seconds (process running for 2.722)
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
WARNING: A Java agent has been loaded dynamically (/root/.m2/repository/net/bytebuddy/byte-buddy-agent/1.14.6/byte-buddy-agent-1.14.6.jar)
WARNING: If a serviceability tool is in use, please run with -XX:+EnableDynamicAgentLoading to hide this warning
WARNING: If a serviceability tool is not in use, please run with -Djdk.instrument.traceUsage for more information
WARNING: Dynamic loading of agents will be disallowed by default in a future release
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.737 s -- in com.example.manticore.mybatis.TestRTMybatisTest
[INFO] Running com.example.manticore.jpa.TestRTJPATest
2023-11-23T15:14:15.252Z INFO 128 --- [ main] t.c.s.AnnotationConfigContextLoaderUtils : Could not detect default configuration classes for test class [com.example.manticore.jpa.TestRTJPATest]: TestRTJPATest does not declare any static, non-private, non-final, nested classes annotated with @Configuration.
2023-11-23T15:14:15.256Z INFO 128 --- [ main] .b.t.c.SpringBootTestContextBootstrapper : Found @SpringBootConfiguration com.example.manticore.Application for test class com.example.manticore.jpa.TestRTJPATest
Hibernate: TRUNCATE TABLE testrt
Hibernate: INSERT INTO testrt(title, content, gid) VALUES( ?, ?, ?)
Hibernate: SELECT * FROM testrt WHERE MATCH(?)
Hibernate: SELECT * FROM testrt WHERE MATCH('list of acer') AND gid > ? ORDER BY gid DESC
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.112 s -- in com.example.manticore.jpa.TestRTJPATest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:13 min
[INFO] Finished at: 2023-11-23T15:14:15Z
[INFO] ------------------------------------------------------------------------
Query log:
root@d61aeaaa4d79:/manticore-demo# cat /var/log/manticore/query.log
/* Thu Nov 23 15:14:15.223 2023 conn 10 real 0.000 wall 0.000 found 0 */ SELECT * FROM testrt WHERE MATCH('Helios');
/* Thu Nov 23 15:14:15.240 2023 conn 10 real 0.000 wall 0.000 found 1 */ SELECT * FROM testrt WHERE MATCH('List of Acer');
/* Thu Nov 23 15:14:15.244 2023 conn 10 real 0.000 wall 0.000 found 1 */ SELECT * FROM testrt WHERE MATCH('list of acer') AND gid>5 ORDER BY gid DESC;
/* Thu Nov 23 15:14:15.331 2023 conn 10 real 0.000 wall 0.000 found 1 */ SELECT * FROM testrt WHERE MATCH('List of Acer');
/* Thu Nov 23 15:14:15.361 2023 conn 10 real 0.000 wall 0.000 found 1 */ SELECT * FROM testrt WHERE MATCH('list of acer') AND gid>5 ORDER BY gid DESC;
What is wrong here? Looks good to me at first glance.
Sorry I didn't make it clear.
The current code does not use HikariCP, so it works fine.
The improvement I hope for is that Manticore can support datasources like HikariCP so that Manticore can be easily used in Spring boot applications.
In DataSourceConfig.java, the following code:
@Bean
public DataSource dataSource(){
return createMyBatisPooledDataSource();
// Uncommenting the following statement will throw "java.sql.SQLException: Could not map transaction isolation '<empty>' to a valid JDBC level."
// return createHikariDataSource();
}
Modify it to:
@Bean
public DataSource dataSource(){
// return createMyBatisPooledDataSource();
// Uncommenting the following statement will throw "java.sql.SQLException: Could not map transaction isolation '<empty>' to a valid JDBC level."
return createHikariDataSource();
}
The problem I reported will occur:
java.sql.SQLException: Could not map transaction isolation '<empty>' to a valid JDBC level.
...
@guofengzh do you know what exactly it expects from mysql?
E.g. Manticore works like this:
root@4362a20a1625:/manticore-demo# mysql -P9306 -h0 -e "SELECT @@session.tx_isolation"
+------------------------+
| @@session.tx_isolation |
+------------------------+
| 0 |
+------------------------+
Is this what breaks it? What's expected instead? Can you please provide a full dialog between it and mysql/mariadb, so we can see what queries need to be handled differently for that to work?
From what you did, I see the result of my Manticore:
mysql> SELECT @@session.tx_isolation;
+------------------------+
| @@session.tx_isolation |
+------------------------+
| <empty> |
+------------------------+
This is the source code of com.mysql.cj.jdbc.ConnectionImpl.getTransactionIsolation(), which failed to work with HikariCP:
@Override
public int getTransactionIsolation() throws SQLException {
synchronized (getConnectionMutex()) {
if (!this.useLocalSessionState.getValue()) {
String s = this.session.queryServerVariable(
versionMeetsMinimum(8, 0, 3) || (versionMeetsMinimum(5, 7, 20) && !versionMeetsMinimum(8, 0, 0)) ? "@@session.transaction_isolation"
: "@@session.tx_isolation");
if (s != null) {
Integer intTI = mapTransIsolationNameToValue.get(s);
if (intTI != null) {
this.isolationLevel = intTI.intValue();
return this.isolationLevel;
}
throw SQLError.createSQLException(Messages.getString("Connection.12", new Object[] { s }), MysqlErrorNumbers.SQL_STATE_GENERAL_ERROR,
getExceptionInterceptor());
}
throw SQLError.createSQLException(Messages.getString("Connection.13"), MysqlErrorNumbers.SQL_STATE_GENERAL_ERROR, getExceptionInterceptor());
}
return this.isolationLevel;
}
}
By debugging, for my case, s is "\<empty>", The following statement failed to translate the value of s to the expected value:
Integer intTI = mapTransIsolationNameToValue.get(s);
mapTransIsolationNameToValue is of HashMap type, and defined statically in this class:
private static Map<String, Integer> mapTransIsolationNameToValue = null;
static {
mapTransIsolationNameToValue = new HashMap<>(8);
mapTransIsolationNameToValue.put("READ-UNCOMMITED", TRANSACTION_READ_UNCOMMITTED);
mapTransIsolationNameToValue.put("READ-UNCOMMITTED", TRANSACTION_READ_UNCOMMITTED);
mapTransIsolationNameToValue.put("READ-COMMITTED", TRANSACTION_READ_COMMITTED);
mapTransIsolationNameToValue.put("REPEATABLE-READ", TRANSACTION_REPEATABLE_READ);
mapTransIsolationNameToValue.put("SERIALIZABLE", TRANSACTION_SERIALIZABLE);
}
I set s to "READ-UNCOMMITTED" by the dubug tool, All the tests passed.
However, I don't know why on the MySQL terminal I can't set up the isolation level.
If the MySQL terminal is connected to MySQL server, I can use the following statement to set it:
SET GLOBAL TRANSACTION ISOLATION LEVEL READ COMMITTED;
Exit the terminal and reconnect to MySQL, I can see the isolation level is changed. But the above statement does not works with Manticore.
@sanikolaev, how can I set the Manticore transaction isolation level by the command line, or how can I change the value of @@session.tx_isolatio, so I can move on.
As documented, Manticore do not aim to provide isolation (it is overly isolated). so there will be no problem setting one of that values, like "READ-COMMITTED", although it is semantically incorrect, but very close.
My test codes in my project use different transaction to operate Manticore. If I place these operations in the same transaction, for example:
@Test
@Transactional
public void testIsolation() {
testRTJPAService.deleteAll();
TestRTJPA testRTJPA = new TestRTJPA();
testRTJPA.setTitle("List of Acer gaming laptops");
testRTJPA.setContent("Predator Helios Nitro");
testRTJPA.setGid(150);
int n = testRTJPAService.save(testRTJPA);
assertEquals(1, n);
List<TestRTJPA> testRTList = testRTJPAService.findMatched("List of Acer");
assertEquals(1, testRTList.size());
assertEquals("List of Acer gaming laptops", testRTList.get(0).getTitle());
}
The test will fail (because of the overly isolated).
So I think setting Manticore transaction isolation level default to "READ-COMMITTED" is acceptable, but this needs to be made explicit in the documentation.
how can I set the Manticore transaction isolation level by the command line
Manticore doesn't support isolation levels. We can only try to mock that, i.e.
SET GLOBAL TRANSACTION ISOLATION LEVEL READ COMMITTED
should not fail which is already okSELECT @@session.tx_isolation
should return READ-COMMITTED
or perhaps just not <empty>
Can you try the latest dev version? I see it the select
returns 0
there unlike the latest release:
snikolaev@dev2:~$ docker run -e EXTRA=1 --name manticore --rm -d manticoresearch/manticore && echo "Waiting for Manticore docker to start. Consider mapping the data_dir to make it start faster next time" && until docker logs manticore 2>&1 | grep -q "accepting connections"; do sleep 1; echo -n .; done && echo && docker exec -it manticore mysql && docker stop manticore
d1d672d24f42b2ac0960b1a9e3c9de4608ebf0016113ee7f64aa9fe1dc9ae618
Waiting for Manticore docker to start. Consider mapping the data_dir to make it start faster next time
..
mysql> SET GLOBAL TRANSACTION ISOLATION LEVEL READ COMMITTED;
mysql> SELECT @@session.tx_isolation;
+------------------------+
| @@session.tx_isolation |
+------------------------+
| <empty> |
+------------------------+
mysql> exit
manticore
snikolaev@dev2:~$ docker run -e EXTRA=1 --name manticore --rm -d manticoresearch/manticore:dev && echo "Waiting for Manticore docker to start. Consider mapping the data_dir to make it start faster next time" && until docker logs manticore 2>&1 | grep -q "accepting connections"; do sleep 1; echo -n .; done && echo && docker exec -it manticore mysql && docker stop manticore
29496da4cf4422c79143e781cb37c1aa75b50dac397de4ac9a5db33f3e2e09f4
Waiting for Manticore docker to start. Consider mapping the data_dir to make it start faster next time
...
mysql> SET GLOBAL TRANSACTION ISOLATION LEVEL READ COMMITTED;
mysql> SELECT @@session.tx_isolation;
+------------------------+
| @@session.tx_isolation |
+------------------------+
| 0 |
+------------------------+
Tried the latest dev version. The error is:
java.sql.SQLException: Could not map transaction isolation '0' to a valid JDBC level.
From the analysis here, we know that @@session.tx_isolation must be a specific string, just like in mysql (here is the output of MySQL 8):
mysql> select @@session.transaction_isolation;
+---------------------------------+
| @@session.transaction_isolation |
+---------------------------------+
| REPEATABLE-READ |
+---------------------------------+
1 row in set (0.00 sec)
I was trying to use manticore with spring data JPA, but Hikari-cp was unable to establish connection witn manticore instance. Probably manticore is returning some empty string to hikari-cp while creating a connection, I have attached log traces below.
Link to repository to reproduce the issue https://github.com/ayaankhan98/manticore-hikari-cp
The exception
java.lang.NumberFormatException: For input string: "<empty>"
indicates that mysql_version_string is not set or the value is incorrect, see Notes on MySQL connectors.
However, after solving this problem, Manticore still cannot work on Spring boot's default data source. See this project for a workarround solution.
As @Nick-S-2018 assumes, this may require a change in the daemon. @Nick-S-2018 is going to look further into it.
Is your feature request related to a problem? Please describe. Manticore failed to work with HikariCP so that it is not easier to use it on Spring boot platform with MySQL connectors. HikariCP is a popular and default data source used on Spring boot platform.
Describe the solution you'd like It is acceptable that Manticore does not support full MySQL wire protocol, but it might be good to implement some features, like version and isolation level requests so that we can use JPA or Mybatis to access Manticore easier.
Describe alternatives you've considered N/A
Additional context Here is a project to demostrate the use of JPA and MyBatis with Manticore, but it uses a simple data source other than HikariCP. It also demostrates what error throws when using HikariCP.