vmware-archive / sql-to-dbsp-compiler

Compiler translating SQL view definitions into DBSP circuits (https://github.com/vmware/database-stream-processor)
Other
28 stars 4 forks source link

Extract code related to sqllogictest to separate library #138

Closed dssysolyatin closed 1 year ago

dssysolyatin commented 1 year ago

Hi! First, your project is awesome! I read an article and it looks very promising!

One of the project in our company can be expose to an user as postgres server (it supports postgres wire protocol and postgres syntax) and I was trying to test it using original sqllogictest util. But I faced with issue that if I run sqllogictest util using -odbc option it generates different hashes for test/select*.test even if use raw postgres. I asked help at sqllite forum about this problem - https://sqlite.org/forum/forumpost/36b1c1fd70, but unfortunately, I have not yet received any responses

I tried your JDBCExecutor and it works as expected! It would be nice if it could be extracted to separate library.

mihaibudiu commented 1 year ago

Indeed, my plan is to refactor out the slt testing program into a separate project and submit it as a PR to the Calcite infrastructure project. In principle the base layer should be independent on both DBSP and Calcite, and you can implement the required interfaces for your own executor.

To debug problems with the hashes it actually helps to dump the whole output for each query, then you can usually figure out pretty quickly what the problem is by comparing the SLT official output and your own output.

To dump the output from SLT I added this line in the file md5.c:

void md5_add(const char *z){
  fprintf(stderr, ">%s<", z); // add this 
  if( !isInit ){

I also notice that I had to change SQLINTEGER to SQLLEN in a couple of places in slt_odbc3.c

mihaibudiu commented 1 year ago

Now we have factored this out into a separate general-purpose tool: https://github.com/hydromatic/sql-logic-test