search

polypheny / Polypheny-DB

A self-adaptive Polystore that provides cost- and workload aware access to heterogeneous data.
https://polypheny.org
Apache License 2.0
122 stars 82 forks source link

Can't query data from a CSV with empty fields #433

Open Anphisa opened 1 year ago

Anphisa commented 1 year ago

Operating System (Polypheny-DB)

Windows

Polypheny-DB Version

0d7256f77c62e8d18959966f3afbda2357b918b2 name:schema-integration

Affected Areas

DB

Browser

Chrome

Client Operating System

No response

Bug Description

When I load a CSV into Polypheny which has empty fields (not all values set), I can't query the respective columns.

After executing the steps from reproduction steps, I get a NullPointerException:

13:35:56.886 ERROR [qtp542743255-162]: Caught exception while executing a query from the console
 org.polypheny.db.webui.Crud$QueryExecutionException: java.lang.RuntimeException: java.lang.NullPointerException
    at org.polypheny.db.webui.Crud.executeSqlSelect(Crud.java:3667) ~[classes/:?]
    at org.polypheny.db.webui.Crud.anySqlQuery(Crud.java:810) ~[classes/:?]
    at org.polypheny.db.webui.crud.LanguageCrud.anyQuery(LanguageCrud.java:119) ~[classes/:?]
    at org.polypheny.db.webui.WebSocket.onMessage(WebSocket.java:119) ~[classes/:?]
    at io.javalin.websocket.WsConnection$onMessage$1.invoke(WsConnection.kt:38) [javalin-4.1.1.jar:4.1.1]
    at io.javalin.websocket.WsConnection$onMessage$1.invoke(WsConnection.kt:38) [javalin-4.1.1.jar:4.1.1]
    at io.javalin.websocket.WsConnection.tryBeforeAndEndpointHandlers(WsConnection.kt:71) [javalin-4.1.1.jar:4.1.1]
    at io.javalin.websocket.WsConnection.onMessage(WsConnection.kt:38) [javalin-4.1.1.jar:4.1.1]
    at jdk.internal.reflect.GeneratedMethodAccessor23.invoke(Unknown Source) ~[?:?]
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
    at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
    at org.eclipse.jetty.websocket.common.events.annotated.CallableMethod.call(CallableMethod.java:70) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.websocket.common.events.annotated.OptionalSessionCallableMethod.call(OptionalSessionCallableMethod.java:68) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.websocket.common.events.JettyAnnotatedEventDriver.onTextMessage(JettyAnnotatedEventDriver.java:301) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.websocket.common.message.SimpleTextMessage.messageComplete(SimpleTextMessage.java:69) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.appendMessage(AbstractEventDriver.java:67) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.websocket.common.events.JettyAnnotatedEventDriver.onTextFrame(JettyAnnotatedEventDriver.java:287) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.incomingFrame(AbstractEventDriver.java:152) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.websocket.common.WebSocketSession.incomingFrame(WebSocketSession.java:326) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.websocket.common.extensions.AbstractExtension.nextIncomingFrame(AbstractExtension.java:148) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.websocket.common.extensions.compress.PerMessageDeflateExtension.nextIncomingFrame(PerMessageDeflateExtension.java:111) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.websocket.common.extensions.compress.CompressExtension.forwardIncoming(CompressExtension.java:169) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.websocket.common.extensions.compress.PerMessageDeflateExtension.incomingFrame(PerMessageDeflateExtension.java:90) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.websocket.common.extensions.ExtensionStack.incomingFrame(ExtensionStack.java:202) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.websocket.common.Parser.notifyFrame(Parser.java:225) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.websocket.common.Parser.parseSingleFrame(Parser.java:259) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:459) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:440) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) [jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) [jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) [jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) [jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) [jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) [jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) [jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) [jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) [jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
    at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) [jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
    at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: java.lang.RuntimeException: java.lang.NullPointerException
    at org.polypheny.db.PolyImplementation.getRows(PolyImplementation.java:234) ~[classes/:?]
    at org.polypheny.db.PolyImplementation.getRows(PolyImplementation.java:192) ~[classes/:?]
    at org.polypheny.db.webui.Crud.executeSqlSelect(Crud.java:3654) ~[classes/:?]
    ... 38 more
Caused by: java.lang.NullPointerException
    at org.polypheny.db.runtime.functions.Functions.cannotConvert(Functions.java:2322) ~[classes/:?]
    at org.polypheny.db.runtime.functions.Functions.toFloat(Functions.java:2530) ~[classes/:?]
    at Baz$1$1.current(Unknown Source) ~[?:?]
    at org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.next(Linq4j.java:683) ~[calcite-linq4j-1.18.0.jar:1.18.0]
    at org.polypheny.db.util.LimitIterator.next(LimitIterator.java:58) ~[classes/:?]
    at org.apache.calcite.avatica.MetaImpl.collect(MetaImpl.java:129) ~[avatica-core-1.17.2-POLYPHENY.jar:1.17.2-POLYPHENY]
    at org.apache.calcite.avatica.MetaImpl.collect(MetaImpl.java:122) ~[avatica-core-1.17.2-POLYPHENY.jar:1.17.2-POLYPHENY]
    at org.polypheny.db.PolyImplementation.getRows(PolyImplementation.java:208) ~[classes/:?]
    at org.polypheny.db.PolyImplementation.getRows(PolyImplementation.java:192) ~[classes/:?]
    at org.polypheny.db.webui.Crud.executeSqlSelect(Crud.java:3654) ~[classes/:?]
    ... 38 more

Reproduction Steps

Example: Use baeume as source csv. baeume.csv

Then query on the console: SELECT hoehe FROM public.baeume

Expected Behavior

Show the column that was queried

Relevant Log Output

No response

Raghavendrapara commented 1 year ago

Let me check this out.

Raghavendrapara commented 1 year ago

The methods in the exception are the symptoms , underlying disease seems to be in the csv loader. Null handling cases might have been missed

Raghavendrapara commented 1 year ago

https://github.com/polypheny/Polypheny-DB/blob/5e3257e3713ccb1fd7db10903160c46136b11109/plugins/csv-adapter/src/main/java/org/polypheny/db/adapter/csv/CsvEnumerator.java#L308

Looks like instead of universally setting all null we need to initialize something else or allow null handling

Raghavendrapara commented 1 year ago

From DBMS perspective null shall be OK, so ig the way injection of null is happening into the database is troubling

Raghavendrapara commented 1 year ago

Filtering does not work on that too, we currently cannot sort based on such fields

Raghavendrapara commented 1 year ago

Interestingly this works SELECT latitude,hoehe FROM public.baeume

Deeppjp116 commented 1 year ago

The error message you provided is a stack trace that indicates an exception occurred in a software application. Here's a breakdown of the information provided:

Based on this information, it seems that the error originates from a web-based user interface (UI) component called "Crud" in the Polypheny-DB system. Polypheny-DB is a database management system, and the error occurred while executing a SQL select query. The cause of the exception appears to be a NullPointerException that occurred within the Functions.toFloat method, which is part of the org.polypheny.db.runtime.functions package.

To further investigate and resolve the issue, it would be necessary to analyze the source code of the Polypheny-DB system, particularly the Crud class and the Functions class. Additionally, examining the specific query being executed and the data involved could provide more insights into the cause of the NullPointerException and potential fixes.

Deeppjp116 commented 1 year ago

try:

Execute the SQL query from the console

result = executeSqlSelect(query)
# Process the result or perform further actions
processResult(result)

except QueryExecutionException as e:

Handle the query execution exception

handleQueryExecutionException(e)

def executeSqlSelect(query): try:

Perform necessary operations to execute the SQL select query

    # ...
    # Return the result
    return queryResult
except RuntimeException as e:
    # Log the error or perform error handling
    logError(e)
    # Raise the exception to the calling code
    raise QueryExecutionException("Error executing SQL select query")

def processResult(result):

Process the query result

# ...
pass

def handleQueryExecutionException(exception):

Log the exception details or perform error handling

logError(exception)
# Handle the exception as required
# ...

def logError(error):

Log the error details to a log file or console

# ...
pass
YassinHemdan commented 6 months ago

can you assign this issue for me @hennlo

mehtasaurav commented 6 months ago

Hi @hennlo , I think you all are facing this bug because in java empty String is by default null. And in your code you are finding the length of null which in case will give NullPointerException. I can help you fixing this.

 case FLOAT:
                    if ( string.length() == 0 ) {
                        return null;
                    }
                    return Float.parseFloat( string );