confluentinc / ksql

The database purpose-built for stream processing applications.
https://ksqldb.io
Other
128 stars 1.04k forks source link

ksqlDB should support n-way joins with multiple sources in the equality if the end result is a single column. #8955

Open abraham-leal opened 2 years ago

abraham-leal commented 2 years ago

Is your feature request related to a problem? Please describe. A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

I have a stream (A) that I'd like to join to two tables (B, and C). However, the join on C is dependent on a column from A and a column from B. Since ksqlDB doesn't support multi-column joins, I've created a composite key in a single field in C to be able to join on to it. The following is the query:

SELECT * FROM A
INNER JOIN B ON B.CODE = CAST(A.STATUS AS INTEGER)
INNER JOIN C ON C.CODE = CAST(CONCAT(A.USERID, CAST(B.CODE AS STRING)) AS INTEGER)
EMIT CHANGES;

The output complains of multiple sources on the equality: Each side of the join comparision must contain references from exactly one source.

Describe the solution you'd like A clear and concise description of what you want to happen.

Ideally, since N-way joins are comparable to serially joining (https://docs.ksqldb.io/en/latest/developer-guide/joins/join-streams-and-tables/#n-way-joins), ksqlDB would treat my CONCAT as a single column in my stream built out from A and B.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

This can be resolved by doing the join serially with multiple steps, but that's what n-way joins aim to avoid.

Additional context Add any other context or screenshots about the feature request here.

suhas-satish commented 2 years ago

Can fix after logical query planner work in progress at the moment

lukaszx0 commented 1 year ago

has there been any progress made on addressing this?

mjsax commented 1 year ago

Unfortunately note.