heavyai / heavydb

HeavyDB (formerly OmniSciDB)
https://heavy.ai
Apache License 2.0
2.92k stars 444 forks source link

SQL Concat #512

Open AndrewShear opened 4 years ago

AndrewShear commented 4 years ago

I have written a few udf's so far but I'm my brian is pretty fried with this issue. The code works outside of omnisci compiling through clang; however, inside I am having issues. This is my only udf that isn't returning int32_t or double so I'm wondering if this may be the issue.

The error message and source code as follows:

Feb 10 11:06:52 $HOSTNAME$ systemd[1]: Started OmniSci database server.
Feb 10 11:06:52 $HOSTNAME$ omnisci_server[7796]: /var/lib/omnisci/udf/udf.cpp:16:1: error: expected unqualified-id
Feb 10 11:06:52 $HOSTNAME$ omnisci_server[7796]: EXTENSION_NOINLINE std::string CONCAT(Args const& ... args ) {
Feb 10 11:06:52 $HOSTNAME$ omnisci_server[7796]: ^
Feb 10 11:06:52 $HOSTNAME$ omnisci_server[7796]: /var/lib/omnisci/udf/udf.cpp:11:35: note: expanded from macro 'EXTENSION_NOINLINE'
Feb 10 11:06:52 $HOSTNAME$ omnisci_server[7796]: #define EXTENSION_NOINLINE extern "C" NEVER_INLINE DEVICE
Feb 10 11:06:52 $HOSTNAME$ omnisci_server[7796]:                                   ^
Feb 10 11:06:52 $HOSTNAME$ omnisci_server[7796]: 1 error generated.
Feb 10 11:06:52 $HOSTNAME$ omnisci_server[7796]: Error while processing /var/lib/omnisci/udf/udf.cpp.
Feb 10 11:06:52 $HOSTNAME$ omnisci_server[7796]: 2020-02-10T11:06:52.637607 F 7796 UDFCompiler.cpp:369 Unable to create AST file for udf compilation

Source

template< typename ... Args >
EXTENSION_NOINLINE std::string CONCAT(Args const& ... args ) {
    std::ostringstream stream;
    using List= int[];
    (void)List{0, ( (void)(stream << args), 0 ) ... };

    return stream.str();
}

Please let me know if there is any known solution for this problem or if anyone has been able to get a concat function to work.

Thanks so much!

randyzwitch commented 4 years ago

Hi @RobAShear -

We're having an on-site company meeting this week, so the response time might be a little slow. But in the meantime, I'll offer that we currently don't have any functions exposed via the SQL interface that return strings. While I believe that functionality is on the near horizon, I suspect that you are running into a technical limitation right now, not an issue specific to you. One of our engineers should be able to confirm shortly.

alexbaden commented 4 years ago

Hi @RobAShear ,

Our extension function framework is strictly typed. We do allow C style casts between types (though not for arrays, currently), but all functions must be created with a type.

We have recently added some experimental varlen return support -- currently implemented for Arrays. Take a look at https://github.com/omnisci/omniscidb/blob/master/QueryEngine/ExtensionFunctionsArray.hpp, you should be able to craft a concat function following that model. Note that we don't yet formally support String arguments, but that is being worked on.