fsprojects-archive / zzarchive-FSharp.Data.HiveProvider

[ARCHIVED] F# Type Provider for Hadoop Hive
Other
9 stars 11 forks source link

The table of built-in functions supported by the query language is incomplete #9

Open dsyme opened 10 years ago

dsyme commented 10 years ago

This is documenting a feature request - users of this type provider who need this feature are encouraged to implement it and submit the implementation back to this repo.

When looking at this code last week we noticed that the table of built-in functions supported in Hive queries is incomplete and needs to be more thoroughly tested.

The Hive documentation can be found here: https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF

The implementation of the F#-to-Hivetable translation is here: https://github.com/fsprojects/FSharp.Data.HiveProvider/blob/84b6bd97c0c1fe18c001b58b563c22d70837591f/src/HiveTypeProvider/HiveTypeProvider.fs#L1209

One particular concern is that there are many, many Hive functions and operators, most of which have no direct equivalent in F#. This means that "fake" F# functions would be needed along with mappings for these. If done badly, however, this may make it hard to discover these functions (unless samples are written for all of them), and could call into question the whole value of using F# query syntax as source for Hive queries. So thought should be given to make sure the translated functions are very regular and easy to discover - perhaps putting them all under a "Hive" module so that all functions can be found via intellisense completion on "Hive."