search

MrPowers / bebe

Filling in the Spark function gaps across APIs
50 stars 5 forks source link

added copy of logic for bebe_regexp_extract_all to versions previous to 3.1.0 #23

Open alfonsorr opened 3 years ago

alfonsorr commented 3 years ago

Not yet finished (cross spark version buid not inserted) But it allows to use the bebe function to (at least tested with 3.0.1 local) versions previous to 3.1.0.

The source code contains a folder to insert code that is not present in previous versions, and sbt adds it in compilations according the spark version.

The code is extracted from the spark repository.

MrPowers commented 3 years ago

@alfonsorr - I like the idea of backporting functionality to earlier versions of Spark. Doesn't seem like this'll increase the maintenance burden of the project too much.

@nvander1 @yaooqinn - this code backports bebe_regexp_extract_all to work on Spark versions earlier than 3.1.1. Can you please take a look and let us know what you think?

alfonsorr commented 3 years ago

This can be done when a first version of all functions are included, to focus in releasing something first.