Open jershi425 opened 3 years ago
you can do this with a lambadop in NVTabular - For example to create a list column from strings like "drama|comedy" then categorical encode them in NVT would be something like:
genres = nvt.ColumnGroup(["genres"]) >> (lambda col: col.str.split("|")) >> nvt.ops.Categorify()
workflow = nvt.Workflow(genres)
workflow.fit_transform(dataset)
Is your feature request related to a problem? Please describe. Currently, for multi-hot features, we have to first convert them into a list format and then process it with NVT. For example, the raw format of "genres" in the movielens data is like "drama|comedy". I have to first convert it to ["drama", "comedy"] using cudf or pandas and then applying label encoding using NVT. Therefore, if the users want to do a label encoding for multi-hot feature, they have to first process and save it with cudf or pandas and then read again with NVT to do the encoding.
Describe the solution you'd like It would be good if NVT can treat both list and strings as multi-hot input. Say if the input is string like "drama|comedy" or "drama,comedy", NVT can automatically treat them as multi-hot input.