IntelPython / sdc

Numba extension for compiling Pandas data frames, Intel® Scalable Dataframe Compiler
https://intelpython.github.io/sdc-doc/
BSD 2-Clause "Simplified" License
646 stars 62 forks source link

Implement converters parameter in read_csv() #942

Closed akharche closed 2 years ago

akharche commented 3 years ago

Implement lambda converters in read_csv()

def category_converter(x):
      if x is '':
          return np.int32(0)
      else:
          return np.int32(int(x, 16))

names = ["{0}".format(i) for i in range(40)]
converter = {name: category_converter for name in names}

df = pd.read_csv(dataset_path, delimiter='\t', names=names, converters=converter)