arrayfire / arrayfire-python

Python bindings for ArrayFire: A general purpose GPU library.
https://arrayfire.com
BSD 3-Clause "New" or "Revised" License
416 stars 65 forks source link

Feature Request: Data Frames #151

Open ghost opened 7 years ago

ghost commented 7 years ago

Hi, In my use case, I have data sets in parquet/CSV format which I then read into a pandas dataframe for processing. Before starting to use ArrayFire Python, I would like to know if the following operations are at all supported.

  1. Reading a dataframe, with its headers into an ArrayFire equivalent data structure
  2. Reading a CSV, with its headers into an ArrayFire equivalent data structure
  3. Writing an ArrayFire equivalent data structure into a dataframe, with its headers
  4. Filtering as follows:
X_df['Rx_10G_1G'] = X_df.apply(lambda x: findGE(x['NE_OBJECT']), axis=1)  
def findGE (str_ne):    
    if str_ne.find('10GE-') !=-1:
        return 10000
    if str_ne.find('GE-') !=-1:
        return 1000
    else:
        return 1
  1. Filtering as follows: X_df=X_df[X_df['Rx_Octets']> 0.0] x_neg_df=X_df[X_df['RxUtilization_pct']< 0]

  2. Sorting by a time stamp based index: X_df = X_df.sort_index(by='ReportTime')

Many thanks,

pavanky commented 7 years ago

@QuantScientist ArrayFire does not support data frames as of now.

ghost commented 7 years ago

Thanks, How about np array to AF array? The documentation states:

2017-07-20 10_14_24-arrayfire_matrix_computations_chrzeszczyk 1 pdf

But I get an error while trying this syntax: 2017-07-20 10_15_45-array_fire

pavanky commented 7 years ago

@QuantScientist you need to use af.to_array(array_foo)

pavanky commented 7 years ago

@QuantScientist Where did you find that documentation in the screenshot?

ghost commented 7 years ago

From "Matrix computations on the GPU with ArrayFire for Python and C/C++" Where did you take the example you quoted?

pavanky commented 7 years ago

@QuantScientist That's from a few years ago and outdated. Please use the documentation from here: http://arrayfire.org/arrayfire-python

Where did you take the example you quoted?

From memory :) I wrote the function.