Now, the data is saved in DataFrames with the default datatype. The objective is to use a suitable type for each field (for example, changing unix epochs to datetime, the network from string to categorical, etc.).
This would save a lot of space both in RAM and disk space.
Now, the data is saved in DataFrames with the default datatype. The objective is to use a suitable type for each field (for example, changing unix epochs to
datetime, the network fromstringtocategorical, etc.).This would save a lot of space both in RAM and disk space.
Further reading