search

intel-analytics / analytics-zoo

Distributed Tensorflow, Keras and PyTorch on Apache Spark/Flink & Ray
https://analytics-zoo.readthedocs.io/
Apache License 2.0
16 stars 3 forks source link

[BIGDL-2.0] enable MemoryType in FeatureSet #59

Closed yangw1234 closed 2 years ago

yangw1234 commented 2 years ago

Code commented out here: https://github.com/intel-analytics/analytics-zoo/blob/bigdl-2.0/scala/dllib/src/main/scala/com/intel/analytics/bigdl/dllib/feature/FeatureSet.scala#L652-L691

Tests commented out here: https://github.com/intel-analytics/analytics-zoo/blob/bigdl-2.0/python/orca/test/bigdl/orca/learn/spark/test_estimator_keras_for_spark.py#L594-L633

yangw1234 commented 2 years ago

@dding3 could you help look at this? I found you commented out the MemoryTpye related code.

I think we can add at least DiskFeatureSet back.