search

daphne-eu / daphne

DAPHNE: An Open and Extensible System Infrastructure for Integrated Data Analysis Pipelines
Apache License 2.0
67 stars 62 forks source link

HDFS tests #837

Open philipportner opened 1 month ago

philipportner commented 1 month ago

Recently, #767 has been merged into main. With that we should add some tests that check the code from the PR. As HDFS is an optional feature it would be fine if these tests only run if one wants to opt-in. This could be done similarly to how the CUDA kernels are only tested when USE_CUDA is defined, but other ways to make it optional are also fine.

corepointer commented 1 month ago

I placed a suggestion to run tests in a single node cluster (that could ideally be set up and torn down automatically via some script) is in the comments of #767 after merging.