timeIndex file is expected to be the same directory as the CSV files. This causes the timeIndex to be read into the RDD, which then fails since the format does not match the rest of the CSV files. To fix this, timeIndex should not be in the same directory as the CSV files or rdd = sc.textFile(path).map should check to not read timeIndex.
sryza
commented
8 years ago
timeIndex file is expected to be the same directory as the CSV files. This causes the timeIndex to be read into the RDD, which then fails since the format does not match the rest of the CSV files. To fix this, timeIndex should not be in the same directory as the CSV files or rdd = sc.textFile(path).map should check to not read timeIndex.