Hi. I have created a Google Datalab on a dataproc cluster. I've managed to link the notebook to my dataset by :
!rm german_credit_data_biased_training.csv!wget https://raw.githubusercontent.com/emartensibm/german-credit/master/german_credit_data_biased_training.csvfrom pyspark.sql import SparkSessionimport pandas as pdimport jsonpd_data = pd.read_csv("german_credit_data_biased_training.csv", sep=",", header=0)
Hi. I have created a Google Datalab on a dataproc cluster. I've managed to link the notebook to my dataset by :
!rm german_credit_data_biased_training.csv
!wget https://raw.githubusercontent.com/emartensibm/german-credit/master/german_credit_data_biased_training.csv
from pyspark.sql import SparkSession
import pandas as pd
import json
pd_data = pd.read_csv("german_credit_data_biased_training.csv", sep=",", header=0)
The above works fine, but when I run this one:
df_data = sc.read.csv(path='german_credit_data_biased_training.csv', sep=',', header=True, inferSchema=True)
I get the error `AnalysisExceptionTraceback (most recent call last)