In Figure 4.2, there is no column named "date", "type", "latitude" and "longitude".
But why in Figure 4.3, it is showing the columns "date", "type", "latitude" and "longitude"?
And if there is no "date" columns, we would not able to do the groupBy("date") in your instruction below in page 125:
from pyspark.sql.functions import count
display(covid_parquet.groupBy("date").agg(count("*").
alias("TotalCount")).orderBy("TotalCount",
ascending=False).limit(20))
In Figure 4.2, there is no column named "date", "type", "latitude" and "longitude". But why in Figure 4.3, it is showing the columns "date", "type", "latitude" and "longitude"?
And if there is no "date" columns, we would not able to do the groupBy("date") in your instruction below in page 125:
from pyspark.sql.functions import count display(covid_parquet.groupBy("date").agg(count("*"). alias("TotalCount")).orderBy("TotalCount", ascending=False).limit(20))
Could you clarify, please? Thanks.