Chapter 4: Pages 124 and 125

[tanthiamhuat]

In Figure 4.2, there is no column named "date", "type", "latitude" and "longitude". But why in Figure 4.3, it is showing the columns "date", "type", "latitude" and "longitude"?

And if there is no "date" columns, we would not able to do the groupBy("date") in your instruction below in page 125:

from pyspark.sql.functions import count display(covid_parquet.groupBy("date").agg(count("*"). alias("TotalCount")).orderBy("TotalCount", ascending=False).limit(20))

Could you clarify, please? Thanks.

[DataSpacon]

I used "confirmed_date"

from pyspark.sql.functions import count display(covid_parquet.groupBy("confirmed_date").agg(count("*").alias("TotalCount")).orderBy("TotalCount", ascending=False).limit(20))