univalence / spark-tools

https://univalence.github.io/spark-tools/
Apache License 2.0
44 stars 8 forks source link

[spark-test] unvalid field name for my struct name field #25

Closed dylandoamaral closed 5 years ago

dylandoamaral commented 5 years ago

"Activite.code is not a valid field name" when comparing two Dataframes using assertEquals.

val from = csvToDf("from.csv")
val to = csvToDf("to.csv")

from.assertEquals(to)`

Because Key from io.univalence.strings doesn't accept "." (line 312). Maybe a preprocessing step to change Activite.code into Activitecode before comparing schemas in spark-test for every "." and also for every unknow character that are not elligible by the regex expression `^[a-zA-Z][a-zA-Z0-9_]*$`?