search

univalence / spark-tools

https://univalence.github.io/spark-tools/
Apache License 2.0
44 stars 8 forks source link

[sparkTest] Two dataframes with no common columns with failOnMissingExpectedCol=false should not be equal #26

Open HarrisonCheng opened 5 years ago

HarrisonCheng commented 5 years ago 
test("with failOnMissingExpectedCol config") {
    val actualDf = dataframe("{c:0}")
    val expectedDf = dataframe("{a:0, b:false}")
    withConfiguration(failOnMissingExpectedCol = false)(
      actualDf.assertEquals(expectedDf)
    )
  }

I think the test should not pass when two dfs have no columns in common, even though both failOnMissingExpectedCol and failOnMissingOriginalCol options are false. We only want it to pass when we find common columns and those are equal.

ahoy-jon commented 5 years ago

Indeed, can you fix it?

On Mon, 28 Oct 2019, 21:29 Harrison Cheng, notifications@github.com wrote:

test("with failOnMissingExpectedCol config") { val actualDf = dataframe("{c:0}") val expectedDf = dataframe("{a:0, b:false}") withConfiguration(failOnMissingExpectedCol = false)( actualDf.assertEquals(expectedDf) ) }

I think the test should not pass when two dfs have no columns in common, even though both failOnMissingExpectedCol and failOnMissingOriginalCol options are false. We only want it to pass when we find common columns and those are equal.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/univalence/spark-tools/issues/26?email_source=notifications&email_token=AAAUKL7DHMMIVXJEHV7K6ADQQ5DS5A5CNFSM4JF7R7Z2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HU374WA, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAUKL32IVQLFHHDYNJPP4DQQ5DS5ANCNFSM4JF7R7ZQ .