Parameters exercised by a test method

[Dovely-ll]

For parameters logged in *-output.txt files with "LOG.WARN", I noticed that they will be overwritten by each test method in the same test class while running the script to identify those parameters, and all test methods have an identical parameter list in the same test class.

Is that how this project is designed to behave? What if a single test method only exercises a subset of those parameters?

[tianyin]

@Dovely-ll Could you be explicit about what scripts you are using and how to reproduce the issue you are talking about?

It's hard to understand the problem based on the very vague description. Please give detailed and reproducible info so folks like @Alex-Lian can help.

[Dovely-ll]

I used ./identify_param.sh hadoop-common and watched for ~/Ctest/hadoop/hadoop-common-project/hadoop-common/target/surefire-reports/*-output.txt.

Basically, while the identify_param.sh is running different test methods in the same test class, the *-output.txt will be overwritten by every test method with the same parameter content but different log times (where * denotes the test class name).

[xylian86]

Hello @Dovely-ll Thanks for the question!

the *-output.txt will be overwritten by every test method with the same parameter content

You are right. For each test method, we will run it individually (code). To address this overwriting issue as you mentioned, we save the *-output.txt file to another location with the name {method + "-log.txt"} after every test method execution. (code). Therefore, each test method retains its own distinct output file which we will parse subsequently.

and all test methods have an identical parameter list in the same test class.

Based on the design just mentioned, this statement is incorrect.

I hope this answers your questions. Feel free to ask if you still have questions on it!

[Dovely-ll]

Thanks for the reply. Yeah, that cleared my head a lot.

So for Hadoop Common, in most cases test methods in the same class share the same parameter lists, it might be related to the way those tests are written or just in coincidence, right? The mechanism of Ctest is still based on a per-method basis.

[tianyin]

@Dovely-ll When you say

for Hadoop Common, in most cases test methods in the same class share the same parameter lists

Can you give concrete examples?

General advice: please always give concrete data points so that others can understand you precisely and be able to help.

[Dovely-ll]

For example below, test methods in the same class in file param_unset_getter_map.json ( or other param*map.json files) all share the same parameters. I used the phrase "in most cases" because I'm not 100% sure about it, but as far as I observed, it holds true for all methods I have looked for now in those files.

"org.apache.hadoop.crypto.TestCryptoStreams#testHasCapability": [ "hadoop.security.crypto.codec.classes.aes.ctr.nopadding", "hadoop.security.crypto.jce.provider", "hadoop.security.crypto.cipher.suite", "hadoop.security.java.secure.random.algorithm" ], "org.apache.hadoop.crypto.TestCryptoStreams#testSyncable": [ "hadoop.security.crypto.codec.classes.aes.ctr.nopadding", "hadoop.security.crypto.jce.provider", "hadoop.security.crypto.cipher.suite", "hadoop.security.java.secure.random.algorithm" ], "org.apache.hadoop.crypto.TestCryptoStreams#testPositionedReadWithByteBuffer": [ "hadoop.security.crypto.codec.classes.aes.ctr.nopadding", "hadoop.security.crypto.jce.provider", "hadoop.security.crypto.cipher.suite", "hadoop.security.java.secure.random.algorithm" ], "org.apache.hadoop.crypto.TestCryptoStreams#testWrite": [ "hadoop.security.crypto.codec.classes.aes.ctr.nopadding", "hadoop.security.crypto.jce.provider", "hadoop.security.crypto.cipher.suite", "hadoop.security.java.secure.random.algorithm" ], "org.apache.hadoop.crypto.TestCryptoStreams#testCryptoIV": [ "hadoop.security.crypto.codec.classes.aes.ctr.nopadding", "hadoop.security.crypto.jce.provider", "hadoop.security.crypto.cipher.suite", "hadoop.security.java.secure.random.algorithm" ], "org.apache.hadoop.crypto.TestCryptoStreams#testRead": [ "hadoop.security.crypto.codec.classes.aes.ctr.nopadding", "hadoop.security.crypto.jce.provider", "hadoop.security.crypto.cipher.suite", "hadoop.security.java.secure.random.algorithm" ], "org.apache.hadoop.crypto.TestCryptoStreams#testSeek": [ "hadoop.security.crypto.codec.classes.aes.ctr.nopadding", "hadoop.security.crypto.jce.provider", "hadoop.security.crypto.cipher.suite", "hadoop.security.java.secure.random.algorithm" ]

[xylian86]

it might be related to the way those tests are written or just in coincidence, right?

@Dovely-ll It is related to the way those tests are written. They include methods marked with @Before and @BeforeClass annotations. These methods read shared configuration parameters, leading to test methods within the same class sharing the same parameters.

[xylian86]

I am going to close this issue.

@Dovely-ll Feel free to reopen it if you have any further questions.