Strategy of identifying Assertion Roulette

[Talismanic]

1. First we will try to identify all the functions and its definition from the script using AST module of python. We will hold this in a 2d array as name value pair. Name is function name and value is function definition.

2. Then we will pass each name value pair to a function which will determine if there is any assertion statement in this function.

3. Next we will examine each function for multi assert where debug messages for identifying each case is not present.

4. We will store the function name and occurance of those in another array

5. Finally we will decide whether a script has this antipattern from the derived output of step 3 & 4 and will give output which are the functions are affected by this issue .

How does this sound?

[akondrahman]

Sounds good. I am guessing you are done identifying all categories for Ansible in #1 ?

[Talismanic]

Sounds good. I am guessing you are done identifying all categories for Ansible in #1 ?

Still not Bhaiya. Manually code checking is little bit of boring. So I think I should start working on coding parallely.

[akondrahman]

OK. Whatever strategy makes you comfortable, I am fine with it. Do you have a timeline on when you will be done with manual code checking for Ansible test scripts?

[Talismanic]

Till now I have been able to identify occurences of multiple Assert statments in a function. But I am afraid I might have overfitted the code. Using AST module is little bit tricky as it parses the code in the line of compilation. I have committed so far identifier function in Git.

An alternative approach could have been examining each line of a function to find which contains the Assert statements. But that seems a little bit naive to me. Ironically, that would be more efficient in identification.

Please let me know if you have any thought on this.

[akondrahman]

You can implement both approaches. We will use an oracle dataset later to see which one gives the most accurate results. Whatever you have done so far is OK for the time being and you can also more than welcome to implement the alternative approach.

I am guessing you will knock out implementation of all categories one by one? If so, do you have a timeline when you will be done?

[Talismanic]

I am estimating 2 weeks per anti pattern. In that case, still 10-12 weeks to cover all.

[akondrahman]

After tool building we need to do empirical analysis and submit bug reports .... this process will require 1 month, at least. Then writing will take 1 month. Given that FSE usually has a March first week deadline, I think we can pull this off at this rate.

Here is the proposed schedule:

Aug/Sep/Oct: Tool building and validation 
Nov: Empirical analysis, bug report submission 
Dec: Writing  

[akondrahman]

@Talismanic

How is your wife? And how is TAMI coming along? I haven't heard from you in a while.

[Talismanic]

@Talismanic

How is your wife? And how is TAMI coming along? I haven't heard from you in a while.

Bhaiya, We had passed some tough time as family. My son and wife both were infected by Typhoid. My wife is ok now and her anti biotic dozes are over. My son was released from hospital last week and his anti-biotics will be over tomorrow. That is why I had not actually worked for last 2 weeks. I have started to work again from today. Hopefully, I will not miss any timelines Inshallah.

[akondrahman]

@Talismanic

Glad to hear that your family is doing well. I knew that your wife had typhoid but didn't suspect the same about your son.

[Talismanic]

Assertion Roulette has been implemented for YAML and Python scripts.