softsys4ai / unicorn

A Framework for Reasoning about System Performance using Causal AI
MIT License
41 stars 7 forks source link

Questions regarding offline mode and entropy-based orientation #30

Open ztz1989 opened 1 year ago

ztz1989 commented 1 year ago

Dear experts of Unicorn,

Thanks a lot for open-sourcing this excellent research. I have read your EuroSys paper and learned a lot! I just have two questions regarding the codebase due to my lack of knowledge:

  1. If I understand correctly, the offline mode of Unicorn debugging experiments cannot be reproduced for all the test scenarios (i.e., hardware+software). In particular, I can only find the measurement.json file under the Single Objective, Image folders of TX2 and Xavier. In the other debug directories, there is only the data.csv file. Could you give me some hints to reproduce the offline experiments for the other scenarios?
  2. I am really interested in the entropy-based edge orientation approach, detailed in Sec. 4 of the EuroSys paper (i.e., resolving partially directed edges). However, I failed to find the corresponding algorithm in the codebase. For instance, in the resolve_edges method of causal_model.py, the edges seem to be oriented based on fixed rules without involving entropies.
        # replace trail and undirected edges with single edges using entropic policy
        for i in range (len(PAG)):
            if trail_edge in PAG[i]:
                PAG[i]=PAG[i].replace(trail_edge, directed_edge)
            elif undirected_edge in PAG[i]:
                    PAG[i]=PAG[i].replace(undirected_edge, directed_edge)
            else:
                continue
    
        for edge in PAG:
            cur = edge.split(" ")
            if cur[1]==directed_edge:
                node_one = self.colmap[int(cur[0].replace("X", ""))-1]               
                node_two = self.colmap[int(cur[2].replace("X", ""))-1]
                options[node_one][directed_edge].append(node_two)
            elif cur[1]==bi_edge:
                node_one = self.colmap[int(cur[0].replace("X", ""))-1]
                node_two = self.colmap[int(cur[2].replace("X", ""))-1]
    
                options[node_one][bi_edge].append(node_two)
            else: print ("[ERROR]: unexpected edges")

I did find a function that computed the entropy for the EnCore method in the debugging_based.py. But maybe it is not the same. Could you point me to the right location of the entropy-based method that orients the undetermined edges of FCI? Thanks in advance!

Best regards, Tianzhu

iqbal128855 commented 1 year ago

@ztz1989 Thanks for your query. Please allow us sometime to look into this.

pooyanjamshidi commented 1 year ago

Hi @iqbal128855 is there any update about the questions @ztz1989 asked earlier? Please clarify the questions and if needed please refactor the code structure to make it clear for others to be able to replicate more easily.

mxl1334 commented 6 months ago

Hi @iqbal128855 @pooyanjamshidi Thank you so much for sharing your findings in the EuroSys paper and the code to reproduce the results. I have the same question. Please let me follow up to see if you have any update? @ztz1989 , also thank you for raising this question!

ztz1989 commented 2 months ago

Hello @iqbal128855 and @pooyanjamshidi, did you have time to look into my previous questions? Thanks a lot in advance.