search

con / opfvta-reexecution

Container-based Replication of https://doi.org/10.1038/s41398-022-01812-5
Apache License 2.0
1 stars 1 forks source link

Analysis and meta-article generation (full) fails #40

Closed asmacdo closed 1 year ago

asmacdo commented 1 year ago

On typhon, I cloned a fresh copy of the repo and pulled down all the data from gin

(NOT outputs/*/article.pdf)

Heres the last part of the output:

------------                                                                   
make[2]: Entering directory '/data/paper/source'                                                                  
make[2]: Leaving directory '/data/paper/source'                                                                                                               
make[2]: *** No rule to make target 'figs/diff_pages.tex'.  Stop.                                                                           
------------                                                                   
Running 'make "figs/diff_pages.jpg"'                                           
------------                                                                                                                                                  
make[2]: Entering directory '/data/paper/source'                                                                                            
make[2]: *** No rule to make target 'figs/diff_pages.jpg'.  Stop.              
make[2]: Leaving directory '/data/paper/source'                                
------------                                                                                     
Running 'make "figs/diff_pages.pdf"'                                                                                                        
------------                                                                   
make[2]: Entering directory '/data/paper/source'                                                 
prepare/paperdiff.sh                                                                                                                                          
GENERATING DIFF FOR EXECUTION in ../../../outputs/podman_20230908162849/article.pdf:             
13:23:30: Error: Unable to initialize GTK+, is DISPLAY set properly?           
Error opening ../../../outputs/original/article.pdf: No such file or directory 
Handling non-zero exit code (3) for differing documents.                                                          
GENERATING DIFF FOR EXECUTION in ../../../outputs/podman_20230920004219/article.pdf:                     
13:23:30: Error: Unable to initialize GTK+, is DISPLAY set properly?           
Error opening ../../../outputs/original/article.pdf: No such file or directory                                    
Handling non-zero exit code (3) for differing documents.                                                                                                      
GENERATING DIFF FOR EXECUTION in ../../../outputs/podman_20230920004416/article.pdf:                              
13:23:30: Error: Unable to initialize GTK+, is DISPLAY set properly?           
Error opening ../../../outputs/original/article.pdf: No such file or directory 
Handling non-zero exit code (3) for differing documents.                                                                                                      
GENERATING DIFF FOR EXECUTION in ../../../outputs/podman_20230922123546/article.pdf:                     
13:23:30: Error: Unable to initialize GTK+, is DISPLAY set properly?                                                                        
Error opening ../../../outputs/original/article.pdf: No such file or directory                   
Handling non-zero exit code (3) for differing documents.                                         
GENERATING DIFF FOR EXECUTION in ../../../outputs/singularity_20230905040926/article.pdf:        
13:23:30: Error: Unable to initialize GTK+, is DISPLAY set properly?                             
Error opening ../../../outputs/original/article.pdf: No such file or directory                                    
Handling non-zero exit code (3) for differing documents.                                         
GENERATING DIFF FOR EXECUTION in ../../../outputs/singularity_20230905204435/article.pdf:        
13:23:30: Error: Unable to initialize GTK+, is DISPLAY set properly?                                              
Error opening ../../../outputs/original/article.pdf: No such file or directory                                    
Handling non-zero exit code (3) for differing documents.                                                          
GENERATING DIFF FOR EXECUTION in ../../../outputs/singularity_20230906012936/article.pdf:                         
13:23:30: Error: Unable to initialize GTK+, is DISPLAY set properly?                                              
Error opening ../../../outputs/original/article.pdf: No such file or directory                                    
Handling non-zero exit code (3) for differing documents.                                                          
GENERATING DIFF FOR EXECUTION in ../../../outputs/singularity_20230908122618/article.pdf:                                                   
13:23:30: Error: Unable to initialize GTK+, is DISPLAY set properly?                                                                        
Error opening ../../../outputs/original/article.pdf: No such file or directory
Handling non-zero exit code (3) for differing documents.                                                                                    
cd prepare; python paperdiff_parse.py                                                                                                       
Processing paperdiff log file `paperdiff_podman_20230908162849.log`...                                                                      
Processing paperdiff log file `paperdiff_podman_20230920004219.log`...                                                                      
Processing paperdiff log file `paperdiff_podman_20230920004416.log`...                                                                      
Processing paperdiff log file `paperdiff_podman_20230922123546.log`...                                                                      
Processing paperdiff log file `paperdiff_singularity_20230905040926.log`...                                                                 
Processing paperdiff log file `paperdiff_singularity_20230905204435.log`...
Processing paperdiff log file `paperdiff_singularity_20230906012936.log`...
Processing paperdiff log file `paperdiff_singularity_20230908122618.log`...                              
cd scripts; python diff_pages.py                                               
Traceback (most recent call last):                                             
  File "/data/paper/source/scripts/diff_pages.py", line 43, in <module>
    sns.pointplot(data=df,                                                     
  File "/usr/lib/python3/dist-packages/seaborn/categorical.py", line 2839, in pointplot                  
    plotter = _PointPlotter(x, y, hue, data, order, hue_order,
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/seaborn/categorical.py", line 1603, in __init__                   
    self.establish_colors(color, palette, 1)         
  File "/usr/lib/python3/dist-packages/seaborn/categorical.py", line 707, in establish_colors                                                                 
    lum = min(light_vals) * .6                                                 
          ^^^^^^^^^^^^^^^                                                      
ValueError: min() arg is an empty sequence                                                               
make[2]: Leaving directory '/data/paper/source'                                                          
make[2]: *** [Makefile:55: figs/diff_pages.pdf] Error 1                                                  
Latexmk: If appropriate, the -f option can be used to get latexmk                                        
  to try to force complete processing.              
Latexmk: Errors, so I did not complete making targets                                                    
Collected error summary (may duplicate other messages):                                                  
  pdflatex: Command for 'pdflatex' gave return code 1                                                    
      Refer to 'article.log' and/or above output for details                                             

make[1]: *** [Makefile:78: article] Error 12                                                             
make[1]: Leaving directory '/data/paper/source'                                                          
make: *** [Makefile:106: article] Error 2                                                                
make[1]: *** [Makefile:72: container-article] Error 2                                                    
make[1]: Leaving directory '/home/asmacdo/tmp/opfvta-2023-9-21/opfvta-replication-2023/paper/source'                                                                                                              
make: *** [Makefile:102: container-article] Error 2

Since the build is working on CI, I suspect the difference is in the generated pdfs.

asmacdo@typhon:~/tmp/opfvta-2023-9-21/opfvta-replication-2023/outputs$ ls -lah */article.pdf
lrwxrwxrwx 1 asmacdo asmacdo 136 Sep 21 16:22 original/article.pdf -> ../../.git/annex/objects/f9/Pv/MD5E-s21852357--cda3904485734ab5c089894db1659fa0.pdf/MD5E-s21852357--cda3904485734ab5c089894db1659fa0.pdf
lrwxrwxrwx 1 asmacdo asmacdo 136 Sep 21 16:22 podman_20230906053037/article.pdf -> ../../.git/annex/objects/qW/zV/MD5E-s21529647--17365d13a65885d2fce3d65d1c96c9f7.pdf/MD5E-s21529647--17365d13a65885d2fce3d65d1c96c9f7.pdf
lrwxrwxrwx 1 asmacdo asmacdo 136 Sep 21 16:22 podman_20230908162849/article.pdf -> ../../.git/annex/objects/80/v6/MD5E-s21545132--cc38b0844b952a39cd8d4cbf96605105.pdf/MD5E-s21545132--cc38b0844b952a39cd8d4cbf96605105.pdf
lrwxrwxrwx 1 asmacdo asmacdo 136 Sep 21 16:22 podman_20230920004219/article.pdf -> ../../.git/annex/objects/PJ/0J/MD5E-s21532977--fd1bd9e0b630500be88a52f3880f7a96.pdf/MD5E-s21532977--fd1bd9e0b630500be88a52f3880f7a96.pdf
lrwxrwxrwx 1 asmacdo asmacdo 136 Sep 21 16:22 podman_20230920004416/article.pdf -> ../../.git/annex/objects/gx/Pj/MD5E-s21514941--990e16b8bb32fb21c65c71a73646301d.pdf/MD5E-s21514941--990e16b8bb32fb21c65c71a73646301d.pdf
-rw-r--r-- 1 asmacdo asmacdo 21M Sep 23 02:44 podman_20230922123546/article.pdf
lrwxrwxrwx 1 asmacdo asmacdo 136 Sep 21 16:22 singularity_20230904214036/article.pdf -> ../../.git/annex/objects/k1/6x/MD5E-s21487275--5ab7da033717219cf2b87dde21317777.pdf/MD5E-s21487275--5ab7da033717219cf2b87dde21317777.pdf
lrwxrwxrwx 1 asmacdo asmacdo 136 Sep 21 16:22 singularity_20230905040926/article.pdf -> ../../.git/annex/objects/16/fV/MD5E-s21487288--ad7b66af092f14b60d45b31ba5e09135.pdf/MD5E-s21487288--ad7b66af092f14b60d45b31ba5e09135.pdf
lrwxrwxrwx 1 asmacdo asmacdo 136 Sep 21 16:22 singularity_20230905204435/article.pdf -> ../../.git/annex/objects/wz/g9/MD5E-s21487302--9f76897f47716f6adda8c778917b920d.pdf/MD5E-s21487302--9f76897f47716f6adda8c778917b920d.pdf
lrwxrwxrwx 1 asmacdo asmacdo 136 Sep 21 16:22 singularity_20230906012936/article.pdf -> ../../.git/annex/objects/8v/q0/MD5E-s21551881--c92d2d5f2941e0a5a4eb25fca28a92ec.pdf/MD5E-s21551881--c92d2d5f2941e0a5a4eb25fca28a92ec.pdf
lrwxrwxrwx 1 asmacdo asmacdo 136 Sep 21 16:22 singularity_20230908122618/article.pdf -> ../../.git/annex/objects/K6/vv/MD5E-s21370913--dd4358584c78e366bc7940ed7c0965a8.pdf/MD5E-s21370913--dd4358584c78e366bc7940ed7c0965a8.pdf

There is one article.pdf present: podman_20230922123546/article.pdf all others are only available via git-annex.

@TheChymera do we need to do datalad get outputs/*/article.pdf even when we have regenerated the outputs? I figured it should have been able to calculate based only on the new data, but I may need to revert a change I made to the readme.

TheChymera commented 1 year ago

@asmacdo yes, the error you see is due to missing article PDF data on your local copy.

do we need to do datalad get outputs/*/article.pdf even when we have regenerated the outputs?

Yes, since we now use multiple data points per platform case in order to visualize variability. That's why we keep them in separate datestamped directories as oppsed to just having one PDF file we update.

asmacdo commented 1 year ago

OK with that step it completed.

TheChymera commented 1 year ago

@asmacdo this should prevent the issue above from happening altogether → https://github.com/con/opfvta-replication-2023/commit/f1399d942501f2914f43d2218d636cca9b52a0f9