Seeking for repoducing data sharing for drawing

[Kingofolk]

hello! A lot of thanks for your great work! Sorry for bothering you but I wonder if you are pleased to share the data related to drawing the following picture. Thanks a lot!

[Kingofolk]

Hello! Sorry for bothering you again, but I really need the option file to repoduce LWF and it's very kind of you for sharing this file with me. Thanks a lot!

[arthurdouillard]

Sorry for bothering you again, but I really need the option file to repoduce LWF and it's very kind of you for sharing this file with me.

I don't have it as I never really bothered to make LWF work, but you can try to customize the code and takes inspiration from others option file by yourself.

[arthurdouillard]

To plot the figure, unzip the file here for PODNET/Bic/iCaRL and then do the following code (the long array is data extracted from UCIR codebase):

import seaborn
from inclearn.lib import results_utils

l = """[[[76.48       75.04       80.36      ]
  [76.18       74.08       79.9       ]
  [76.16       74.16       79.86      ]]

 [[67.82352941 67.03921569 68.74509804]
  [66.07843137 65.76470588 68.35294118]
  [66.23529412 66.2745098  68.90196078]]

 [[66.78846154 66.48076923 68.25      ]
  [65.88461538 65.61538462 68.17307692]
  [65.90384615 66.25       68.84615385]]

 [[60.60377358 58.9245283  60.94339623]
  [60.13207547 58.30188679 61.03773585]
  [61.         59.50943396 61.54716981]]

 [[59.18518519 58.81481481 60.55555556]
  [59.11111111 58.74074074 60.68518519]
  [59.90740741 59.18518519 61.51851852]]

 [[57.81818182 58.49090909 59.67272727]
  [58.27272727 57.87272727 59.87272727]
  [59.2        59.03636364 61.49090909]]

 [[57.64285714 57.60714286 58.26785714]
  [57.91071429 57.28571429 58.57142857]
  [58.96428571 58.48214286 60.58928571]]

 [[58.05263158 57.42105263 57.66666667]
  [57.43859649 57.57894737 57.70175439]
  [58.38596491 58.64912281 59.75438596]]

 [[57.25862069 56.86206897 57.18965517]
  [56.62068966 56.87931034 57.89655172]
  [57.89655172 57.81034483 59.68965517]]

 [[56.52542373 56.42372881 56.89830508]
  [56.05084746 56.79661017 56.86440678]
  [57.45762712 57.28813559 58.49152542]]

 [[55.96666667 55.5        56.76666667]
  [55.13333333 55.43333333 55.71666667]
  [56.41666667 55.88333333 57.35      ]]

 [[54.60655738 54.70491803 55.6557377 ]
  [54.68852459 54.42622951 55.31147541]
  [55.72131148 55.52459016 56.75409836]]

 [[53.82258065 54.17741935 55.03225806]
  [53.91935484 53.53225806 54.91935484]
  [55.03225806 54.90322581 56.85483871]]

 [[53.3968254  53.52380952 54.73015873]
  [53.20634921 52.49206349 54.41269841]
  [54.3015873  54.46031746 55.80952381]]

 [[52.375      52.421875   54.609375  ]
  [52.328125   51.890625   53.96875   ]
  [53.6875     53.46875    55.53125   ]]

 [[51.23076923 51.21538462 54.07692308]
  [51.61538462 50.90769231 53.81538462]
  [53.10769231 52.66153846 55.09230769]]

 [[50.40909091 50.71212121 51.75757576]
  [49.96969697 49.89393939 51.40909091]
  [50.87878788 51.42424242 52.78787879]]

 [[50.34328358 51.02985075 51.2238806 ]
  [49.8358209  50.26865672 51.20895522]
  [50.80597015 51.67164179 52.3880597 ]]

 [[49.77941176 50.94117647 51.60294118]
  [49.36764706 50.32352941 51.16176471]
  [50.07352941 51.36764706 52.91176471]]

 [[49.04347826 49.98550725 50.94202899]
  [48.55072464 49.53623188 49.91304348]
  [50.14492754 51.         51.75362319]]

 [[48.57142857 50.3        50.2       ]
  [48.2        50.02857143 50.05714286]
  [49.55714286 50.84285714 51.6       ]]

 [[48.77464789 50.15492958 50.47887324]
  [48.43661972 49.23943662 49.98591549]
  [49.87323944 50.5915493  51.21126761]]

 [[47.93055556 50.22222222 49.36111111]
  [47.81944444 49.02777778 48.90277778]
  [49.34722222 50.30555556 50.45833333]]

 [[47.94520548 49.19178082 48.80821918]
  [47.08219178 48.50684932 48.36986301]
  [48.8630137  49.39726027 49.89041096]]

 [[47.78378378 48.33783784 48.68918919]
  [46.97297297 47.87837838 47.93243243]
  [48.72972973 49.18918919 49.48648649]]

 [[47.36       48.08       48.34666667]
  [46.08       47.70666667 47.45333333]
  [48.33333333 49.04       49.01333333]]

 [[47.21052632 48.17105263 48.69736842]
  [45.59210526 47.48684211 47.86842105]
  [47.81578947 48.67105263 49.21052632]]

 [[46.45454545 47.55844156 48.1038961 ]
  [44.88311688 47.05194805 46.66233766]
  [47.48051948 48.33766234 49.01298701]]

 [[46.21794872 46.75641026 47.70512821]
  [45.1025641  46.42307692 46.69230769]
  [47.08974359 48.05128205 48.35897436]]

 [[45.5443038  45.98734177 46.3164557 ]
  [43.98734177 45.30379747 45.4556962 ]
  [46.02531646 46.72151899 47.08860759]]

 [[46.0375     46.3875     45.5375    ]
  [44.675      45.55       44.775     ]
  [45.825      47.3125     46.8       ]]

 [[45.24691358 45.7654321  45.92592593]
  [44.0617284  45.         44.80246914]
  [45.66666667 46.48148148 46.56790123]]

 [[45.03658537 45.09756098 45.14634146]
  [43.74390244 44.85365854 43.97560976]
  [45.65853659 46.42682927 46.03658537]]

 [[45.27710843 45.34939759 45.08433735]
  [43.78313253 44.6626506  44.10843373]
  [45.63855422 46.28915663 45.87951807]]

 [[44.61904762 44.67857143 44.83333333]
  [43.69047619 43.88095238 43.79761905]
  [45.         45.36904762 45.76190476]]

 [[42.         42.10588235 42.55294118]
  [40.56470588 40.77647059 41.37647059]
  [42.08235294 42.58823529 42.97647059]]

 [[42.27906977 42.15116279 42.97674419]
  [41.3372093  41.26744186 41.74418605]
  [42.86046512 43.10465116 43.3255814 ]]

 [[42.71264368 41.68965517 43.06896552]
  [41.50574713 41.20689655 42.        ]
  [42.89655172 43.17241379 43.64367816]]

 [[42.70454545 42.19318182 42.93181818]
  [41.88636364 41.29545455 42.29545455]
  [43.30681818 43.22727273 43.90909091]]

 [[42.87640449 42.26966292 42.4494382 ]
  [42.05617978 41.23595506 41.17977528]
  [43.1011236  43.15730337 42.8988764 ]]

 [[42.82222222 42.11111111 42.25555556]
  [41.66666667 41.48888889 40.86666667]
  [43.18888889 43.4        42.81111111]]

 [[42.30769231 42.10989011 41.84615385]
  [41.10989011 41.         40.6043956 ]
  [42.68131868 43.12087912 42.72527473]]

 [[41.25       40.90217391 41.32608696]
  [39.66304348 40.14130435 40.33695652]
  [41.92391304 42.06521739 41.97826087]]

 [[41.34408602 41.29032258 41.16129032]
  [39.80645161 40.31182796 40.27956989]
  [41.78494624 42.55913978 41.80645161]]

 [[41.37234043 40.74468085 41.42553191]
  [40.13829787 40.12765957 40.18085106]
  [41.80851064 42.29787234 41.67021277]]

 [[41.01052632 40.75789474 40.52631579]
  [39.74736842 40.11578947 39.21052632]
  [41.88421053 42.05263158 40.64210526]]

 [[40.38541667 39.90625    39.86458333]
  [39.21875    39.91666667 38.75      ]
  [41.66666667 41.25       40.66666667]]

 [[40.54639175 40.         39.29896907]
  [39.07216495 39.36082474 38.34020619]
  [41.34020619 41.01030928 40.05154639]]

 [[39.55102041 39.20408163 38.86734694]
  [37.66326531 37.96938776 37.83673469]
  [39.67346939 39.79591837 39.76530612]]

 [[39.50505051 38.97979798 39.35353535]
  [37.8989899  37.97979798 37.83838384]
  [39.86868687 40.02020202 39.84848485]]

 [[39.1        39.07       39.        ]
  [37.42       38.16       37.53      ]
  [39.57       39.79       39.19      ]]]"""

import re
import numpy
import ast

def analyze(s):
    pattern = r'''# Match (mandatory) whitespace between...
                  (?<=\]) # ] and
                  \s+
                  (?= \[) # [, or
                  |
                  (?<=[^\[\]\s]) 
                  \s+
                  (?= [^\[\]\s]) # two non-bracket non-whitespace characters
               '''

    # Replace such whitespace with a comma
    fixed_string = re.sub(pattern, ',', s, flags=re.VERBOSE)

    output_array = numpy.array(ast.literal_eval(fixed_string))
    return output_array

output_array = analyze(l)

results_utils.plot(
    [
        dict(
            path="path to icarl folder,
            label="iCaRL",
            #kwargs={"color": "red"},
        ),
        dict(
            path="path to bic folder",
            label="BiC",
            #kwargs={"color": "blue"},
        ),
        dict(
            runs_accs=[
                output_array[..., 1, 0].tolist(), output_array[..., 1, 1].tolist(), output_array[..., 1, 2].tolist()
            ],
            label="UCIR NME",
            #kwargs={"color": "cyan"}
        ),
        dict(
            runs_accs=[
                output_array[..., 0, 0].tolist(), output_array[..., 0, 1].tolist(), output_array[..., 0, 2].tolist()
            ],
            label="UCIR CNN",
            #kwargs={"color": "purple"}
        ),
        dict(
            path="path to podnet cnn folder",
            label="PODNet (CNN)",
        ),
        dict(
            path="path to podnet nme folder",
            label="PODNet (NME)",
        ),
    ],

    increment=1,
    initial_increment=50,
    x_ticks=5,
    total=100,
    max_acc=80,
    min_acc=30,
    figsize=(6, 5),
    metric="avg_inc",
    title="CIFAR100, 50 steps of 1 increment",
    path_to_save="/path/figure.pdf"
);

[Kingofolk]

Thank you very much! It really helps a lot!! best wishes!!

[arthurdouillard]

I don't have the raw data for increment 2, but I guess you could infer each datapoint from the figure? Sorry I cannot help more

[Kingofolk]

Got it! thanks for your kindness again!

[arthurdouillard]

Oops, I forgot to send you the link to the zip file containing podnet/bic/icarl, here it is: https://drive.google.com/file/d/1B0VBomychhoP43Fnyc5Ev-lzYNx6T3Vc/view?usp=sharing