anagainaru / iSBatch

Code to compute the optimal request time for applications submitted for execution using HPC batch schedulers
Other
0 stars 1 forks source link

Possibility to specify the min difference between two requests #94

Closed anagainaru closed 4 years ago

anagainaru commented 4 years ago

For example, for a min increment of 10 minutes a sequence of (50minutes, 55minutes) will not be allowed.

Right now we can get sequences like:

jovyan@~/work/repos/SchedulerSimulator/github/iSBatch/examples (algo *)$ python get_sequence.py logs/CT_eye_segmentation.log
Request sequence: [(22678.0, 1), (180.0, 1), (1377.0, 1), (219.0, 1), (222.0, 1), (134.0, 1), (990.0, 1), (425.0, 1), (168.0, 1), (741.0, 1), (181.0, 1), (992.0, 1), (345.0, 1), (52.0, 1), (86.0, 1), (335.0, 1), (83.0, 1), (549.0, 1), (23.0, 1), (608.0, 1), (221.0, 1), (773.0, 1), (389.0, 1), (127.0, 1), (111.0, 1), (118.0, 1), (379.0, 1), (316.0, 1), (914.0, 1), (495.0, 1), (30.0, 1), (271.0, 1), (168.0, 1), (312.0, 1), (60.0, 1), (153.0, 1), (150.0, 1), (117.0, 1), (38.0, 1), (221.0, 1), (505.0, 1), (79.0, 1), (34.0, 1), (86.0, 1), (57.0, 1), (6.0, 1), (451.0, 1), (1252.0, 1), (598.0, 1), (113.0, 1), (206.0, 1), (515.0, 1), (69.0, 1), (170.0, 1), (330.0, 1), (185.0, 1), (367.0, 1), (103.0, 1), (128.0, 1), (125.0, 1), (570.0, 1), (49.0, 1), (561.0, 1), (68.0, 1), (312.0, 1), (804.0, 1), (19.0, 1), (31.0, 1), (422.0, 1), (425.0, 1), (5.0, 1), (45.0, 1), (52.0, 1), (62.0, 1), (38.0, 1), (120.0, 1), (544.0, 1), (64.0, 1), (287.0, 1), (317.0, 1), (118.0, 1), (4.0, 1), (54.0, 1), (70.0, 1), (193.0, 1), (327.0, 1), (128.0, 1), (252.0, 1), (58.0, 1), (147.0, 1), (189.0, 1), (235.0, 1), (9.0, 1), (517.0, 1), (4.0, 1), (502.0, 1), (107.0, 1), (70.0, 1), (110.0, 1), (86.0, 1), (91.0, 1), (28.0, 1), (196.0, 1), (343.0, 1), (97.0, 1), (10.0, 1), (432.0, 1), (94.0, 1), (99.0, 1), (164.0, 1), (861.0, 1), (42.0, 1), (504.0, 1), (88.0, 1), (487.0, 1), (14.0, 1), (118.0, 1), (16.0, 1), (12.0, 1), (60.0, 1), (259.0, 1), (21.0, 1), (12.0, 1), (320.0, 1), (65.0, 1), (198.0, 1), (66.0, 1), (51.0, 1), (198.0, 1), (29.0, 1), (77.0, 1), (154.0, 1), (414.0, 1), (252.0, 1), (90.0, 1), (516.0, 1), (11.0, 1), (131.0, 1), (66.0, 1), (125.0, 1), (47.0, 1), (95.0, 1), (54.0, 1), (61.0, 1), (49.0, 1), (49.0, 1), (129.0, 1), (197.0, 1), (54.0, 1), (20.0, 1), (45.0, 1), (157.0, 1), (318.0, 1), (237.0, 1), (429.0, 1), (127.0, 1), (76.0, 1), (58.0, 1), (233.0, 1), (92.0, 1), (52.0, 1), (121.0, 1), (130.0, 1), (56.0, 1), (31.0, 1), (286.0, 1), (167.0, 1), (499.0, 1), (48.0, 1), (226.0, 1), (138.0, 1), (98.0, 1), (85.0, 1), (37.0, 1), (295.0, 1), (37.0, 1), (73.0, 1), (54.0, 1), (61.0, 1), (17.0, 1), (348.0, 1), (276.0, 1), (114.0, 1), (90.0, 1), (107.0, 1), (146.0, 1), (572.0, 1), (209.0, 1), (338.0, 1), (362.0, 1), (83.0, 1), (10.0, 1), (438.0, 1), (99.0, 1), (9.0, 1), (68.0, 1), (124.0, 1), (31.0, 1), (190.0, 1), (216.0, 1), (155.0, 1), (196.0, 1), (55.0, 1), (104.0, 1), (21.0, 1), (130.0, 1), (27.0, 1), (166.0, 1), (505.0, 1), (15.0, 1), (178.0, 1), (83.0, 1), (321.0, 1), (9.0, 1), (56.0, 1), (298.0, 1), (386.0, 1), (497.0, 1), (427.0, 1), (283.0, 1), (90.0, 1), (156.0, 1), (617.0, 1), (249.0, 1), (295.0, 1), (488.0, 1), (211.0, 1), (165.0, 1), (222.0, 1), (221.0, 1), (97.0, 1), (59.0, 1), (353.0, 1), (344.0, 1), (11.0, 1), (415.0, 1), (765.0, 1), (127.0, 1), (111.0, 1), (48.0, 1), (330.0, 1), (101.0, 1), (5.0, 1), (48.0, 1), (246.0, 1), (236.0, 1), (167.0, 1), (184.0, 1), (516.0, 1), (167.0, 1), (600.0, 1), (617.0, 1), (405.0, 1), (597.0, 1), (34.0, 1), (9.0, 1), (165.0, 1), (482.0, 1), (15.0, 1), (50.0, 1), (283.0, 1), (30.0, 1), (528.0, 1), (432.0, 1), (26.0, 1), (23.0, 1), (52.0, 1), (120.0, 1), (77.0, 1), (106.0, 1), (24.0, 1), (370.0, 1), (17.0, 1), (1159.0, 1), (475.0, 1), (217.0, 1), (85.0, 1), (1278.0, 1), (635.0, 1), (517.0, 1), (312.0, 1), (22.0, 1), (311.0, 1), (2296.0, 1), (1029.0, 1), (1095.0, 1), (426.0, 1), (365.0, 1), (4979.0, 1), (117.0, 1), (2200.0, 1), (699.0, 0)]
Max request 99019.0
Request sequence: [(85813.0, 0), (99019.0, 0)]
Max request 99019.0

And it is even worse when going through interpolation when checkpoint is allowed

jovyan@~/work/repos/SchedulerSimulator/github/iSBatch/examples (algo *)$ python get_sequence.py logs/SLANT_walltime.log 
Request sequence: [(5620.0, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.9559118236475115, 1), (7.9559118236475115, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 1), (7.955911823648421, 1), (7.955911823646602, 0)]
Max request 9590.0
Request sequence: [(9414.96993987976, 0), (9590.0, 0)]
Max request 9590.0

The solution is to filter our of j-s for which makespan - original_makespan < limit