search

HEPCompSim / DCSim

Implementations for simulation of distributed systems with clusters, compute nodes, storages and data caches using WRENCH and SimGrid
GNU Lesser General Public License v3.0
4 stars 1 forks source link

Storage-buffer-size "infinity" breaks job monitoring for WLCG platform simulation #55

Closed HerrHorizontal closed 1 year ago

HerrHorizontal commented 1 year ago

When executing a simulation for the WLCG platform with --storage-buffer-size inf the monitor CSV output file containing the simulated jobs' quantities gets only filled with the header.

HerrHorizontal commented 1 year ago

Apparently at least one action fails, which triggers the abort of the simulation.

henricasanova commented 1 year ago

I'll investigate.

henricasanova commented 1 year ago

I am not able to reproduce this error... would you have a particular command-line? I've tried several runs and passing 'inf' (or 'infinity') gives me no error and the same results as passing 100000000000...

HerrHorizontal commented 1 year ago

when I try running, e.g. 

dc-sim --platform data/platform-files/WLCG_disklessTier2_reduced100.xml --workload-configurations data/workload-configs/T1_DE_KIT_workloads.json data/workload-configs/T2_DE_DESY_workloads.json data/workload-configs/Dummy_workloads.json -o test.csv -b inf

on the scenario/wlcg branch, the simulation runs fine. However, the output file test.csv should contain much more jobs. Running with another value than inf for the storage buffer size fills much more jobs.

henricasanova commented 1 year ago

Thanks! I am able to reproduce the issue, and it happens for a buffer size = inf, but also for a buffer size = 1000000000. So it's not something special about inf it's something else. BUT, when upgrading WRENCH to the current master, and fixing the code to compile due to the API change, it all works. So it was a bug, but given the major WRENCH refactoring, which is all about the storage service, it all works.

If you upgrade WRENCH/SimGrid, then merge the DCSim main branch into the scenario/wlcg branch, the bug will magically disappear :)

HerrHorizontal commented 1 year ago

Nice to hear. Is there a specific version of WRENCH and SimGrid I have to checkout? Or are both the main branches fine?

HerrHorizontal commented 1 year ago

I have tried it with the latest commits on SimGrid and WRENCH and can confirm that it solves the issue. Thank you very much!