MohamedBassem / r-cluster

A tool to distributively schedule R scripts on a mesos cluster
7 stars 2 forks source link

Wrong sequential execution on single machine #9

Closed matefh closed 8 years ago

matefh commented 8 years ago

I have been hunting (and suffering from) this bug for a long time. I am finally able to reproduce it consistently. Here are two scenarios where it happens:

Scenario 1

Sys.sleep(30)
text = "this is 0"
print(text)
write(text, file = paste0(text, ".txt"))
Sys.sleep(30)
text = "this is 1"
print(text)
write(text, file = paste0(text, ".txt"))
Sys.sleep(30)
text = "this is 2"
print(text)
write(text, file = paste0(text, ".txt"))

The 2nd scenario may be less important and more unpredictable. Fixing 1 and defining how to use the R console properly will be satisfying and usable enough. Currently I cannot queue more than 2 tasks reliably.

@MohamedBassem @saherneklawy

MohamedBassem commented 8 years ago

I have a theory, I'll check it in a couple of hours

On Wed, Jan 27, 2016, 8:41 AM Mohammad Atef notifications@github.com wrote:

I have been hunting (and suffering from) this bug for a long time. I am finally able to reproduce it consistently. Here are two scenarios where it happens: Scenario 1

  • Open r-console, execute the following task

Sys.sleep(30) text = "this is 0" print(text) write(text, file = paste0(text, ".txt"))

  • Refresh the page, execute another task

Sys.sleep(30) text = "this is 1" print(text) write(text, file = paste0(text, ".txt"))

  • Refresh the page one more time and execute the last task

Sys.sleep(30) text = "this is 2" print(text) write(text, file = paste0(text, ".txt"))

  • On mesos, you will see that the cluster will execute 0 then 2 then
  • But actually it will never execute 1 it will execute 0, then 2 then 2 again! If you look at the working_dir of the task you will not find the file "this is 1.txt"

Scenario 2

  • Open 3 tabs in parallel and paste the code in of each task in a tab.
  • Hit run code on each of them.
  • Mesos will show you as if it is running them 0, 1, then 2. But it will still mess up the execution.

The 2nd scenario may be less important and more unpredictable. Fixing 1 and defining how to use the R console properly will be satisfying and usable enough. Currently I cannot queue more than 2 tasks reliably.

@MohamedBassem https://github.com/MohamedBassem @saher.neklawy

— Reply to this email directly or view it on GitHub https://github.com/MohamedBassem/r-cluster/issues/9.

Best Regards, Mohamed Bassem

MohamedBassem commented 8 years ago

Should be fixed by #10 , Please try it and close this issue if fixed

matefh commented 8 years ago

Great its fixed, thanks @MohamedBassem