[Bug] command: unit_testing_run_scheduler run not one time

Kristinita commented 7 years ago

1. Summary

If I start Sublime Text, command: unit_testing_run_scheduler run not one time for me. The more packages I have installed, the more times I see command: unit_testing_run_scheduler in console.

2. Settings

For example, I have 400+ packages for Sublime Text.

3. Steps to reproduce

I reproduce the problem in a version of Sublime Text without plugins and user settings.

I install UnitTesting → I restart Sublime Text → I open Sublime Text console.

4. Expected behavior

command: unit_testing_run_scheduler

5. Actual behavior

command: unit_testing_run_scheduler
command: unit_testing_run_scheduler
command: unit_testing_run_scheduler
command: unit_testing_run_scheduler
command: unit_testing_run_scheduler
command: unit_testing_run_scheduler
command: unit_testing_run_scheduler
command: unit_testing_run_scheduler
command: unit_testing_run_scheduler
command: unit_testing_run_scheduler
command: unit_testing_run_scheduler
command: unit_testing_run_scheduler
command: unit_testing_run_scheduler
command: unit_testing_run_scheduler
command: unit_testing_run_scheduler
command: unit_testing_run_scheduler
command: unit_testing_run_scheduler
command: unit_testing_run_scheduler
command: unit_testing_run_scheduler

If I disable some my packages, I get command: unit_testing_run_scheduler fewer time, but still no one time. If I disable still some packages, I get command: unit_testing_run_scheduler fewer time and so on.

6. Environment

Operating system and version: Windows 10 Enterprise LTSB 64-bit EN Sublime Text: Build 3126 Package: The latest stable version of UnitTesting for Sublime Text 3

Thanks.

randy3k commented 7 years ago

It i a kind of expected. Sublime does not accept running any TextCommand until the API has loaded. And unittesting is trying to fire the scheduler command until it has been executed. I cannot think of a better way to fire the command right now.

Let's keep this issue until there is a better solution.

gerardroche commented 6 years ago

The scheduler is really only for CI isn't it? I'd prefer if this didn't fire at all on a dev machine.

randy3k commented 6 years ago

One can also trigger the tests on command line locally by using the run_tests scripts.

cd your_package
python path/to/run_tests.py PACKAGE

gerardroche commented 6 years ago

I'm assuming the reason scheduler exists at all is because subl --command "unit_testing_run_scheduler" doesn't work when starting ST code i.e. the command never runs.

The run_tests.py script has this:

if sublime_is_running:
    subprocess.Popen(["subl", "-b", "--command", "unit_testing_run_scheduler"])
else:
    subprocess.Popen(["subl"])

The command runs fine when ST already running, but won't if starting cold. I'd say that's an issue with ST, it should really wait and run the command when the api is ready.

Nonetheless, how about instead of polling with a scheduler on ST start, start ST, and wait until it's started and then run the command. In pseudo code:

start ST
while not started:
    wait...
# now ST is already, run the command...
subprocess.Popen(["subl", "-b", "--command", "unit_testing_run_scheduler"])

Would that work or am I missing something?

randy3k commented 6 years ago

Delaying the call could be a solution, but we still need to know when the api will be ready.

A solution will be again polling from run_tests.py (instead of running from Sublime) with something like

 subprocess.Popen(["subl", "-b", "--command", "unit_testing_api_ready"])

where UnitTestingApiReady writes to a local file and run_tests.py checks the existence of that file to determine if the api is ready.

randy3k commented 6 years ago

implemented unit_testing_api_ready at dev branch, but not sure why it doesn't work on linux but only macOS. https://travis-ci.org/randy3k/UnitTesting/builds/285907560

gerardroche commented 6 years ago

I'll test it on Linux and figure out what's going on. Looks good.

randy3k commented 6 years ago

It's running fine on my local virtual machine. Not sure what's wrong on travis.

gerardroche commented 6 years ago

Yeah, it's running fine for me locally too, but I haven't been able to figure out why it doesn't work on Travis.

gerardroche commented 6 years ago

I still can't figure out why it won't run on Travis. After some debugging I've noticed several things:

During the bootstrap process, dependencies are installed, then ST is installed. ST is supposed to killed after that, but it looks like it's not being killed, or at least sometimes it's still running when the run_tests.py script runs. This is not issue in itself, just something I noticed.
The --command option to subl never to seems to run the command i.e. subprocess.Popen(["subl", "-b", "--command", "unit_testing_run_scheduler"]) in run_tests.py script. The --command option seems to be irrelevant. The tests still run because the plugin_loaded hook runs the scheduler. The scheduler starts a thread and tries to run the the tests command, which if course will always run the first time it's called, because the run_tests.py script has already created a schedule before calling subprocess.Popen(["subl", "-b", "--command", "unit_testing_run_scheduler"]) . I still have no idea why the --command won't execute the command if ST is still running. Commands generally don't run properly if ST isn't already started (re: https://github.com/SublimeTextIssues/Core/issues/1457), but if ST is running it should run the command.
```
class UnitTestingRunSchedulerCommand(sublime_plugin.ApplicationCommand):
    ready = False

    def run(self):
        UnitTestingRunSchedulerCommand.ready = True
        scheduler = Scheduler()
        scheduler.run()

def try_running_scheduler():
    while not UnitTestingRunSchedulerCommand.ready:
        sublime.set_timeout(
            lambda: sublime.run_command("unit_testing_run_scheduler"), 1)
        if UnitTestingRunSchedulerCommand.ready:
            break
        else:
            time.sleep(0.5)

def run_scheduler():
    th = threading.Thread(target=try_running_scheduler)
    th.start()
```

When the above is run from the plugin_loaded it will run the tests first time if there is a schedule.

def plugin_loaded():
    # run the schedule
    run_scheduler()

Try it yourself by commenting out the run command code in the run_tests.py:

# launch scheduler
tasks = subprocess.check_output(['ps', 'xw']).decode('utf8')
sublime_is_running = "Sublime" in tasks or "sublime_text" in tasks

if sublime_is_running:
    subprocess.Popen(["subl", "-b", "--command", "unit_testing_run_scheduler"])
else:
    subprocess.Popen(["subl"])

The tests will still run, at least on Linux, or at least sometimes.

gerardroche commented 6 years ago

Just to let you know, because I'll have to come back to it later and I don't knwo when that might be. I've added a hoard of debugging info to this branch: https://github.com/gerardroche/UnitTesting/tree/travis-debugging, a log which can seen here: https://api.travis-ci.org/jobs/286103952/log.txt?deansi=true.

It looks like the plugin_host process is left running when we kill ST. When we check later in the run_tests.py script to see if ST is still running :

tasks = subprocess.check_output(['ps', 'xw']).decode('utf8')
sublime_is_running = "Sublime" in tasks or "sublime_text" in tasks

It evaluates as true.

starting Sublime Text...
*** TASKS ***
   PID TTY      STAT   TIME COMMAND
   180 ?        Ss+    0:00 bash /home/travis/build.sh
  2241 ?        S+     0:00 sh travis.sh bootstrap
  2246 ?        Sl     0:00 /usr/bin/Xvfb :99 -ac -screen 0 1024x768x24
  2291 ?        S+     0:00 sh /home/travis/.config/sublime-text-3/Packages/UnitTesting/sbin/install_sublime_text.sh
  2306 ?        Ssl    0:00 /home/travis/sublime_text_3/sublime_text
  2312 ?        Sl     0:00 /home/travis/sublime_text_3/plugin_host 2306
  2317 ?        R+     0:00 ps wx
killing Sublime Text...
*** TASKS ***
   PID TTY      STAT   TIME COMMAND
   180 ?        Ss+    0:00 bash /home/travis/build.sh
  2241 ?        S+     0:00 sh travis.sh bootstrap
  2246 ?        Sl     0:00 /usr/bin/Xvfb :99 -ac -screen 0 1024x768x24
  2291 ?        S+     0:00 sh /home/travis/.config/sublime-text-3/Packages/UnitTesting/sbin/install_sublime_text.sh
  2312 ?        Sl     0:00 /home/travis/sublime_text_3/plugin_host 2306
  2338 ?        R+     0:00 ps wx
*** outdir contents ***
total 4
drwxr-xr-x 2 travis travis 76 Oct 12 16:35 .
drwx------ 3 travis travis 24 Oct 12 16:35 ..
-rw-r--r-- 1 travis travis  0 Oct 12 16:35 plugin_loaded_2312
-rw-r--r-- 1 travis travis  0 Oct 12 16:35 run_scheduler_2312
-rw-r--r-- 1 travis travis  4 Oct 12 16:35 schedule.json

This could be what is causing the issue or be part of the issue.

randy3k commented 6 years ago

Somehow subl -b --command is not working as expected on travis. After multiple callings, ps wx gives

 2087 ?        S      0:00 sshd: travis@pts/0  
 2088 pts/0    Ss+    0:00 bash /home/travis/build.sh
 3654 ?        Sl     0:00 /usr/bin/Xvfb :99 -ac -screen 0 1024x768x24
 3733 pts/0    S+     0:00 sh sbin/travis.sh run_tests --coverage
 3734 pts/0    S+     0:00 python /home/travis/.config/sublime-text-3/Packages/UnitTesting/sbin/run_tests.py --coverage UnitTesting
 3737 ?        Ssl    0:00 subl
 3745 ?        Sl     0:00 /home/travis/sublime_text_3/plugin_host 3737
 3752 ?        Ssl    0:00 subl -b --command unit_testing_api_ready
 3760 ?        Sl     0:00 /home/travis/sublime_text_3/plugin_host 3752
 3766 ?        Ssl    0:00 subl -b --command unit_testing_api_ready
 3774 ?        Sl     0:00 /home/travis/sublime_text_3/plugin_host 3766
 3780 ?        Ssl    0:00 subl -b --command unit_testing_api_ready
 3788 ?        Sl     0:00 /home/travis/sublime_text_3/plugin_host 3780
 3792 pts/0    R+     0:00 ps xw

somehow subl --command is not able to pick up the opening window.

gerardroche commented 6 years ago

somehow subl --command is not able to pick up the opening window.

Yep. Same for me. I couln't figure out why this was happening. Injecting the scheduler is good workaround.

SublimeText / UnitTesting