Using PyRTOS on windows. High CPU level

Hi, like I've written in the title I'm using PyRTOS on windows. I've been successful in this, I've created a basic task that prints hello world every 5 seconds.

But even if the task is lightweight, the CPU consumption of the python process is about 7% in task manager, and the overall CPU consumption go from 2-3% (when the program is not running) to 17-20% (when PyRTOS is running).

I know that PyRTOS is an embedded system, and its main use is for embedded platform, but I'd like to use it to create a collection of scripts that are always running on my systems, and at a certain event (for example at a certain hour) they trigger a certain action. Basically I'd like to use it to automate a few stuff on my pc. So I'd keep PyRTOS always running, executing it at system startup, and create a few tasks that runs with a certain frequency.

I'd like to know if there's a way to decrease the impact of PyRTOS on the CPU, also because I guess that this drains a lot of battery over time. Can we do something, or that high CPU percentage is just due to the RTOS that is doing heavy stuff under the hood?

You are right that it behaves this way because it is designed for embedded systems. Because it is designed for real-time applications, it needs to be running as fast as the hardware will allow. On a desktop OS like Windows, where the RTOS isn't being used as the OS for the entire device, that can be problematic. For the intended use case, nothing is running outside of the RTOS, so it's fine for it to consume all of the CPU power. That said, I would like it to still be usable outside of embedded systems, so let's figure this out!

The solution is actually not that difficult! The problem here is the scheduler. The scheduler runs as fast as the hardware allows (or in this case, as fast as Windows allows). We want it to run slower.

I can think of two ways to solve this. One is to write your own scheduler. I designed PyRTOS to allow for this, but I don't have any instructions for doing so. If you choose this route, I would suggest copying the code for the scheduler I wrote, and then modifying it to add some kind of delay or timing thing that causes it to sleep for a small period of time for each scheduler loop. If precision timing is not important to your application, a delay of 0.01 (10 milliseconds) using time.sleep() might be a good starting place. Use trial and error to work out how much lag you are willing to tolerate between iterations and sleep for some value a little less than that. If timing precision is important, Python's time.sleep() won't be sufficient, and you will need a more precise timing module. Just make sure it suspends the process rather than using a busy loop, otherwise it won't reduce CPU usage. Also, high precision timing loops require some unintuitive techniques. (Here's an article I wrote a while back on how to do this in terms of video game scheduling. There's some code in there a little past halfway down that you could probably adapt to your use case.)

If you don't need high precision though, the other way to solve this would be to write a service routine for PyRTOS that has a brief sleep. In the OS API section of the documentation, you will find a "Service Routines" subsection. I haven't added example code, but service routines are really easy to add. The most basic service routine is just a function that takes no arguments and has no return value. You register it with pyRTOS.add_service_routine(service_routine). From there, it will be called every time the scheduler runs. For a (low precision) 1 millisecond delay (which might be enough), just make a function with the line time.sleep(0.001) (don't forget to import time). From there, test your program, look at how much CPU it is using, and adjust the delay value until it is using an acceptable amount of CPU time. (Of course, also make sure your tasks are running like they should and not suffering from excessive lag. There may be a balance necessary here.)

I think that should get you where you want to be. If you need more help, let me know.

I just added the OS API -> Service Routines section to the table of contents. I've also added an example service routine that does exactly what you need. It's at the end of the Templates & Examples section, and it has been added to the TOC as well, to make it easy to find. I hope that makes it really easy to solve your problem!

You are right that it behaves this way because it is designed for embedded systems. Because it is designed for real-time applications, it needs to be running as fast as the hardware will allow. On a desktop OS like Windows, where the RTOS isn't being used as the OS for the entire device, that can be problematic. For the intended use case, nothing is running outside of the RTOS, so it's fine for it to consume all of the CPU power. That said, I would like it to still be usable outside of embedded systems, so let's figure this out!

The solution is actually not that difficult! The problem here is the scheduler. The scheduler runs as fast as the hardware allows (or in this case, as fast as Windows allows). We want it to run slower.

I can think of two ways to solve this. One is to write your own scheduler. I designed PyRTOS to allow for this, but I don't have any instructions for doing so. If you choose this route, I would suggest copying the code for the scheduler I wrote, and then modifying it to add some kind of delay or timing thing that causes it to sleep for a small period of time for each scheduler loop. If precision timing is not important to your application, a delay of 0.01 (10 milliseconds) using time.sleep() might be a good starting place. Use trial and error to work out how much lag you are willing to tolerate between iterations and sleep for some value a little less than that. If timing precision is important, Python's time.sleep() won't be sufficient, and you will need a more precise timing module. Just make sure it suspends the process rather than using a busy loop, otherwise it won't reduce CPU usage. Also, high precision timing loops require some unintuitive techniques. (Here's an article I wrote a while back on how to do this in terms of video game scheduling. There's some code in there a little past halfway down that you could probably adapt to your use case.)

If you don't need high precision though, the other way to solve this would be to write a service routine for PyRTOS that has a brief sleep. In the OS API section of the documentation, you will find a "Service Routines" subsection. I haven't added example code, but service routines are really easy to add. The most basic service routine is just a function that takes no arguments and has no return value. You register it with pyRTOS.add_service_routine(service_routine). From there, it will be called every time the scheduler runs. For a (low precision) 1 millisecond delay (which might be enough), just make a function with the line time.sleep(0.001) (don't forget to import time). From there, test your program, look at how much CPU it is using, and adjust the delay value until it is using an acceptable amount of CPU time. (Of course, also make sure your tasks are running like they should and not suffering from excessive lag. There may be a balance necessary here.)

I think that should get you where you want to be. If you need more help, let me know.

I just added the OS API -> Service Routines section to the table of contents. I've also added an example service routine that does exactly what you need. It's at the end of the Templates & Examples section, and it has been added to the TOC as well, to make it easy to find. I hope that makes it really easy to solve your problem!

Thank you very much for the exhaustive response! I've tried adding a service routine, with just a 1ms time delay and it does the job! Very low CPU usage. Only thing I've noticed: the service routine must be added before starting the task, otherwise it won't get executed (and we won't have any delay, leaving cpu usage untouched)

That's odd. It should start running any service routine as soon as it is added. Service routines are even run outside the scheduler, so it should run regardless of what is added first.

Hmm, I'm noticing a few functions in the pyRTOS module that should have a global statement at the top to pull in module level variables, and one of those is the add_service_routine() function. I wonder if that has anything to do with it.

So I just fixed those two bugs. I have no clue if they are the cause of the problem, but they needed fixed anyway, and they might be. If you have a chance to update to the newest version and see if that bug still persists, let me know if it is still an issue.

Anyhow, aside from that, thanks for posting this issue! It really helps me cover things I didn't think about, like documentation for uses cases I hadn't considered much. And of course, it helped me discover a few minor bugs, which I really appreciate.

You're welcome, thanks to you too for helping me reducing CPU usage! I've tested it using the new version, but it still does the same. You need to start the task after you've defined the service routine, otherwise s.r. won't start. To me it's not a big issue, maybe you just need to write in documentation this behaviour, that service routine should be defined before starting the task. This is the code I'm executing, if you want to give a look:

import pyRTOS
import time

def setup():
    pass

def thread_loop():
    print("Hello world")

# self is the thread object this runs in
def task(self):

    ### Setup code here

    setup()

    ### End Setup code

    # Pass control back to RTOS
    yield

    # Thread loop
    while True:

        # Remember to yield once in a while (to give control back to the OS)

        ### Start Work code
        thread_loop()
        ### End Work code

        # Adjust the timing here (in seconds) to fix the interval between each
        # re-wake of the thread (the os will automatically wake it every tot time)
        # THIS IS A BLOCKING DELAY! TASK EXECUTION IS BLOCKED FOR THIS TIME
        yield [pyRTOS.timeout(5)]

# Now we create the task
# OSS: This is the entry point of the file. Execution starts here.
# The name of the task you need to pass as first parameter is the name
# of the function that implements the task. In this case, it's 
# the "task()" function implemented above.
# Mailboxes (for messages) are disabled
pyRTOS.add_task(pyRTOS.Task(task, name="task1"))

# Let's add a service routine that implements a 1ms delay every time the scheduler
# is called, in order to slow down the execution, and having a minor impact on CPU
pyRTOS.start()
pyRTOS.add_service_routine(lambda: time.sleep(0.1))

PS: Another hint I give you is that you should publish the code on pypi, so we can install this with pip.. It's true that this is an embedded platform, but could become useful on windows/mac/linux too.. I've put PyRTOS folder inito site-packages folder of my main python installation, so I can call PyRTOS when I want

Technically, it is best practice to register service routines before starting tasks or the scheduler (and yeah, I probably should mention this in the documentation), but my code does not seem to require this, and it annoys me that it does. Thanks for the code. I'll have to look into this later, when I have more time, but thanks for identifying the issue.

You are right that I should put this on pypi. I think I should probably review all of the issues and implement (or formally abandon, if its not worth implementing) anything hanging first, but it's on my list of things to do.

Anyhow, you are welcome, and thank you! I'm closing this issue, but I'll open a new one for this strange behavior for SRs.

Lol! Alright, I was writing up my own notes about this bug with the SR, and I realized it wasn't a bug with my code.

Here are the last two lines of your code:

pyRTOS.start()
pyRTOS.add_service_routine(lambda: time.sleep(0.1))

pyRTOS.start() enters an infinite loop. It never returns. I misunderstood what you meant when you talked about starting the "task", because tasks can be manually initialized (this runs through their initialization code to the first yield, and it happens automatically if you don't do it manually).

So the service routine never gets added, because pyRTOS.start() never returns. It never gets to that line of code. In theory, you could add the service routine from a task though (probably shouldn't, but I can think of at least one use case for this).

Anyhow, I'll add a note to the documentation mentioning that pyRTOS.start() does not return, so any code after it will never be run. If you made this mistake, I'm sure others will, and I can't reasonably expect those not familiar with embedded RTOS usage to automatically understand this. Thanks for helping me to discover this oversight on my part!

Rybec / pyRTOS

Using PyRTOS on windows. High CPU level #11