Enhancement proposal for server module example and usage

GoogleCodeExporter commented 9 years ago

Happy new year and thanx for the ongoing work with webiopi.

I have done some implementation work with custom python server modules now and 
implemented a way that helps to make this task easier for me. Maybe we can 
share this with other users of webiopi server modules.

I implemented a server module called webiopi_mysrv.py that has all necessary 
things inside it to be run as server module in interactive mode (development 
time) as well in deamon mode (just runtime). I place this module in a known 
path (e.g. /home/pi/webiopi_mysrv/ and register it once with a custom 
webiopi_mysrv.py.init (see attachments) that gets installed as webiopi_mysrv in 
/etc/init.d/ the usual way. This way that module can be started and stopped as 
deamon or even automatically at startup without having to do the installation 
stuff too often. I also can run it in interactive mode and stop in with CTRL-C 
during development and bugfixing. I can have multiple different versions of 
webiopi_mysrv.py and just by renaming them I can test different alternatives 
even in deamon mode without having to do the installation stuff again and again.

I also discovered that when my module is terminated in deamon mode, the 
cleaning up code at the end is not executed. For that reason I added some code 
that catches the SIGTERM when the module is terminated in deamon mode so that 
my cleanup code is also executed in this case. I have attached a version of 
webiopi_mysrv.py that is implemented according to demo.py but contains all the 
stuff that makes server initialization and finalization more streamlined and 
does cleanup the same way in interactive and deamon mode to be consistent. It 
also contains mode comments for others to understand how to use it and adds an 
example macro without parameters for completeness of pythton server module 
usage possibilities.

Maybe this is a good improvement for the python server module example and its 
usage documentation.

Best regards.

Original issue reported on code.google.com by andreas....@googlemail.com on 4 Jan 2013 at 10:26

Attachments:

GoogleCodeExporter commented 9 years ago

Original comment by tro...@trouch.com on 4 Jan 2013 at 8:24

Added labels: Type-Enhancement
Removed labels: Type-Defect

GoogleCodeExporter commented 9 years ago

Just made few changes on the SVN, please take a look on r680
I've added signal handling directly in WebIOPi core to break the loop, any code 
after runloop should now be executed when SIGTERM is received. I also updated 
the demo.
For the init script, I will add it on the wiki as we only have to copy the 
default init script and change $DAEMON_ARGS

Original comment by tro...@trouch.com on 7 Jan 2013 at 12:26

Changed state: Started

GoogleCodeExporter commented 9 years ago

Thanx, I have seen the changes to webiopi.py and the update of demo.py. I 
assume that SIGINT is automatically thrown when CTRL-C is pressed so this way 
we have now a very uniform termination handling embedded into webiopi core. And 
with the updated comments in demo.py it will be much easier to develop own 
server scripts as demo.py is mostly self-documenting now.

Concerning the other ideas behind webiopi_mysrv my intent was not to integrate 
that all into the webiopi.py core and its installation setup. My intent was 
more like that I assume when someone uses the Raspi and Webiopi over time more 
intesively as a e.g. home automation server, sooner or later will have the need 
to have some functionality implemented as an additional server script that will 
also provide some remote usable macros as you provide and maybe some 
initializations or cleanup actions. It's absolutely clear that this additional 
functionality should never never be inside webiopi.py as this implements the 
what I would call basic WEB API functionalities and thats fully ok so.

So my idea behind the other things with webiopi_mysrv is more like having a 
kind of luxury "example/demo" for all those that have to implement a server 
script on their own. With "luxury" I mean the good practice of símple 
ready-made reusability for others. So it comes with a predefined name being a 
little bit more distinct that just "demo" but indicating its clear relationship 
and purpose (so I choose webiopi_mysrv). It also comes with a complete 
init-script so that its installation as a server script is equally easy as the 
installtion of the webiopi core is already today. It is placed on a 
(choosable?)standard path (so I choose ../webiopi_mysrv to have it clearly 
separated from the webiopi core, but any other path can of course be possible) 
which in turn allows the init-script to remain simple. It could also comes with 
a very simple default implementation doing nothing more than just the server 
instantiation with default values, the loop definition with an empty loop and 
e.g. the 1 second delay and the server.stop() from demo.py and has comments 
inside where to place the specific things (which you already did in the 
r680-version demo.py right now).

The full automatic setup of webiopi would then comprise the following things:
Already in place:
- Setup of the webiopi core with all the native C stuff and its python 
integration
- Setup of the deamon-related scripting related to init.d allowing webiopi 
start/stop and manually update-rc.d if wanted
Additionally:
- Setup of the webiopi_mysrv extension with the "empty" default implementation 
that does exactly the same as webiopi core
- Setup of the modified additional deamon script with the changed $DAEMON_ARGS
 pointing to the webiopi_mysrv default location and thus providing the webiopi_mysrv start/stop and manually update-rc.d options

Having this setup, a developer will have the following options on server side:
-A Use only the webiopi core from shell level ($ python -m webiopi ...)
-B Use only the webiopi core in deamon mode ($ webiopi start/stop)
-C Use only the webiopi core in atomatic startup mode ($ update-rc.d webiopi  
defaults)
-D Use the webiopi_mysrv "empty" from shell level ($ python webiopi_mysrv.py 
...)
-E Use the webiopi_mysrv "empty" in deamon mode ($ webiopi_mysrv start/stop)
-F Use the webiopi_mysrv "empty" in atomatic startup mode ($ update-rc.d 
webiopi_mysrv defaults)
- Change webiopi_mysrv.py and add whatever is needed and just use D, E, F from 
above to test or run the "enriched" version of webiopi_mysrv.py without having 
to do any setup again

I even could setup a complete SD-Card with Raspbian, DHCP + SSH enabled and 
Webiopi like above once, duplicate it, put it into a new Raspi, change the 
hostname, throw any changed webiopi_mysrv.py onto it via SSH or some FTP and 
let it run the new webiopi server with added functionalities.

That are my thoughts behind webiopi_mysrv (or whatever name ist is given), but 
you already added some things which is also ok for me.

Best regards

Original comment by andreas....@googlemail.com on 7 Jan 2013 at 11:56

GoogleCodeExporter commented 9 years ago

Thanks for your feedback.
Yes, SIGINT handles CTRL-C.

If I well understand, you actually want to give more options by default to the 
user.
I will reorganize a little the example, and add a simpler one. I will also add 
some init example as you suggest.

The thing tricky with this project, is that I have to manage users and 
developers from totally noob which use WebIOPi with the PiStore, to advanced 
developers which use the WebIOPi server library.
I have also in mind issue #4 http://code.google.com/p/webiopi/issues/detail?id=4

There is one thing I still don't understand, what do you mean by "setup" in 
"without having to do any setup again" ? Is it setup.sh script or the custom 
following setup ?

Original comment by tro...@trouch.com on 7 Jan 2013 at 2:00

GoogleCodeExporter commented 9 years ago

It's just the comfort of setting up the webiopi_mysrv.py thing in a standard 
way completely only once direct after webiopi core setup. Afterwards I can just 
modify the webiopi_mysrv.py module or rename different versions of it like 
webiopi_mysrv_1.py and webiopi_mysrv_2.py and have all the stuff concerning 
init.d handling ready to use in place. This is very useful in intensive testing 
scenarios.

To understand this type of comfort a little better I can explain my 
"development environment" a little more. As I mentioned before in another post, 
I use mainly the webiopi REST-API via HTTP calls from Android native Apps that 
I write myself using Eclipse (on Windows). The Raspi is great hardware for low 
cost/low energy consumption runtime, but only limited for intensive interactive 
software development. For that reason I also have Python on my Windows machine 
just beside the Eclipse Java stuff for Android. In order to be able to run 
webiopi for development and testing purposes on the same Windows machine I have 
written a "fake" GPIO.py module sitting in a /_webiopi subdirectory including 
an empty __init__.py module. This allows to have the import statement in 
webiopi.py or even demo.py unchanged and so I can run webiopi or any server 
script of my own without using a Raspi obn my Windows machine. The fake GPIO.py 
implements all (most?) calls from the original one and even remembers the state 
of setting outputs or pwm to something. If something is set e.g. like 
GPIO.output(5, GPIO.HIGH) it stores the value in state variables/lists and 
prints a log message on the console to see the call taking place.

So when I test an Andriod app using webiopi, I do it this way:
- Run webiopi or my own server script from within IDLE so see the printed output
- The webiopi http server gets bound to the local ip which is also accessible 
via localhost
- Run the Android emulator from Eclipse
- Deploy the App to run on the emulator
- Direct the Emulator-hosted App webiopi-related http REST calls to 10.0.2.2 
which get bridged to localhost by the Android ADB
- This way I can do any webiopi call within Android and see inside the IDLE 
webiopi console the activities.
- Sometimes it's additionally necessary to trigger some http calls from 
outside. To do that, I use Firefox and the Addon "HttpRequester" on the same 
machine. From there I can send any kind of HTTP request also to localhost and 
trigger within webiopi whatever I want (and see/manipulate the http headers, 
responses, sometimes the http errors ...). This way I also can simulate the 
change of GPIO inputs via Firefox. I just add 2 test macros 
(gpio2high(channel), gpio2low(channel) to my server script (simple to guess 
what they do ...) and see if my Android app does the right thing e.g. when 
scanning for GPIO input value changes.
- I can then modify my webiopi server script (and/or my Android app) until they 
do what they should without bugs.
- If my webiopi server script is final or has reached some stable state, I just 
can FTP my server script to a real Raspi with the predefined name to the 
predefined path, deploy the APK to my real Android Smartphone, reboot the Raspi 
and can finish testing instantly with real devices (provided webiopi_mysrv has 
undergone the update-rc.d stuff).

Advantages are:
- I can do Android app and Python server script development (and test) on one 
hardware machine (without the need for a real Raspi or a complicated Raspi 
emulator)
- I can manipulate the GPIOs whatever I want without the risk of damaging real 
hardware or peripherals attached to the GPIOs
- As my webiopi_myserv.py resides on the physical Raspi in a userspace path 
(e.g. /home/pi/webiopi_mysrv/ ) I don't need any SSH sudo ... to update it. I 
can just open a WinSCP session to the Raspi, drag'n-drop the newest 
webiopi_myserv.py to the correct location, reboot the raspi (e.g. with one of 
the Android remote control apps for the Raspi) and try the result now with real 
Smartphone and Raspi hardware.

So that's the kind of development comfort that I personally like and that I 
want to share. On the other hand, you are right, this kind of development setup 
is nothing for noobs and may be too complicated for simple usage requirements 
like switching LEDs on and off.

Anyway, the REST API you already provide is a very good base providing a simple 
and performant way to control Linux server GPIOs from an Android smartphone via 
TCP/IP. If some day in the future also I2C and SPI get integrated, something 
like a dream for real open source home automation will become true: Using and 
controlling I2C and SPI based electronics via Smartphones (either as native 
apps or via Browser and Javascript) and TCP/IP in an affordable and efficient 
runtime environment that is very rich and dynamic (Python). And, with the 
server script and its macro functionality, it's also possible to implement 
functionality on server side when it makes sense. But this would be another 
long discussion ...

Concerning issue #4 that you mentioned it may make sense to also add a shutdown 
script hook in addition to the startup script mentioned there.

Best reagards

Original comment by andreas....@googlemail.com on 7 Jan 2013 at 4:21

GoogleCodeExporter commented 9 years ago

Okay, what a complex setup you have ! On my side:
- A computer with Eclipse J2EE + CDT + Python + Shell + Android plugins.
- A network shared folder (samba or netatalk) on the Pi, mounted on my computer
- An Eclipse project on my computer with a folder symlinked to the webiopi 
folder on my Pi through the network sharing.
- A terminal on my computer with a SSH connection to my Pi, in order to launch 
and debug the server on the Pi.
So I can do all my coding stuff on my computer and use my Pi to execute 
WebIOPi. I can also execute some part of the server directly within Eclipse on 
my computer.

I only run setup.sh when :
- I'm working on C native code
- Or if I'm working on both examples and webiopi.py

When I change and debug webiopi.py, I simply execute "sudo python webiopi.py" 
in the python folder. It warns about an existing webiopi, but it works.

Please take a look on r686, I've moved demo to examples/custom, and added a 
basic example too.
There is also daemon script examples we can directly use in each basic and 
custom folder by typing "sudo ./webiopi_custom.daemon start" for instance. Or 
we can copy it in /etc/init.d then change the HOME var to point the the correct 
folder.
I also display again webiopi usage at the end of install and removed default 
boot startup to avoid previous install corruption.

Original comment by tro...@trouch.com on 7 Jan 2013 at 4:41

GoogleCodeExporter commented 9 years ago

0.5.3 released with suggested changes, wiki updated

Original comment by tro...@trouch.com on 8 Jan 2013 at 9:08

Changed state: Done

luking-dev / webiopi

Enhancement proposal for server module example and usage #17