Give a setup.py script ?

[sveetch]

It has been asked to furnish a script.py to install the manager. This would be nice to integrate it during Recalbox compilation part.

Problem is, it seems required to be compatible with distutils.

[sveetch]

• 'virtualenv' usage is not really usable within Distutils install, we should be able to drop its usage on Recalbox as it should be safe enough;
• Distutils is a low level module to perform install and is not able to install dependancies from setup.py file like setuptools or pip do.
• We have actually three dependancies, autobreadcrumbs and Django, that are required, the third dep is psutils, is optional and should be allready installed on last Recalbox 3.3.x branch;
• autobreadcrumb is safe enough to install (only depend on Django);
• Django is more safe to install because it does not have any required dependancy;
• Installing webapp project with distutils may seem a little tricky, because it's not really targeted to be a package, just an user-end stuff. It have static files and media files, potentially these files should be modified/deleted during instance work and new ones should be created. This is not really something you can do within a Python package in your /usr/lib/** or other system dir;
• We can install these files somewhere else than into the package, like /tmp/manager/static
• Actually, there is only the sqlite3 database file that should change during instance work. This may be avoided with disabling the 'session' module from Django, but then we totally lose capacity for enabling authentication. Then the only solution would be to have some SGBD instance like mysql/postgres working on the Recalbox, so data could be registered there without to update a sqlite3 database file;
• Also maybe storing the sqlite3 file within /tmp ? (Can't be just stored in memory, else we would have to init it for each Recalbox boot).

[sveetch]

So to resume, some point to check with Recalbox team(@digitalLumberjack @rockaddicted @MikaXII) :

• Need Python packages autobreadcrumb and Django==1.8.x to be ready before (in buildroot?). (This would require to maintain there last package version on Recalbox build system);
• Find a directory to use where the manager can write files like static files, media files or even the database file;
• Maybe use a SGBD to store the manager database ? (not very confident with this as SGBD is a big memory consumption);

When all these point are fixed, i can start to add the 'setup.py' for Distutil usage.

[MikaXII]

It's not easy to create a package with your virtualenv. Currently I don't use virtualenv.

Sample of mk file :

################################################################################
#
# recalbox-manager
#
################################################################################
RECALBOX_MANAGER_VERSION = 1.0.0
RECALBOX_MANAGER_SITE = $(call github,sveetch,recalbox-manager,$(RECALBOX_MANAGER_VERSION))
RECALBOX_MANAGER_DEPENDENCIES = python python-psutil python-django python-autobreadcrumbs

#define RECALBOX_MANAGER_BUILD_CMDS
#    $(MAKE) CC="$(TARGET_CC)" LD="$(TARGET_LD)" -C $(@D) install
#endef

define RECALBOX_MANAGER_INSTALL_TARGET_CMDS
    mkdir -p $(TARGET_DIR)/usr/recalbox-manager
    cp -r $(@D)/* $(TARGET_DIR)/usr/recalbox-manager

    #NOT WORK
    #$(TARGET_DIR)/usr/bin/python2.7 $(TARGET_DIR)/usr/recalbox-manager/manage.py migrate
    #$(INSTALL) -D $(@D)/* $(TARGET_DIR)/usr/recalbox-manager
endef

define RECALBOX_MANAGER_INSTALL_INIT_SYSV
    $(INSTALL) -m 0755 -D package/recalbox-manager/script/S94manager $(TARGET_DIR)/etc/init.d/S94manager
endef

$(eval $(generic-package))

RECALBOX_MANAGER_DEPENDENCIES = python python-psutil python-django python-autobreadcrumbs

There are all dependencies to run your manager but, I need make a script or find an another solution to execute /manager.py migrate one time. After that I try if the manager can launch manually on recalbox and after the init script, I have a sample not complete :

#!/bin/sh
# Start/stop recalbox-manager

PIDFILE=/var/run/recalbox-manager.pid
SITEPATH=/usr/recalbox-manager
#. $SITEPATH/bin/activate

NAME="S94manager"
RUN_AS=`id -u root`
CMD=/usr/recalbox-manager/manage.py
OPTS="runserver 0.0.0.0:80 --noreload" 

do_start() {
    start-stop-daemon --start --background --user $RUN_AS --pidfile $PIDFILE --chuid $RUN_AS --startas $CMD $OPTS
}

do_stop() {
    start-stop-daemon --stop --user $RUN_AS
}

case "$1" in
start)
    echo "Starting $NAME"
    do_start
        ;;
stop)
    echo "Stopping $NAME"
    do_stop
    ;;
restart)
    echo "Restarting $NAME"
    do_stop
    do_start
    ;;
*)
    echo "Usage: /etc/init.d/S94Manager {start|stop|restart}"
    exit 2
    ;;
esac
exit 0

PS : I don't receive notification with your @MikaXII

[sveetch]

You should have continued discussion in #29 as this issue was a mind note about preparing package with "setup.py", but no problem, let's continue here.

Well first, virtualenv is only used to don't have to manage package system for project dependancies. It also avoid package versions conflict on system. I understand this is not really suitable within an OS building.

But in the Recalbox context, you can avoid virtualenv requirement, because there should never have conflict as there are no other Python webapp projects.

For the 'migrate' command step, there is a little trick you could use because this migrate is only here to initialize project database. But we don't really use it, Django require it but don't care if you created it from your install or not. So the trick is to install the manager project everywhere you want on a Linux system, copy/paste the created db.sqlite3 into your manager project install (within your Recalbox building) and that's all, Django won't never complain again about the migrate command.

From there you should be able to launch the manage.py and the runserver command.

A note about python-autobreadcrumbs, i dont have knowledge about a package system for autobreacrumb that i created, if you don't allready packaged it, you will have to because i dont think it allready exists anywhere.

[MikaXII]

"copy/paste the created db.sqlite3 into your manager project install (within your Recalbox building) " I've exactly the same idea =D.

For python-autobreadcrumbs I create the buildroot package, we have all dependencies required I test it today :)

[sveetch]

Great news, let me know about if you succeed or not to achieve this. For autobreadcrumb the only requirement is Django and i dont plan to add any dependancy.

I'm letting this issue open until you confirm any success.

[sveetch]

Ok seems it's not really needed now that the wonderful @MikaXII has resolved its issue #29

And frankly, it saved my ass :)