JamesNewton
commented
5 years ago Ok, a few changes:
-
There is a sub folder under job called
run, and that run folder is what is watched and triggers the service, not the job folder. That allows things in job to be edited / changed without re-triggering the service. -
There is a sub folder under job called
logswhere log files end up. -
The systemd files were changed as follows:
ddejob.path[Unit] Description=DDEJob [Path] PathModified=/srv/samba/share/job/run [Install] WantedBy=multi-user.target
ddejob.service
[Unit]
Description=DDEJob
[Service]
ExecStart=/srv/samba/share/job/RunJobsand then started with
sudo systemctl enable ddejob.path
sudo systemctl start ddejob.pathNote: if you edit those, do a
sudo systemctl daemon-reloadThe RunJob script is:
#!/usr/bin/env bash
# first remove prior job logs and junk
cd /root/Documents/dde
for i in /srv/samba/share/job/run/*.dde; do
[ -f "$i" ] || break
jobname=$(basename -- "$i")
echo "running $jobname"
echo "Logfile for $jobname on $(date +%Y%m%d_%H%M%S)">/srv/samba/share/job/logs/$jobname.tmp
sudo node core define_and_start_job $i >> /srv/samba/share/job/logs/$jobname.tmp
# must use sudo or node doesnt know who the user is and cant find the dde_apps folder.
rm /srv/samba/share/job/logs/*.log
mv /srv/samba/share/job/logs/$jobname.tmp /srv/samba/share/job/logs/$jobname.log
# copy large log files only after they are fully written so dde doesnt seem them until complete
rm $i
# remove the job file
doneso now the DDE job:
new Job({name: "test_job_engine", do_list: [
Dexter.write_to_robot(
"new Job({name: \"my_job\", do_list: [Dexter.move_all_joints([30, 45, 60, 90, 120]), Dexter.move_all_joints([0, 0, 0, 0, 0])]})"
, "job/run/try.dde"
)
]})has the same result, but we can add code to read_from_robot("my_job_log", "job/logs/try.dde.log") and when that returns nothing, just loop until it returns the result.
Seems to work quite nicely.
cfry
Starting this issue to record development history so we know why something was done in the future. The current status (only available on the very latest images) is here:
https://github.com/HaddingtonDynamics/Dexter/wiki/DDE/#job-engine-on-dexter
Goal: Make it easy for DDE on a PC to write jobs to Dexter which are then run ON Dexter. These are one time jobs, with DDE in control of dispatch, so you don't have to SSH in, or depend on SAMBA or anything else. The functionality can even be integrated into DDE.
And this allows us to start the job ourselves, it doesn't have to be started automatically when Dexter fires up. Experts can figure out how to add an /etc/rc.local
First, we must have DDE on Dexter. In this case, we don't need to make a distributable package, we just want to run the source. And having the source directly run means we can develop on Dexter and also use parts of it in other ways (see Job Engine below). So instead of installing the Electron package, we just install the source and use npm to pull in the dependencies. Note: This requires the 16.04 version of the operating system on Dexter, and the Dexter must be connected to the internet (e.g. plugged into a router, or wifi)
Note: If you need to go back to a prior version, use
git checkout v3.5.2or whatever. v3.0.7 is known to work as well. Also, thenode run startmay not work until after you comment out the serial stuff (see below).Of course, the GUI part of the app will only be visible with an X-Server running and since Dexter does not have a video adapter, this must be a remote the X-Windows Desktop. On the current images, an icon is provided to launch DDE from the desktop when logged in via X-Windows. The program takes a while to load (need faster SD Card and interface?) but operation isn't horribly slow.
A "dde_apps" folder for the GUI run of DDE is created under the "/root" folder (alongside Documents, not in it) for the DDE application. Setting the dexter0 ip address to
localhostin the/root/dde_apps/dde_init.jsfile allows local connection of DDE to DexRun.To run DDE jobs without the full DDE GUI interface, e.g. via SSH, you can start them from
~/Documents/ddewith the command:node core define_and_start_job job_file.ddeThere are a few things to tweek before that will work:
From /root/Documents/dde,
nano core/serial.jsand comment out the line:const SerialPort = require('serialport'). There is some mismatch between the component that manages serial ports on our OS vs others.Job Engine Initialization: When run for the first time, the job engine creates a
dde_init.jsfile in the/root/Documents/dde_appsfolder. (note this is different than for the GUI DDE on Dexter which is in/root/dde_init.js). The job engine defaults to simulate, so the jobs don't actually make the robot move until the dde_init file is edited to add,simulate: falseafter the IP address in the definition of dexter0. The IP address is set tolocalhostso it will work no matter what IP address Dexter is actually assigned./root/Documents/dde_apps/dde_init.js:Keep in mind the version of DDE on Dexter may need to be updated. It's 3.0.7 on the initial release of the 16.04 image.
You may want to use
node core define_and_start_job /srv/samba/share/job_file.ddeWhen you 'run a job' as defined above, it sets window.platform to "node". If you are in dde,
window.platform == "dde"will evaluate to true. That means you can customize any code written based on this "platform" i.e.The system software takes advantage of this. One important case is that the "out" function is defined as:
Thus when running on node, 'out' only pays attention to its first arg, and it sends the first arg directly to the console.
On the development image, (for the next release) there is an updated /etc/systemd/system path and service which looks for any changes in the /srv/samba/share/job folder and when seen, executes the following script "RunJob" (also in that folder):
which then fires off the job engine, does the job, and puts the resulting output in aYYMMDD_HHMMSS.log file.
For example:
makes Dexter move, and then the file /job/try.dde.20190327_233018.log contains:
I'm happy about the ability to get a job onto Dexter and run it /on Dexter/ using nothing more than DDE. I like that the job file is deleted after it's run. I like that systemd is /apparently/ smart enough not to start the script until the file is done being written (apparently?) and I like that the output is retained.
What I don't like is that it's retained forever. If people use that alot, it's going to fill up the folder with junk. I see a few easy solutions and some harder ones.
<filename>string. This on the TODO list anyway, sometime after slaying the dragon. (note: it's on READ not write, because the output of the bash shell gets buffered back to DDE, you write the file, then read the result, which actually triggers running the file). I don't want to do that yet, because it's complex.Which reminds me, there is currently no way to know what the log file name will be, so DDE can't really read back the result. Since the job file gets whacked anyway, I think the log file should NOT include the date and time and just be.log. So the test.dde job would result in a test.dde.log file, DDE can read that, and (as per option 1) next time you write test.dde, the old test.dde.log file gets whacked automatically and replaced with a new one. That seems like a simple solution, no?
Last problem, DDE doesn't know when the job is finished. So the log file keeps expanding, and shouldn't be read_from_robot'ed until it's done. I think the solution to that is to always write the log into the file with date and time, then when the job has finished, rename that file to the.log name. DDE can poll and when it returns more than a null string, it can be sure it's getting the entire string.