usage of other cam than raspicam ?

[robbyleh]

Hello Joakim,

I'm glad about your project is documented and public for all of us who are having this problems :-) Thank you very much! I'll keep you up2date with my project. I'm not really a good programmer in C, I would categorize myself in the box of hobbyists always having fun with that ;-), but out of C i'm definitely technically gifted:-) My question: can I use a standard webcam with your code? I have a RPI 3 Model B and a used Logitech C270 webcam which is also supported by Raspberry. Following your REAME.txt I got stuck at this position: ... " command line options used by the raspivid program" ... After investigating I got the fact, this binarys can be used only with the raspi-cam... So I installed some kind of software which belongs to standard USB webcams: ... sudo apt-get install fswebcam ... Now I'm able to take captrure some picture for example with ... fswebcam -r 640x480 --no-banner ~/test.jpg ... But I have no idea to combine this usb webcam with your code. The last few days planning time in mind I was hoping this code would work also wtih other cameras ... Thak you in advance. Robby

[robbyleh]

Hi Joakim,

in the configuration file I don't understand how the variable are evaluated by cacierge. Example: The Email script output in the log seems to work with the variable names but not with the variable values.

In config file the email statement is:

# Mail image and json.
match_group_done_cmd=python /usr/local/share/catcierge/catcierge-sendmail-new.py --to to@mail.de --from from@mail.de --smtp smtp.strato.de:587 --password pwpwpw --json "%template_path:match_group_done_full%" --images %match_group_id%-combined-steps.png --extra "%template_path:match_group_done_full%"

The output shows this

...
[2017-06-10 12:03:42.209852]  Saving image /home/robby/Katzenklappe/run/images//374d9fc8755ab5ca193e11dbb36e397bbbdf243d/match_fail_2017-06-10_12_03_42.123392__4.png
/home/robby/Katzenklappe/tools/lock.py:5: RuntimeWarning: This channel is already in use, continuing anyway.  Use GPIO.setwarnings(False) to disable warnings.
  GPIO.setup(5, GPIO.OUT)
[2017-06-10 12:03:42.223665]  Unknown template variable "template_path:match_group_done_full"
[2017-06-10 12:03:42.223741]  Failed to execute command "python /usr/local/share/catcierge/catcierge-sendmail-new.py --to to@mail.de --from from@mail.de --smtp smtp.strato.de:587 --password "pwpwpw" --json "%template_path:match_group_done_full%" --images %match_group_id%-combined-steps.png --extra "%template_path:match_group_done_full%""!
...

Can you show me what is wrong here ? Thanks Robby

[JoakimSoderberg]

Hey,

This is maybe not the most user friendly part of the project.

But it is kind of hard to help without also having your full config.

---

In your first error you got this:

[2017-06-10 00:57:58.587013] Unknown template variable "RO5 --json ""
[2017-06-10 00:57:58.587098] Failed to execute command "python /usr/local/share/catcierge/catcierge-sendmail-new.py --to myemail@mail.de --from myemail@mail.de --smtp smtp.strato.de:587 --password MyPassword --json "%template_path:match_group_done_full%" --images %match_group_id%-combined-steps.png --extra "%template_path:match_group_done_full%""!

That error makes me suspect you have a % sign in your password? If that is the case you need to escape it with %%

---

To understand how this works... first you can start by looking at the command help:

Command settings:
  These are commands that will be executed when certain events occur.
  Variables can be passed to these commands such as:
    %state%, %match_success% and so on.
  To see a list of variables use --cmdhelp

    --cmdhelp                                Shows command output variable help.
    --match_group_done_cmd CMD [CMD ...]     Command to run when the match_group_done event is triggered. Event when all steps for
                                             match group has been successfully performed. This is most likely what you want to
                                             trigger most stuff on.
    --state_change_cmd CMD [CMD ...]         Command to run when the state_change event is triggered. State machine state changed.
    --do_lockout_cmd CMD [CMD ...]           Command to run when the do_lockout event is triggered. Triggered right before a lockout
                                             is performed.
    --do_unlock_cmd CMD [CMD ...]            Command to run when the do_unlock event is triggered. Triggered right before a unlock
                                             is performed.
    --save_img_cmd CMD [CMD ...]             Command to run when the save_img event is triggered. Event after all images for a match
                                             group have been saved to disk.
    --match_done_cmd CMD [CMD ...]           Command to run when the match_done event is triggered. Triggered after each match in a
                                             match group.
    --frame_obstructed_cmd CMD [CMD ...]     Command to run when the frame_obstructed event is triggered. Right after the camera
                                             view has been obstructed and the obstruct image has been saved.
    --rfid_detect_cmd CMD [CMD ...]          Command to run when the rfid_detect event is triggered. Triggered on each time one of
                                             the readers have detected a tag
    --rfid_match_cmd CMD [CMD ...]           Command to run when the rfid_match event is triggered. When an RFID match has been
                                             completed, this includes both readers having been read.
    -u, --uservar USERVAR [USERVAR ...]      Adds a user defined variable that can then be used when generating templates or
                                             executing custom commands. This is useful when passing passwords or similar, so those
                                             don't have to be defined in the script but when catcierge is started instead. And can
                                             then also be used in multiple places.

as you see above, what your setting does is that once all steps are done in a match the command match_group_done_cmd is run, this is when we want to send the mail.

We send the mail with an external python script.

But of course we somehow need to pass WHAT to send to that script... This is when these variables are used:

Variables can be passed to these commands such as:
    %state%, %match_success% and so on.

So a variable starts with a % and ends with a %. So if you have any other text in the command that needs to show a % it has to be escaped with %%.

To get a list of all the available variables on can use --cmdhelp:

Output template variables:
                         state   The current state machine state.
                    prev_state   The previous state machine state.
                       matcher   The matching algorithm used.
                     matchtime   Value of --matchtime.
             ok_matches_needed   Value of --ok_matches_needed
             no_final_decision   Value of --no_final_decision
                lockout_method   Value of --lockout_method.
                  lockout_time   Value of --lockout_time.
                 lockout_error   Value of --lockout_error.
           lockout_error_delay   Value of --lockout_error_delay.
                   output_path   The output path specified via --output_path.
             match_output_path   The output path specified via --match_output_path.
             steps_output_path   The output path specified via --steps_output_path.
          obstruct_output_path   The output path specified via --obstruct_output_path.
          template_output_path   The output path specified via --template_output_path.
                match_group_id   Match group ID.
        match_group_start_time   Match group start time.
          match_group_end_time   Match group end time.
           match_group_success   Match group success status. 1 or 0
       match_group_success_str   Match group success status, as a string 'success' or 'fail'.
     match_group_success_count   Match group success count.
    match_group_final_decision   Did the match group veto the final decision?
              match_group_desc   Match group description.
         match_group_direction   The match group direction (based on all match directions).
             match_group_count   Match group count o matches so far.
         match_group_max_count   Match group max number of matches that will be made.
             obstruct_filename   Filename for the obstruct image for the current match group.
                 obstruct_path   Path for the obstruct image (excluding filename).
                    matchcur_*   Gets the current match while matching.
                    match#_idx   Gets the current match index, that is #. Makes sense to use with matchcur_*
                     match#_id   Unique ID for match #.
               match#_filename   Image filenamefor match #.
                   match#_path   Image output path for match # (excluding filename).
                match#_success   Success status for match #.
              match#_direction   Direction for match #.
            match#_description   Description of match #.
                 match#_result   Result for match #.
                   match#_time   Time of match #.
         match#_step#_filename   Image filename for match step # for match #.
             match#_step#_path   Image path for match step # for match # (excluding filename).
             match#_step#_name   Short name for match step # for match #.
             match#_step#_desc   Description for match step # for match #.
           match#_step#_active   If this match step was used for match #.
             match#_step_count   The number of match steps for match #.
                          time   The current time when generating template.
                    time:<fmt>   The time using the given format stringusing strftime formatting (replace % with @ or &).
                      git_hash   The git commit hash for this build of catcierge.
                git_hash_short   The short version of the git commit hash.
                   git_tainted   Was the git working tree changed when building.
                       version   The catcierge version.
                           cwd   Current working directory.
                 template_path   Path to the first template in the list (makes sense if you're only using one).
          template_path:<name>   Get the template with the given name (if multiple templates are used).

Template matcher output variables:
----------------------------------
                   snout_count   Number of snouts given via --snout.
                        snout#   Snout paths given via --snout (1 to snout_count).
                     threshold   Value of --threshold.
                 match_flipped   Value of --match_flipped

Haar matcher output variables:
------------------------------
                       cascade   Cascade XML given via --cascade.
                  in_direction   The in direction left or right, same as --in_direction.
                      min_size   Minimum size of a match in the format WxH. Given by --min_size.
                min_size_width   Minimum width of a match. Given --min_size.
               min_size_height   Minimum height of a match. Given by --min_size.
              no_match_is_fail   Value of --no_match_is_fail.
                  eq_histogram   Value of --eq_histogram.
                   prey_method   Value of --prey_method.
                    prey_steps   Value of --prey_steps.

So once the command gets run catcierge will replace the contents of the variables first and then execute.

This is however not the only way to output these variables. Say you wanted to dump all of them ambove as JSON or XML... It would be quite tedious to pass those as commandline arguments to a script.

So catcierge also supports outputting data to "Templates". That is, a text file containing the same %variable% stuff as for the commands. And from this catcierge generates the final file with the data inserted.

To see the related setting for this you can see here:

Output settings:
  Note that all the *_path variables below can contain variables of the format %var%.
  See --cmdhelp for available variables, and --eventhelp for a list of events.

    --eventhelp                              Show a list of the events that are triggered by catcierge. Templates specified using
                                             --input can filter based on these so that they generate output only for a specific
                                             event. This also shows general help regarding input template settings.
    --save                                   Save match images (both ok and failed).
    --save_obstruct                          Save the image that triggered the "frame obstructed" event.
    --save_steps                             Save each step of the matching algorithm. (--save must also be turned on)
    --input INPUT [INPUT ...]                Path to one or more template files generated on specified events. (Not to be confused
                                             with the template matcher). See --eventhelp for details on generating these on
                                             specific events as well as input template settings.
    --output, --output_path PATH             Path to where the match images and generated templates should be saved.
    --match_output_path PATH                 Override --output_path for match images and save them here instead. This can be
                                             relative to --output_path by using %%output_path%% in the path. Note that this applies
                                             to all paths, other path variables can be used to create a nested structure. Just make
                                             sure you do not add a recursive dependence. Also any other %%vars%% can of course be
                                             used to build the path. See --cmdhelp for details.
    --steps_output_path PATH                 If --save_steps is enabled, save step images to this path. Same as for
                                             --match_output_path, overrides --output_path.
                                             Example: --steps_output_path %%match_output_path%%/steps
    --obstruct_output_path PATH              Path for the obstruct images. Overrides --output_path.
    --template_output_path PATH              Output path for templates (given by --template). Overrides --output_path.

Here --input is used to pass a set of templates... Then catcierge will generate files based on those into %output_path% as specified via --output_path.

Then there are some more granular/advanced control where you can create tree structures for things using --match_output_path and so on.

Here are the default templates that come with catcierge: https://github.com/JoakimSoderberg/catcierge/tree/master/extra/templates

The E-mail script you are now using is designed to use the output from one of these. https://github.com/JoakimSoderberg/catcierge/blob/master/extra/templates/match_group_done_full.json (It is quite long since I hadn't implemented looping in the template language when I wrote that stuff A more concise version with loops: https://github.com/JoakimSoderberg/catcierge/blob/master/extra/templates/event.json )

Anyway the resulting generated file from this template (with all variables filled out) is supposed to be written to the disk and then passed to the e-mail script like this:

--json "%template_path:match_group_done_full%"

Where the part after the : is the name. Which is specified in the template: https://github.com/JoakimSoderberg/catcierge/blob/master/extra/templates/match_group_done_full.json#L2

%!name match_group_done_full

...

But ! from the output you pasted I see you don't have this turned on. So you are getting an error that no such template variable exists (Since a template with that name has not been passed via --input).

[2017-06-10 12:03:42.223665]  Unknown template variable "template_path:match_group_done_full"

Compare that to my output where I have it turned on:

[2017-06-10 04:28:01.936654]  Generate template: /home/catcierge/images/2017-06-10/f4512d1efbb9987a33220e4b168cd4c3d51c8072.json

....

Ah ok I see now this is my fault in the example config: The only template I pass is the new event.json here: https://github.com/JoakimSoderberg/catcierge/blob/master/cmake/catcierge-example.cfg.in#L33

Which has a name of: https://github.com/JoakimSoderberg/catcierge/blob/master/extra/templates/event.json#L2

%!name catcierge_event

But later in the mail script I pass the old name:

--json "%template_path:match_group_done_full%"

so what you need to do is change the above to:

--json "%template_path:catcierge_event%"

And any other occurance where template_path:match_group_done_full is used.

--

Hopefully you did not get too confused by that dump :)))

[robbyleh]

Hi Joakim,

thank you very much for your good explaination.

Yes I had a % character in the passw... But this morning I deliverd the final catflap to my friend Timo and I had no time to implement your helping notes regarding the email problem. I'll do that next time when I have a chance to find the time. ( The Catflap is now based in Timo's home 1km ago from here, so I'm not as flexible for changes as in the recent weeks) ( Timo is also very interested to met you :-) Unfortunatly and foreseeable we didn't get the chance to see a voluntary pass of the cat, but with a little help of some hands it turned out, that the head of his cat was recognized by the software and the solenid opend in two of two cases. :-) The cat didn't pass the open door, she preferd to make a u-turn :-) (till now) But the final "Yeahhh" I don't like to send now, this is only a status update ;-)

Sorry that I can't send pictures now, I'll do that later.

Next open topics on my plan:

• email notification in case of prey detect ( will implement your recent helpfull notes... )
• The backlight: I prepared GPIO2 and the needed elecronic to be able to control the backlight with on or off.

  I ordered a movement sensor and write a script which turns the backlight on when the cat is coming... The light will keep on for about twi minutes...

Will keep you up2date. Robby

[robbyleh]

By the way: the approximal power consumption of the whole flap ( one power supply for all ) is 110 mA on the 220V net. In my opinion this is very low, yeahhh.... The power supply (4,5 Ampere 19.5Volt) is one of an old Vajo Sony Laptop of my doughter. The power supply keeps cold or hand warm after hours... When the movement sensor is working with all of that, the approx power consumption can decrease untill 70mA ! I'm curious ;-) Robby

[JoakimSoderberg]

Oh nice! That is awesome to hear :)

Unfortunatly and foreseeable we didn't get the chance to see a voluntary pass of the cat, but with a little help of some hands it turned out, that the head of his cat was recognized by the software and the solenid opend in two of two cases. :-)

Hehe yes, cats can be quite sceptical about new contraptions... I introduced all this stuff in steps so she would get used to it.

So one tip would be to just Put the cat door fastened in the open state, so the cat gets used to that it can pass through the hole. Then after a while just put it in it's normal position.

---

I haven't measured the power use of mine, but I'm sure its higher since I have my heating element in there that will consume more power at least in the winter :)

Looking forward to more pics and maybe a video of how it all looks!

[robbyleh]

Hello Joakim,

here are some pictures. Please copy the link. Once you got it, I'll delete the link from this discussion here, because ther are some private things in background sometime.

... deleted ...

( The box is not directly under free sky, there is a terrase above. But my friend will also save the wood with some colors for outside (acryl or so ).

I'll send more pics and videos from the final destination later. My iphone was really full on the day I delivered the box. So there are only photos on the camera of my friend.

Robby

[robbyleh]

ahh, and yes we build a blocker in the solenoid, that it keeps open for the first time as you also mentioned..

[JoakimSoderberg]

That looks great! Very tidy and nice compared to mine :) I like the detail with the skull and the LEDs hehe

I meant to physically make the door be in the open state. Like putting a stick that holds it open or soemthing. Just so the cat knows it can walk through the flap. Then maybe put it half closed, and finally fully closed... Then the cat for sure will know it's a door. But then again the cat has been using that other door before, so I guess it'll learn quickly :)

[robbyleh]

thanks for delete... I took a new raspi to build a development stage afterwards . (now the catflap is not here). I ordered a raspicam with it. I installed all without problem on my dev stage, commented out the config statement "no_rpi_cam=1" from the config file to enable a normal raspi cam. But I get this error message.

robby@catflap_dev:~/Katzenklappe/catcierge/doc $ sudo catcierge_grabber --show

Catcierge Grabber v0.6.5-2956e4a (2956e4a-tainted)
(C) Joakim Soderberg 2013-2016

Library versions:
 OpenCV v2.4.9

[2017-06-13 23:53:01.917257]  Reading Raspberry Pi camera config: /etc/catcierge/catcierge-rpi.cfg
[2017-06-13 23:53:01.917812]  Failed to open file /etc/catcierge/catcierge-rpi.cfg
[2017-06-13 23:53:01.918166]  Could not read Raspberry Pi camera config
[2017-06-13 23:53:01.918469]  Using default config
[2017-06-13 23:53:01.918508]  Config path: /etc/catcierge/catcierge.cfg
[2017-06-13 23:53:01.920613]  Parsed config
--------------------------------------------------------------------------------
Settings:
--------------------------------------------------------------------------------
General:
       Startup delay: 0.0 seconds
            Auto ROI: 0
          Show video: 1
        Save matches: 1
       Save obstruct: 0
          Save steps: 0
     Highlight match: 0
       Lockout dummy: 0
      Lockout method: 3
           Lock time: 15 seconds
       Lockout error: 0 (off)
   Lockout err delay: 3.0
       Match timeout: 10 seconds
            Log file: -
            No color: 1
        No animation: 1
   Ok matches needed: 2
         Output path: /home/robby/Katzenklappe/run/images/
   Match output path: %output_path%/%match_group_id%
   Steps output path: %match_output_path%/%matchcur_idx%

        Matcher type: haar
Haar Cascade Matcher:
           Cascade: /home/robby/Katzenklappe/catcierge/extra/catcierge.xml
      In direction: Right
          Min size: 80x80
Equalize histogram: 0
  No match is fail: 1
       Prey method: Adaptive
        Prey steps: 2

RFID:
          Inner RFID: -
          Outer RFID: -
     Lock on no RFID: 0
      RFID lock time: 0.00 seconds
        Allowed RFID: -
--------------------------------------------------------------------------------
[2017-06-13 23:53:01.922537]  Skipping GPIO setup since a custom lockout command is set:[2017-06-13 23:53:01.922563]    �m�[2017-06-13 23:53:01.922581]  Initialized GPIO pins
[2017-06-13 23:53:01.942810]  Initialized catcierge image recognition
Adding user defined variable: 'to_email xxxxx
Adding user defined variable: 'from_email xxxx
Adding user defined variable: 'from_email_passw xxxxxxxxxx
[2017-06-13 23:53:01.943204]  Initialized output templates
[2017-06-13 23:53:01.943273]  Initialized RFID readers
mmal: mmal_vc_port_enable: failed to enable port vc.ril.camera:out:1(I420): ENOSPC
mmal: mmal_port_enable: failed to enable port vc.ril.camera:out:1(I420)(0xd8f080) (ENOSPC)
mmal: camera video callback2 error
mmal: raspiCamCvCreateCameraCaptureEx: Failed to create camera component

^C[2017-06-13 23:53:34.611543]  Received SIGINT, stopping...

Do you have an idea ? When I test the cam with other phyton software it shows me the frames... So cam is working...

Robby

[JoakimSoderberg]

Hmmm well first of all you should not need to run it under sudo.

Does the raspivid command work? Because basically I am sharing code from this one: https://www.raspberrypi.org/documentation/usage/camera/raspicam/raspivid.md

First thing I suspect you already have something else running that might be using the camera?

Also I find this on stack exchange regarding the same issue: https://raspberrypi.stackexchange.com/questions/13764/what-causes-enospc-error-when-using-the-raspberry-pi-camera-module

I had the same problem. A firmware update solved it. sudo rpi-update

One of the answers there was that he had something else running (motion) that used it already.

[robbyleh]

hmm starnge. Yes I had the test phyton script still active. But now I get:

root@catflap_dev:~# catcierge_grabber 

Catcierge Grabber v0.6.5-2956e4a (2956e4a-tainted)
(C) Joakim Soderberg 2013-2016

Library versions:
 OpenCV v2.4.9

[2017-06-14 00:30:47.394958]  Reading Raspberry Pi camera config: /etc/catcierge/catcierge-rpi.cfg
[2017-06-14 00:30:47.395386]  Failed to open file /etc/catcierge/catcierge-rpi.cfg
[2017-06-14 00:30:47.395651]  Could not read Raspberry Pi camera config
[2017-06-14 00:30:47.395883]  Using default config
[2017-06-14 00:30:47.395922]  Config path: /etc/catcierge/catcierge.cfg
[2017-06-14 00:30:47.398050]  Parsed config
--------------------------------------------------------------------------------
Settings:
--------------------------------------------------------------------------------
General:
       Startup delay: 0.0 seconds
            Auto ROI: 0
          Show video: 0
        Save matches: 1
       Save obstruct: 0
          Save steps: 0
     Highlight match: 0
       Lockout dummy: 0
      Lockout method: 3
           Lock time: 15 seconds
       Lockout error: 0 (off)
   Lockout err delay: 3.0
       Match timeout: 10 seconds
            Log file: -
            No color: 1
        No animation: 1
   Ok matches needed: 2
         Output path: /home/robby/Katzenklappe/run/images/
   Match output path: %output_path%/%match_group_id%
   Steps output path: %match_output_path%/%matchcur_idx%

        Matcher type: haar
Haar Cascade Matcher:
           Cascade: /home/robby/Katzenklappe/catcierge/extra/catcierge.xml
      In direction: Right
          Min size: 80x80
Equalize histogram: 0
  No match is fail: 1
       Prey method: Adaptive
        Prey steps: 2

RFID:
          Inner RFID: -
          Outer RFID: -
     Lock on no RFID: 0
      RFID lock time: 0.00 seconds
        Allowed RFID: -
--------------------------------------------------------------------------------
[2017-06-14 00:30:47.400061]  Skipping GPIO setup since a custom lockout command is set:[2017-06-14 00:30:47.400085]    ��i
[2017-06-14 00:30:47.400103]  Initialized GPIO pins
[2017-06-14 00:30:47.418624]  Initialized catcierge image recognition
Adding user defined variable: 'to_email xx
Adding user defined variable: 'from_email xx
Adding user defined variable: 'from_email_passw xx
[2017-06-14 00:30:47.418971]  Initialized output templates
[2017-06-14 00:30:47.419040]  Initialized RFID readers
[2017-06-14 00:30:47.710222]  Failed to setup camera
root@catflap_dev:~# echo $?
255

[JoakimSoderberg]

Ah ok, yea I ran into that bug myself ... I introduced it when adding support for the generic cam stuff you needed.

The good thing is that I can see exactly what version you are using:

Catcierge Grabber v0.6.5-2956e4a (2956e4a-tainted)

Here is the fixed commit (which came after 2956e4a that you are running): https://github.com/JoakimSoderberg/catcierge/commit/1339835723795fc2b613d168c3b3a240a9307944

so do:

cd ~/Katzenklappe/catcierge/build   # Assuming this is hat you named the build dir
git stash    # (If you have some changed files in the repo?)
git pull --rebase origin master    # Get latest version
cmake ..
make
cpack -G DEB
sudo dpkg -i <debfile>  # Where <debfile> is what cpack outputted

Then it should work...

[robbyleh]

Hi Joakim,

I guess I found a good hint. As I told, I deleted "no_rpi_cam=1" from the config file to enable a normal raspi cam. Then I always get "Failed to setup camera". (after closing the other test programs) Now I installed backwards all my compiled versions from catcierge:

root@catflap_dev:/home/robby/Katzenklappe/catcierge/build# ls -lrt *.deb -rw-r--r-- 1 robby robby 1690458 Apr 30 15:55 catcierge-0.6.5-c04a0b9-armhf.deb -rw-r--r-- 1 robby robby 1544036 May 3 09:16 catcierge-0.6.5-64ea8da-armhf.deb -rw-r--r-- 1 robby robby 1545024 May 13 11:30 catcierge-0.6.5-f15cc83-armhf.deb -rw-r--r-- 1 robby robby 1545032 May 13 13:58 catcierge-0.6.5-2956e4a-armhf.deb

In version from 3rd of May I got a second errormessage that the option "no_unlock_after_lockout" is unknown (ok this is foreseeable). The I installed the version from 30th April and the camera is working fine.

I guess it has to do with the change from around 30th April.

No hurry.... Thanks a lot.

Robby

[JoakimSoderberg]

Yea you were on the right track to try another version, because it's in that specific verison that the bug was :) ... But see my previous message for the fix

[robbyleh]

Ohh we are working in parallel, so I didn't realize your previous message ;-) Wow, you are so fast !!! Thanks a lot, I'll try it tomorrow. Have to sleep now :-/

[robbyleh]

It works now !

perfect!

Have a good night .... :-)

Am 14.06.2017 um 00:59 schrieb Joakim Söderberg:

Yea you were on the right track to try another version, because it's in that specific verison that the bug was :) ... But see my previous message for the fix

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/JoakimSoderberg/catcierge/issues/20#issuecomment-308272422, or mute the thread https://github.com/notifications/unsubscribe-auth/Aa6iQN6FlBvFkVtdnmPixrnXdCb332YGks5sDxRJgaJpZM4NM-Sd.

[robbyleh]

Hi Joakim,

How can I influence or set the logfile ? When I start the process I see some optione like this:

... Lockout err delay: 3.0 Match timeout: 10 seconds Log file: - No color: 1 No animation: 1 Ok matches needed: 2 ...

but I cat influence the Logfile Output... Can't find a way to set this file.

Thanks, Robby

[JoakimSoderberg]

That way of logging is deprecated I have been meaning to remove it completely.

Instead since catcierge runs as a systemd service you can read the log from there.

sudo systemctl start catcierge   # Make sure it's started
sudo systemctl status catcierge
sudo journalctl -n 200 -u catcierge  # Look at the latest 200 lines
sudo journalctl -f -u catcierge   # Follow the log

And ofcourse if you want to log some specific event you can use the template output + commands to run on each event.

The systemd service file is found here:

cat /lib/systemd/system/catcierge.service
[Unit]
Description=Catcierge automatic cat door awesomeness
After=catcierge_rpi_gpio.service

[Service]
ExecStart=/usr/local/bin/catcierge_grabber
RemainAfterExit=yes
StandardOutput=journal+console
StandardError=journal+console
User=catcierge
Group=users

[Install]
WantedBy=multi-user.target

I'm no expert at systemd or logging connected to this. But here's a guide: https://www.digitalocean.com/community/tutorials/how-to-use-journalctl-to-view-and-manipulate-systemd-logs

P.S.

I updated the example config today: https://github.com/JoakimSoderberg/catcierge/blob/master/cmake/catcierge-example.cfg.in

I added some clearer explination and fixed the bugs in there (I have not fully tested them yet though).

But that helps you better understand how things fit together with events, cmds and generating files from templates.

[JoakimSoderberg]

Maybe if you describe exactly what you want to do instead?

Get the console output? Then the service log Write each event to a file? Use output templates

This is for example how I save all my matches in a directory structure. Each match will generate it's own dir with its unique ID + json file with the event described.

I then also zip all of this and send that in a mail to myself (acts as a backup as well), as well as attaching an image to the mail, that is a composed image of all these sub images

$ ls -R /home/catcierge/images/2017-06-15/
/home/catcierge/images/2017-06-15/:
ded08bd77ee0e8726ee7164b7f999bd521575159       match_obstruct_2017-06-15_00_09_22.233417.png
ded08bd77ee0e8726ee7164b7f999bd521575159.json

/home/catcierge/images/2017-06-15/ded08bd77ee0e8726ee7164b7f999bd521575159:
1  3  match__2017-06-15_00_09_22.293248__1.png  match__2017-06-15_00_09_22.371186__3.png
2  4  match__2017-06-15_00_09_22.339288__2.png  match__2017-06-15_00_09_22.396789__4.png

/home/catcierge/images/2017-06-15/ded08bd77ee0e8726ee7164b7f999bd521575159/1:
match__2017-06-15_00_09_22.293248__1_00_gray.png      match__2017-06-15_00_09_22.293248__1_02_roi.png
match__2017-06-15_00_09_22.293248__1_01_haar_roi.png  match__2017-06-15_00_09_22.293248__1_03_thresh.png

/home/catcierge/images/2017-06-15/ded08bd77ee0e8726ee7164b7f999bd521575159/2:
match__2017-06-15_00_09_22.339288__2_00_gray.png          match__2017-06-15_00_09_22.339288__2_06_opened.png
match__2017-06-15_00_09_22.339288__2_01_haar_roi.png      match__2017-06-15_00_09_22.339288__2_07_dilated.png
match__2017-06-15_00_09_22.339288__2_02_roi.png           match__2017-06-15_00_09_22.339288__2_08_combined.png
match__2017-06-15_00_09_22.339288__2_03_thresh.png        match__2017-06-15_00_09_22.339288__2_09_contours.png
match__2017-06-15_00_09_22.339288__2_04_adp_thresh.png    match__2017-06-15_00_09_22.339288__2_10_final.png
match__2017-06-15_00_09_22.339288__2_05_inv_combined.png

/home/catcierge/images/2017-06-15/ded08bd77ee0e8726ee7164b7f999bd521575159/3:
match__2017-06-15_00_09_22.371186__3_00_gray.png      match__2017-06-15_00_09_22.371186__3_02_roi.png
match__2017-06-15_00_09_22.371186__3_01_haar_roi.png  match__2017-06-15_00_09_22.371186__3_03_thresh.png

/home/catcierge/images/2017-06-15/ded08bd77ee0e8726ee7164b7f999bd521575159/4:
match__2017-06-15_00_09_22.396789__4_00_gray.png      match__2017-06-15_00_09_22.396789__4_02_roi.png
match__2017-06-15_00_09_22.396789__4_01_haar_roi.png  match__2017-06-15_00_09_22.396789__4_03_thresh.png

I've been thinking of creating a webpage where these events are uploaded, that's why I want each to have its own unique ID like this.

[robbyleh]

Hi Joakim,

the logfile issue is not really important anymore. But thanks for the hints regarding the systemd. I saw also in the net, it is a very powerfull thing...

I still have trouble to set up the email notification. Here the current status.

( I intend to send mail each mach with pray and each match with "No cat head found" ) For a certain time I'll also send mail in a good match "(no prey detect)".

The current output and problem is:

[2017-06-19 21:20:45.276782]  No Match in - No cat head detected (d0a1dafdc278b4cc3c232ce2206d7d4b393312cf)
[2017-06-19 21:20:45.331164]  No Match in - No cat head detected (2a4f19763885147018e8d38b471f2ec3aadd8984)
[2017-06-19 21:20:45.343641]  Lockout! 4 out of 4 matches failed (for 15 seconds).
[2017-06-19 21:20:45.343744]  Wait for lockout timer to finish OR frame to clear (Lockout method 3)
[2017-06-19 21:20:45.343783]  [Matching] -> [Lockout]
[2017-06-19 21:20:45.348408]  Called program "/home/robby/Katzenklappe/tools/lock.sh"
[2017-06-19 21:20:45.348820]  Saving image /home/robby/Katzenklappe/run/images//f6555ad21465484762f2c8cb888ec80e87c9717/match_fail_2017-06-19_21_20_45.141681__1.png
[2017-06-19 21:20:45.363998]    00 Grayscale original                  /home/robby/Katzenklappe/run/images//f6555ad21465484762f2c8cb888ec80e87c9717/1/match_fail_2017-06-19_21_20_45.141681__1_00_gray.png
[2017-06-19 21:20:45.379595]  Saving image /home/robby/Katzenklappe/run/images//f6555ad21465484762f2c8cb888ec80e87c9717/match_fail_2017-06-19_21_20_45.195847__2.png
[2017-06-19 21:20:45.392913]    00 Grayscale original                  /home/robby/Katzenklappe/run/images//f6555ad21465484762f2c8cb888ec80e87c9717/2/match_fail_2017-06-19_21_20_45.195847__2_00_gray.png
[2017-06-19 21:20:45.406230]  Saving image /home/robby/Katzenklappe/run/images//f6555ad21465484762f2c8cb888ec80e87c9717/match_fail_2017-06-19_21_20_45.251391__3.png
[2017-06-19 21:20:45.419713]    00 Grayscale original                  /home/robby/Katzenklappe/run/images//f6555ad21465484762f2c8cb888ec80e87c9717/3/match_fail_2017-06-19_21_20_45.251391__3_00_gray.png
[2017-06-19 21:20:45.432623]  Saving image /home/robby/Katzenklappe/run/images//f6555ad21465484762f2c8cb888ec80e87c9717/match_fail_2017-06-19_21_20_45.305994__4.png
[2017-06-19 21:20:45.445639]    00 Grayscale original                  /home/robby/Katzenklappe/run/images//f6555ad21465484762f2c8cb888ec80e87c9717/4/match_fail_2017-06-19_21_20_45.305994__4_00_gray.png
[2017-06-19 21:20:45.466072]  Generate template: /home/robby/Katzenklappe/run/images//f6555ad21465484762f2c8cb888ec80e87c9717.json
[2017-06-19 21:20:45.473777]  Called program "python /usr/local/share/catcierge/catcierge-compose.py --json "/home/robby/Katzenklappe/run/images/f6555ad21465484762f2c8cb888ec80e87c9717.json" --steps --output "f6555ad21465484762f2c8cb888ec80e87c9717-combined-steps.png""
[2017-06-19 21:20:45.481033]  Called program "python /usr/local/share/catcierge/catcierge-sendmail-new.py --to robby.lehmann@cesp.de --from robby.lehmann@cesp.de --smtp smtp.strato.de:587 --password ack5%RO5 --json "/home/robby/Katzenklappe/run/images/f6555ad21465484762f2c8cb888ec80e87c9717.json" --images f6555ad21465484762f2c8cb888ec80e87c9717-combined-steps.png --extra "/home/robby/Katzenklappe/run/images/f6555ad21465484762f2c8cb888ec80e87c9717.json""
/home/robby/Katzenklappe/tools/lock.py:5: RuntimeWarning: This channel is already in use, continuing anyway.  Use GPIO.setwarnings(False) to disable warnings.
  GPIO.setup(5, GPIO.OUT)
Traceback (most recent call last):
  File "/usr/local/share/catcierge/catcierge-compose.py", line 5, in <module>
    from wand.image import Image
ImportError: No module named wand.image
Json: /home/robby/Katzenklappe/run/images/f6555ad21465484762f2c8cb888ec80e87c9717.json
Traceback (most recent call last):
  File "/usr/local/share/catcierge/catcierge-sendmail-new.py", line 140, in <module>
    if __name__ == '__main__': main()
  File "/usr/local/share/catcierge/catcierge-sendmail-new.py", line 91, in main
    fp = open(f, 'rb')
IOError: [Errno 2] No such file or directory: 'f6555ad21465484762f2c8cb888ec80e87c9717-combined-steps.png'

the current config file is:

noanim=1
nocolor=1
haar=1
cascade=/home/robby/Katzenklappe/catcierge/extra/catcierge.xml
in_direction=right
min_size=80x80
non_rpi_cam=0
no_match_is_fail=1
matchtime=10
lockout=15
lockout_method=3
no_unlock_after_lockout=1
save=0
save_obstruct=0
save_steps=1
do_unlock_cmd=/home/robby/Katzenklappe/tools/unlock.sh
do_lockout_cmd=/home/robby/Katzenklappe/tools/lock.sh
output_path=/home/robby/Katzenklappe/run/images/
match_output_path=%output_path%/%match_group_id%
steps_output_path=%match_output_path%/%matchcur_idx%
input=/usr/local/share/catcierge/templates/event.json
uservar=to_email robby.lehmann@mail.de
uservar=from_email robby.lehmann@mail.de
uservar=from_email_passw xxxx
match_group_done_cmd=python /usr/local/share/catcierge/catcierge-compose.py --json "%template_path:catcierge_event%" --steps --output "%match_group_id%-combined-steps.png"
match_group_done_cmd=python /usr/local/share/catcierge/catcierge-sendmail-new.py --to robby.lehmann@mail.de --from robby.lehmann@cesp.de --smtp smtp.strato.de:587 --password xxxxx --json "%template_path:catcierge_event%" --images %match_group_id%-combined-steps.png --extra "%template_path:catcierge_event%"

Thanks, Robby

[JoakimSoderberg]

Hi,

So you need to read the updated config example first: https://github.com/JoakimSoderberg/catcierge/blob/master/cmake/catcierge-example.cfg.in

You are using the commands as if they run one after another. They do not! They run in parallel, completely independent.

I suggest you start with the simple case. Always send a mail. The example scripts where only designed for this task.

( I intend to send mail each mach with pray and each match with "No cat head found" ) For a certain time I'll also send mail in a good match "(no prey detect)".

Not sure what you mean by this exactly? I mean, you have the 4 matches done each time the cat walks past.... That is called a matchgroup.

Most likely what you want is to trigger a mail when this event is raised (And that is what you are doing when triggering the script on match_group_done_cmd). The entire match group either succeeds or fails.

I suggest you use this example: https://github.com/JoakimSoderberg/catcierge/blob/master/cmake/catcierge-example.cfg.in#L93

But it is important you read all my explanations in that file, especially if you want to customize the behavior. To be clear, these variables are only used by that script.

uservar=to_email robby.lehmann@mail.de
uservar=from_email robby.lehmann@mail.de
uservar=from_email_passw xxxx

My script always sends a mail to me, no matter the type of match. It's not like it will be spamming you.

But if you want to change that behavior you need to modify the actual scripts that are run.

Errors

Anyway, moving on to the errors you are getting:

/home/robby/Katzenklappe/tools/lock.py:5: RuntimeWarning: This channel is already in use, continuing anyway.  Use GPIO.setwarnings(False) to disable warnings.
  GPIO.setup(5, GPIO.OUT)

You should do as this suggests to get rid of that warning :)

But here is your real issue. And I have forgotten

Traceback (most recent call last):
  File "/usr/local/share/catcierge/catcierge-compose.py", line 5, in <module>
    from wand.image import Image
ImportError: No module named wand.image
Json: /home/robby/Katzenklappe/run/images/f6555ad21465484762f2c8cb888ec80e87c9717.json
Traceback (most recent call last):
  File "/usr/local/share/catcierge/catcierge-sendmail-new.py", line 140, in <module>
    if __name__ == '__main__': main()
  File "/usr/local/share/catcierge/catcierge-sendmail-new.py", line 91, in main
    fp = open(f, 'rb')
IOError: [Errno 2] No such file or directory: 'f6555ad21465484762f2c8cb888ec80e87c9717-combined-steps.png'

You are missing a library that the compose script requires:

File "/usr/local/share/catcierge/catcierge-compose.py", line 5, in <module>
    from wand.image import Image
ImportError: No module named wand.image

You can install this Python package:

sudo apt-get install python python-pip libmagickwand-dev
sudo pip install Wand

I realise I never mention this in any documentation

[JoakimSoderberg]

Oh and this script assumes these are set to 1

save=0
save_obstruct=0
save_steps=1

This is what saves the images

[robbyleh]

HI Joakim,

thank you ! After installation of ... as you mentioned above ... it is working fine :-)

The cat will not pass the box after two weeks... she is old, I guess it will take further weeks... But we are looking forward, the cat will do it someday :-)

Robby

[JoakimSoderberg]

Ok good to hear it works :)

Hehe yea hard to get them trying... Maybe put some very nice food it likes at the other side of the door, and then hold it open with something so it gets used to a treat :D

[robbyleh]

Hi Joakim,

Yes my firend works with all these tricks now... I or we guess and hope the cat will do it some day ... I'm currently assemble the things to control the back-light by a motion sensor. Is there any option or way to give catcierge a signal, that it shall start to ignore all motions in the frame and a second signal to tell them it should start to watch for all motions in the frame? My idea which I realized now is, that the backlight is turned off all the time until the motion sensor recognizes a motion 5 meters (120 degree) around the entry of the box. Then the backlight turns on for a certain time ( currently set to 180 seconds ). Problem now: as long the backlight is off, catcierge constantly generates matches with "no cat head found"... ( 10 - 20 times per minute ).

I created a python script which turns the backlight on as soon a motion in the motion sensor is detected. So I would be able to send a signal to catcierge via SIGUSRX = ignoreall and SIGUSRY = watchforall. Other way could be : I touch a file like /tmp/backlight_is_off or /tmp/backlight_is_on and catcierge will check each time if the file /tmp/backlight_is_on is existend or not. If not, it will ignore all motions on the frame.

Is this a difficult thing ?

And how can we clear our depts? Do you like red wine ? I create my own each year, I could send you some bottles ;-)

Thanks, Robby

[JoakimSoderberg]

Hey,

I've been a bit busy now before vacation time + building on another project all evenings.

Is there any option or way to give catcierge a signal, that it shall start to ignore all motions in the frame and a second signal to tell them it should start to watch for all motions in the frame?

...

So I would be able to send a signal to catcierge via SIGUSRX = ignoreall and SIGUSRY = watchforall.

I made a change that should solve your scenario using signals as you propose... It is now instead possible to define the behaviour for the SIGUSR signals:

https://github.com/JoakimSoderberg/catcierge/commit/e02ac835d24f1485e34cf254afed0e6531939883

Signal settings:
  The program can receive signals that can be sent using the kill command.
    SIGUSR1
    SIGUSR2
  These settings can be used to customize what happens when catcierge receives these signals.
  Available behaviors:
    none = Nothing is peformed
    lock = Lock the cat door for lockout time
    unlock = Unlock the cat door
    ignore = Ignores any events, until 'attention'
    attention = Stops ignoring events

    --sigusr1 SIGUSR1                        Set the behavior when the SIGUSR1 signal is received.
    --sigusr2 SIGUSR2                        Set the behavior when the SIGUSR2 signal is received.

So you would pass --sigusr1 ignore and --sigusr2 attention

Then do kill -s SIGUSR2 on the motion and kill -s SIGUSR1 when the motion ends.

I could also enable triggering a command on these events I guess if needed.

I now realise when writing this that it would be useful to have an event that is triggered soon after catcierge has finished starting, so that you can trigger an kill -s SIGUSR1 right away as well...

git pull --rebase origin master
cd build/
cmake ..
make
cpack -G DEB
# You know the drill.....

And how can we clear our depts? Do you like red wine ? I create my own each year, I could send you some bottles ;-)

As you might have noticed with my enthusiasm, just someone building a cat door and using my project is a reward in itself :)

I'm more of a beer drinker myself. Actually Czech and German lagers is kind of my favorites. Was a while since I drank any red wine, but that would probably go down as well ;)

[robbyleh]

HI Joakim,

works perfect. Now when the backlight is off, it ignores all events... Cool :-)

some bottles ... Yes I'm also more of a beer drinker, but few years ago I planted wine plants as sight blocker at the border of our roof garden, so we have a lot of grapes , but it turned out our childrens does not like the grapes because of the cores :-O OMG, I could not persuade my daughters to accept the grapes with the cores :-( So I just tried to make red wine. Noe few years later it turned out I can make it ;- Yeahhh So I'll try to send you some tasty bottles. Will check what are the requirements to send this. Or , are you here by chance in the near of Frankfurt/Main in Germany next time ?

Thank you! Robby

[JoakimSoderberg]

Hello,

I've been on vacation and kept away from the computer some. Forgot to answer your question. No I've never been to Germany before so not very likely I will pass by in the near future either :)

How has the cat door worked, did he get the cat to pass through it yet? :)

/Joakim

[robbyleh]

Hi Joakim, I sent an answer to your last message also on the 11th August via Email Client. I just realize now, that this reply din't reach this forum conversation here ... starnge... Here my last answer to your message:

Am 11.08.2017 um 13:54 schrieb Robby Lehmann:

Hi Joakim,

same for us, we just came back from vacation. My friend Timo ceased to force the cat passing the box, because in the warm summertime the cat does also not come with any prey ...

He decided to force him later when it's geting cold. He think he will pass then box when it is colder than now.

I extended the function wit a motion detection and the backlight control which will be always off when no motion is detected. I wrote also a watchdog which monitors all the processes, tempereatur, GPIOs ( Solenoid should not get power for more than 90 seconds...) , diskspace, wrote also a logfile cleanup script which keeps the disk away to get full...

Currently I examine the problem that the solenoid is closing the the door after one or two seconds some time. Not often but two times from 10 unlocks...

Strange, when it happens then the lock.py was not executed by catcierge... ( I monitored the process list with while true; do ps -ef | grep lock.py; done ) .

Some other process will lock the door or sets the GPIO back to false. I think it is not caused by catcierge.... I'm investigating...

Thanks,

[robbyleh]

and here my newest message:

[robbyleh]

Hi Joakim,

I whish you Marry Christmas! Hoping you are doing well!?!

After few months a little cold season started here. So, Timo and me have got some some news to share with you :-) ( plenty of attempts in the recent months failed. The bright backlight were probably also a problem for the old tomcat )

• arround 15th December : Timo placed the Catflap directly to the cat whole outside the house wall. He kept power off, and the door inside we fixed with tape to remain open. 5 Days later she passed the box first time alone in the night. After that night the tomcat passes the box with a lot of attention, but she does always ...

• arround 18th December: Timo removed the tape from the door inside the box, so the door is closed now, but the power is still off and the solenoid is fixed to unlock, so that the tomcat can open the door with her head...

• today 26th December we activated the power / rasperi / backlight. The solenoid is still mechanicaly fixed to kept open, so all is active except the solenoid is always in unlock position.

Now. after 4 hours the tomcat passed the (almost) active box two times without a problem. The program / electronic and email notiofication shows us, the box is working very good.

We remove the mechanical unlock next time. Will send you an update.

Robby

[visualheroes]

Hey @robbyleh did you finish your project? Greetings, JeF

[robbyleh]

Hi visualheros,

Sorry I didn't see your post till now ... :-O Yes I finished this project. And yes I forgot to close the issue , ohhh....

@JoakimSoderberg : Thank you very much again for your great help that time. I hope you are doing well. :-)

The catflap is still runing in the same setup like the golive in August 2017. Only things what I had to adjust:

• eletrical magnet (50% self made) for the opener didn't generate enough pull power when tempereature went below ~ -2° .
• Backlight was turned on all the time with around 3 Watt on primary side of the PS (220v here). I implemented a moving sensor so that backlight is turning on just for 30 seconds.
• STATUS LED on the Box to see each progress, ( blinking codes )
• Email Reports for unexpected issues ( simple anomaly detection in the logfiles )
• regular status mail reports once a day with temperature filesystem stats e.g.
• frequently houskeeping ( filessystem / logfile cleanup e.g )
• SMTP server enabling encrypted port. @visualheroes let me know if you want me to share more infos. @JoakimSoderberg Thank you again and again. Let me know if you join in near of Frankfurt Germany... ;-) Robby

[robbyleh]

Ahh few points more :

• the powersupply is a Laptop PS from Dell made in 2012. It delivers 19,6V when I am not mistaken. It is inside the house on the floor made of stone and is still running fine. There is already a spare PS ready but it will not break :-D
• In 2020 the cat started to keep his tail inside the door waiting until the bagnet pushs the locker pin in to the door before he finaly went out of the box. We guess the cat could hear the magnet and found out this trick. When he came back the dore was not locked and he came in. We spent 3 months untill we found out what was causing this. Cats are so clever, unbelievable... I fixed this with a frequently unlock every 20 minutes. The locker pin will be pulled just for a second every 20 minutes. Robby

[robbyleh]

[JoakimSoderberg]

Hello Robby,

Thanks for the update! Fun to see this project is still in use. I haven't had my own door active since I got a new cat with a lot less killer instinct

Fun to hear about the tricks the cat came up with. Mine also did some of that after a while, which made me tweak the code. For example retrying multiple times based on the sound the lock made etc

/Joakim

Den tors 9 mars 2023 19:57Robby Lehmann @.***> skrev:

[image: IMG_7070] https://user-images.githubusercontent.com/28222016/224127063-c989f188-0b6c-4acd-8c84-4b31fe1c7919.JPG

— Reply to this email directly, view it on GitHub https://github.com/JoakimSoderberg/catcierge/issues/20#issuecomment-1462602029, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALBFYQ4OLCOJHERX2EXFITW3IR2NANCNFSM4DJT4SOQ . You are receiving this because you were mentioned.Message ID: @.***>