Mkwrapper Stop block: ModuleNotFoundError: No module named 'preview'

[the-snowwhite]

The whole premise in this thread is to see:

A Is it possible to get the qtquickvcp gui's running by adding the missing preview.so lib ? Edit: Answer: allmost

B.Find out what else is missing. Edit Answer at bottom

Based on this suggestion I compiled Emcapp with full python 3 support using this method:

git clone https://github.com/machinekit/emcapplication.git
cd emcapplication
sudo apt install build-essential fakeroot devscripts equivs python3 python3-tk lsb-release apt-cudf
debian/configure machinekit-hal=0.4.21003-1.git39389db84~$(lsb_release -cs) no-docs
mk-build-deps -irs sudo -t 'apt-cudf-get --solver aspcud -o Debug::pkgProblemResolver=0 -o APT::Install-Recommends=0'
cd src
./autogen.sh
./configure --with-hal=machinekit-hal --with-python=python3 --with-boost-python=boost_python-py35
time make -j$(nproc)
sudo make install
source ~/emcapplication/scripts/rip-environment
linuxcnc

Then tested the axis sim config: ran without any problems execpt this message:

1: unspecified exception detail=0x55c7de3db8 ts=0x7facd0b164

---

Mkwrapper:

I used the newest package from machinekit-hal on purpose.

I used this hal config updated for linuxcnc 2.9 (and the newest added soc) I have a similar version that is tested to work with the compiled in axis gui.

Here is the result of a python3 ./run.py run:

machinekit@mksocfpga-fz3-soc:~/Hm2-soc_FDM/Cramps/HAL/FZ3$ ./run.py
starting mklauncher... done
starting configserver... done
starting linuxcnc... LINUXCNC - 2.9.0~pre0
Machine configuration directory is '/home/machinekit/Hm2-soc_FDM/Cramps/HAL/FZ3'
Machine configuration file is 'ox.ini'
Starting LinuxCNC...
rtapi_msgd command:  /usr/libexec/machinekit/rtapi_msgd --instance=0 --rtmsglevel=1 --usrmsglevel=1 --debug=1 --halsize=524288
rtapi_app command:  /usr/libexec/machinekit/rtapi_app --instance=0 --debug=1
3::3910:rt INFO:  Picked default flavor 'posix' automatically
done
Found file(REL): ./ox.hal
emcTrajSetJoints(3) returned 0
emcTrajSetSpindles(1) returned 0
emcTrajSetAxes(3, 7)
emcTrajSetUnits(1.0000, 1.0000)
emcTrajSetVelocity(0.0000, 20.0000) returned 0
emcTrajSetMaxVelocity(200.0000) returned 0
emcTrajSetAcceleration(999999999999999967336168804116691273849533185806555472917961779471295845921727862608739868455469056.0000) returned 0
emcTrajSetMaxAcceleration(999999999999999967336168804116691273849533185806555472917961779471295845921727862608739868455469056.0000)
emcTrajSetHome(0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000) returned 0
emcJointSetType(0, 1)
emcJointSetUnits(0, 1.0000)
emcJointSetBacklash(0, 0.0600) returned 0
emcJointSetMinPositionLimit(0, -0.5000) returned 0
emcJointSetMaxPositionLimit(0, 320.0000) returned 0
emcJointSetFerror(0, 800.0000) returned 0
emcJointSetMinFerror(0, 200.0000) returned 0
emcJointSetHomingParams(0, 0.0000, 0.0000, -1.0000, -10.0000, 1.0000, 0, 1, 0, 1, 0) returned 0
emcJointSetMaxVelocity(0, 50.0000) returned 0
emcJointSetMaxAcceleration(0, 80.0000) returned 0
emcJointActivate(0) returned 0
emcJointSetType(1, 1)
emcJointSetUnits(1, 1.0000)
emcJointSetBacklash(1, 0.0800) returned 0
emcJointSetMinPositionLimit(1, -1.0000) returned 0
emcJointSetMaxPositionLimit(1, 500.1000) returned 0
emcJointSetFerror(1, 800.0000) returned 0
emcJointSetMinFerror(1, 200.0000) returned 0
emcJointSetHomingParams(1, 0.0000, 0.0000, -1.0000, -20.0000, 1.0000, 0, 1, 0, 1, 0) returned 0
emcJointSetMaxVelocity(1, 50.0000) returned 0
emcJointSetMaxAcceleration(1, 80.0000) returned 0
emcJointActivate(1) returned 0
emcJointSetType(2, 1)
emcJointSetUnits(2, 1.0000)
emcJointSetBacklash(2, 0.0200) returned 0
emcJointSetMinPositionLimit(2, -40.2000) returned 0
emcJointSetMaxPositionLimit(2, 0.2000) returned 0
emcJointSetFerror(2, 800.0000) returned 0
emcJointSetMinFerror(2, 200.0000) returned 0
emcJointSetHomingParams(2, 0.0000, 0.2000, -1.0000, 8.0000, -1.0000, 0, 1, 0, 0, 0) returned 0
emcJointSetMaxVelocity(2, 10.0000) returned 0
emcJointSetMaxAcceleration(2, 20.0000) returned 0
emcJointActivate(2) returned 0
emcAxisSetMinPositionLimit(0, -0.5000) returned 0
emcAxisSetMaxPositionLimit(0, 320.0000) returned 0
emcAxisSetMaxVelocity(0, 50.0000) returned 0
emcAxisSetMaxAcceleration(0, 80.0000) returned 0
emcAxisSetLockingJoint(0, -1) returned 0
emcAxisSetMinPositionLimit(1, -1.0000) returned 0
emcAxisSetMaxPositionLimit(1, 500.1000) returned 0
emcAxisSetMaxVelocity(1, 50.0000) returned 0
emcAxisSetMaxAcceleration(1, 90.0000) returned 0
emcAxisSetLockingJoint(1, -1) returned 0
emcAxisSetMinPositionLimit(2, -40.0000) returned 0
emcAxisSetMaxPositionLimit(2, 0.2000) returned 0
emcAxisSetMaxVelocity(2, 10.0000) returned 0
emcAxisSetMaxAcceleration(2, 20.0000) returned 0
emcAxisSetLockingJoint(2, -1) returned 0
hm2: loading Mesa HostMot2 driver version 0.15

hm2_soc_ol: loading Mesa AnyIO HostMot2 socfpga overlay driver version 0.9

hm2/hm2_fz3_.0: Low Level init 0.15

hm2/hm2_fz3_.0: 36 I/O Pins used:

hm2/hm2_fz3_.0:     IO Pin 000 (GPIO0.P0-01): IOPort

hm2/hm2_fz3_.0:     IO Pin 001 (GPIO0.P0-02): IOPort

hm2/hm2_fz3_.0:     IO Pin 002 (GPIO0.P0-03): IOPort

hm2/hm2_fz3_.0:     IO Pin 003 (GPIO0.P0-04): IOPort

hm2/hm2_fz3_.0:     IO Pin 004 (GPIO0.P0-05): StepGen #0, pin Direction (Output)

hm2/hm2_fz3_.0:     IO Pin 005 (GPIO0.P0-06): StepGen #0, pin Step (Output)

hm2/hm2_fz3_.0:     IO Pin 006 (GPIO0.P0-07): StepGen #1, pin Direction (Output)

hm2/hm2_fz3_.0:     IO Pin 007 (GPIO0.P0-08): StepGen #1, pin Step (Output)

hm2/hm2_fz3_.0:     IO Pin 008 (GPIO0.P0-09): StepGen #2, pin Direction (Output)

hm2/hm2_fz3_.0:     IO Pin 009 (GPIO0.P0-10): StepGen #2, pin Step (Output)

hm2/hm2_fz3_.0:     IO Pin 010 (GPIO0.P0-11): StepGen #3, pin Direction (Output)

hm2/hm2_fz3_.0:     IO Pin 011 (GPIO0.P0-12): StepGen #3, pin Step (Output)

hm2/hm2_fz3_.0:     IO Pin 012 (GPIO0.P0-13): Hostmot2 DPLL #0, pin Ref Out Pin (Output)

hm2/hm2_fz3_.0:     IO Pin 013 (GPIO0.P0-14): Encoder #0, pin A (Input)

hm2/hm2_fz3_.0:     IO Pin 014 (GPIO0.P0-15): Encoder #0, pin B (Input)

hm2/hm2_fz3_.0:     IO Pin 015 (GPIO0.P0-16): Encoder #0, pin Index (Input)

hm2/hm2_fz3_.0:     IO Pin 016 (GPIO0.P0-17): IOPort

hm2/hm2_fz3_.0:     IO Pin 017 (GPIO0.P0-18): PWMGen #0, pin Out0 (PWM or Up) (Output)

hm2/hm2_fz3_.0:     IO Pin 018 (GPIO0.P1-19): IOPort

hm2/hm2_fz3_.0:     IO Pin 019 (GPIO0.P1-20): IOPort

hm2/hm2_fz3_.0:     IO Pin 020 (GPIO0.P1-21): IOPort

/usr/lib/python3/dist-packages/pyftpdlib/authorizers.py:244: RuntimeWarning: write permissions assigned to anonymous user.
  RuntimeWarning)
Process Process-1:
Traceback (most recent call last):
  File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/bin/mkwrapper", line 322, in run
    import preview  # must be imported in new process to work properly
ModuleNotFoundError: No module named 'preview'
send_pbcontainer: type: MT_PING

send_pbcontainer: type: MT_PING

task: main loop took 5.014869 seconds
send_pbcontainer: type: MT_PING

send_pbcontainer: type: MT_PING

send_pbcontainer: type: MT_PING

^Cstopping mklauncher... done
stopping configserver... done
stopping linuxcnc... done
machinekit@mksocfpga-fz3-soc:~/Hm2-soc_FDM/Cramps/HAL/FZ3$ task: main loop took 5.003195 seconds
task: 27 cycles, min=0.000181, max=5.014869, avg=0.375194, 2 latency excursions (> 10x expected cycle time of 0.010000s)

Seems like config is parsed ok however mkwrapper cannot fínd the module preview.

Traceback (most recent call last):
  File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/bin/mkwrapper", line 322, in run
    import preview  # must be imported in new process to work properly
ModuleNotFoundError: No module named 'preview'

I wonder where this python preview module requested here is supposed to come from.

BTW With these changes to the linuxcnc script Im able to run a similar python based hal config however the end result is the same error traceback

[the-snowwhite]

I had some similar module not found issues with my python configs in the fdm folder I got module errors in the base.py file

before i changed the lines from:

import rcomps
import storage
import motion

to:

from fdm.config import rcomps
from fdm.config import storage
from fdm.config import motion

Like python 3.7 handles local paths different ?

[the-snowwhite]

This is not a python 2/3 issue: I revived my Emcapp python2 setup with this patch specified here and the problem persists: ( I even tried copying over an old mkwrapper from my running machinekit-cnc ox-router setup, same result)

/usr/lib/python2.7/dist-packages/pyftpdlib/authorizers.py:244: RuntimeWarning: write permissions assigned to anonymous user.
RuntimeWarning)
Process Process-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 267, in _bootstrap
    self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
File "/usr/bin/mkwrapper", line 322, in run
    import preview  # must be imported in new process to work properly
ImportError: No module named preview

@cerna Can this have something to do with how the python include/search paths are setup in Linuxcnc differing from Machinekit-cnc ?

[the-snowwhite]

Poking around in my old ob-ox setup I can find out where the preview comes from there:

machinekit@mksocfpga-ob-ox:~$ python
Python 2.7.13 (default, Aug 22 2020, 10:03:02) 
[GCC 6.3.0 20170516] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import preview
>>> preview.__file__
'/usr/lib/python2.7/dist-packages/preview.so'
>>> exit()
preview: socket shutdown
machinekit@mksocfpga-ob-ox:~$ cat /etc/*-release
PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
NAME="Debian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
VERSION_CODENAME=stretch
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

And find out that its missing in my newest setup:

machinekit@mksocfpga-ultra-soc:~/Hm2-soc_FDM/Cramps/HAL/ULTRA$ python
Python 2.7.16 (default, Oct 10 2019, 22:02:15) 
[GCC 8.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import preview
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named preview
>>> exit()
machinekit@mksocfpga-ultra-soc:~/Hm2-soc_FDM/Cramps/HAL/ULTRA$ cat /etc/*-release
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

However struggling to find out where the: '/usr/lib/python2.7/dist-packages/preview.so' file is from

[the-snowwhite]

Well I can find it in the (from machinekit.io) machinekit-cnc-rt-preempt_0.1.1553365560.git3686a54-1_stretch_armhf.deb Still struggling to find out how its compiled

[the-snowwhite]

The preview module is compiled in from: https://github.com/machinekit/machinekit-cnc/blob/master/src/emc/rs274ngc/Submakefile#L77-L87

[the-snowwhite]

The whole compile blob shows it to depend on haltalk, czmq and the protobuff stuff:

PREVIEWMODULESRCS := \
    emc/rs274ngc/previewmodule.cc
PYSRCS += $(PREVIEWMODULESRCS)

$(call TOOBJSDEPS, $(PREVIEWMODULESRCS)) : Makefile.inc
$(call TOOBJSDEPS, $(PREVIEWMODULESRCS)) : EXTRAFLAGS += -Ihal/haltalk $(CZMQ_CFLAGS) $(PROTOBUF_CFLAGS)

PREVIEWMODULE := ../lib/python/preview.so
$(PREVIEWMODULE): $(call TOOBJS, $(PREVIEWMODULESRCS)) \
    ../lib/librs274.so.0 \
    ../lib/libmachinetalk-pb2++.so.0 \
    ../lib/libmtalk.so
    $(ECHO) Linking python module $(notdir $@)
    $(Q)$(CXX) $(LDFLAGS) -shared -o $@ $^ -lstdc++  $(CZMQ_LIBS) \
        $(PROTOBUF_LIBS)

PYTARGETS += $(PREVIEWMODULE)

@cerna I'm not sure if the QtQuickvcp repo would be the natural place to compile this code in from. Comes to mind a more natural place would be a haltalk repo based on this folder.

[the-snowwhite]

I'm able to get the preview.so to compile in machinekit-hal however ~I'm unable to get past this issue:~

ImportError: /usr/lib/librs274.so.0: undefined symbol: _Z18GET_EXTERNAL_SPEEDi

/usr/lib/python2.7/dist-packages/pyftpdlib/authorizers.py:244: RuntimeWarning: write permissions assigned to anonymous user.
  RuntimeWarning)
Process Process-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 267, in _bootstrap
    self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/bin/mkwrapper", line 322, in run
    import preview  # must be imported in new process to work properly
ImportError: /usr/lib/librs274.so.0: undefined symbol: _Z18GET_EXTERNAL_SPEEDi
^Cstopping mklauncher... done
stopping configserver... done
stopping linuxcnc... task: 3456 cycles, min=0.000134, max=0.010570, avg=0.010017, 0 latency excursions (> 10x expected cycle time of 0.010000s)
exiting HAL component storage
done

$ c++filt _Z18GET_EXTERNAL_SPEEDi
GET_EXTERNAL_SPEED(int)

[the-snowwhite]

Turns out I needed to use the files from the Emc app src/emc folder instead of the ones from mk-cnc and update previewmodule.cc via looking at the gcodemodule.cc file from emc app. The "patches" are currently in 2 separate branches: arm64 armhf These patches are for preview only and intended as somthing to copy and paste from (ie. previewmodule.cc)

[the-snowwhite]

Results running in machinekit-client service view:

There are 4 services missing compared to a correctly running instance:

HAL Rcommand service
HAL Rcomp service
HAL Group service
Log service

[the-snowwhite]

running the gui (MachinekitClient) gives following execption:

Exception in thread process_sockets:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/bin/mkwrapper", line 678, in process_sockets
    self.update_status(self.stat)
  File "/usr/bin/mkwrapper", line 1758, in update_status
    self.update_motion(stat)
  File "/usr/bin/mkwrapper", line 1510, in update_motion
    modified |= self.update_motion_value(name, getattr(stat, name))
AttributeError: 'linuxcnc.stat' object has no attribute 'spindle_brake'

[the-snowwhite]

Looks like mkwrapper needs to be updated to the linuxcnc 2.8 , 2.9 changes: axis --> joints motion.spindle.xx --> spindle..xx

[the-snowwhite]

This exception seems to be caused by mkwrapper not being updated to the new lcnc axis --> joints update in 2.8 , 2.9. A way to see the changes needed seem to be to compare emctaskmain.cc from machinekit-cnc to emcapp

---

Also same update seems to be needed to the protobuf stuff here

[the-snowwhite]

By editing mkwrapper.py and status.proto I'm finally able to get the Cetus gui to start up execpt for now all the spindle stuff is commented out. I'm able to get my machine to home however when I try to jog I get this:

/usr/lib/python2.7/dist-packages/pyftpdlib/authorizers.py:244: RuntimeWarning: write permissions assigned to anonymous user.
  RuntimeWarning)
ERROR: uncaught exception integer argument expected, got float
ERROR: uncaught exception function takes at least 3 arguments (2 given)

[the-snowwhite]

Got my machine to jog with standard Cetus Gui Turns out the jog functions in mkwrapper needed an extra jogmode parameter. Now I get this in Cetus: (after some homing and jogging)

must be in joint mode to home

[cerna]

@the-snowwhite,

Sorry I was unaccounted for the party, had to prioritize my engagements.

Can this have something to do with how the python include/search paths are setup in Linuxcnc differing from Machinekit-cnc ?

My first reaction is to probably delegate it to its own repository. I think it is a better solution to have multiple smallish project with limited functionality than one big doing-it-all. (Mainly because of the dependency graph and maintenance issues stemming from it.)

Can this have something to do with how the python include/search paths are setup in Linuxcnc differing from Machinekit-cnc ?

I am not sure, but I don't think so.(?) All Python packages for Machinekit projects and for LinuxCNC (as far as I know) are comming from apt Debian repository, which installs it in one place. There is no use of _Virtual Environment_s or Pyenv, because combining it with Debian packaging is extremely painful. (And it would make more sense to try something like Nix anyway.)

There could be a dichotomy between Python2.x and Python3 version of packages.

---

All in all, it's great that you were able to reach the bottom of it! :+1:

However struggling to find out where the: '/usr/lib/python2.7/dist-packages/preview.so' file is from

My personal opinion on this issue is that all compiled files (here shared libraries) should have a prefix identifying it as a Machinekit file (mkh_, mkemca_, mkcnc_ and so on) and maybe some binary header.

[summerlotus513]

Got my machine to jog with standard Cetus Gui Turns out the jog functions in mkwrapper needed an extra jogmode parameter. Now I get this in Cetus: (after some homing and jogging)

must be in joint mode to home

This is not a python 2/3 issue: I revived my Emcapp python2 setup with this patch specified here and the problem persists: ( I even tried copying over an old mkwrapper from my running machinekit-cnc ox-router setup, same result)

/usr/lib/python2.7/dist-packages/pyftpdlib/authorizers.py:244: RuntimeWarning: write permissions assigned to anonymous user.
RuntimeWarning)
Process Process-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 267, in _bootstrap
    self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
File "/usr/bin/mkwrapper", line 322, in run
    import preview  # must be imported in new process to work properly
ImportError: No module named preview

@cerna Can this have something to do with how the python include/search paths are setup in Linuxcnc differing from Machinekit-cnc ?

Hello, I also encountered the same problem, how did you solve it?

[the-snowwhite]

@summerlotus513 Try looking at my latest attempts to fix the mkwrapper qtquickvcp stuff here: https://github.com/the-snowwhite/machinekit-hal/commits/mkwrapper_proto_work-py3 and here: https://github.com/the-snowwhite/EMCApplication/commits/python3