Build errors

[limitless-code]

Build Environment

• Raspberry pi 4 2 GB

• Ubuntu 20.04.2 LTS

• gcc, gcc++

• subversion

• qttools5-dev

• qttools5-dev-tools

• python

• git clone latest copy of OpenVoiceOS as per README

warning occurs when running make rpi4_64-gui-config: -

config rpi4_64-gui
make -C buildroot BR2_EXTERNAL=../buildroot-external "rpi4_64-gui_defconfig"
make[1]: Entering directory '/home/ubuntu/OpenVoiceOS/buildroot'

WARNING: unmet direct dependencies detected for BR2_PACKAGE_PYTHON_PROTOBUF
  Depends on [n]: (BR2_PACKAGE_PYTHON [=n] || BR2_PACKAGE_PYTHON3 [=y]) && (BR2_HOSTARCH [=aarch64]=x86_64 || BR2_HOSTARCH [=aarch64]=x86)
  Selected by [y]:
  - BR2_PACKAGE_PYTHON_PYCHROMECAST [=y]

WARNING: unmet direct dependencies detected for BR2_PACKAGE_PYTHON_PROTOBUF
  Depends on [n]: (BR2_PACKAGE_PYTHON [=n] || BR2_PACKAGE_PYTHON3 [=y]) && (BR2_HOSTARCH [=aarch64]=x86_64 || BR2_HOSTARCH [=aarch64]=x86)
  Selected by [y]:
  - BR2_PACKAGE_PYTHON_PYCHROMECAST [=y]
#
# configuration written to /home/ubuntu/OpenVoiceOS/buildroot/.config
#
make[1]: Leaving directory '/home/ubuntu/OpenVoiceOS/buildroot'

Then after running make rpi4_64-gui the following error occurs: -

/home/ubuntu/OpenVoiceOS/buildroot/output/host/bin/ccache /usr/bin/g++ -fno-PIE -c   -O2 -I/home/ubuntu/OpenVoiceOS/buildroot/output/host/include -DIN_GCC  -DCROSS_DIRECTORY_STRUCTURE   -fno-exceptions -fno-rtti -fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wmissing-format-attribute -Woverloaded-virtual -pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings   -DHAVE_CONFIG_H -I. -I. -I../../gcc -I../../gcc/. -I../../gcc/../include -I../../gcc/../libcpp/include -I/home/ubuntu/OpenVoiceOS/buildroot/output/host/include -I/home/ubuntu/OpenVoiceOS/buildroot/output/host/include -I/home/ubuntu/OpenVoiceOS/buildroot/output/host/include  -I../../gcc/../libdecnumber -I../../gcc/../libdecnumber/dpd -I../libdecnumber -I../../gcc/../libbacktrace   -o caller-save.o -MT caller-save.o -MMD -MP -MF ./.deps/caller-save.TPo ../../gcc/caller-save.c
/home/ubuntu/OpenVoiceOS/buildroot/output/host/bin/ccache /usr/bin/g++ -fno-PIE -c   -O2 -I/home/ubuntu/OpenVoiceOS/buildroot/output/host/include -DIN_GCC  -DCROSS_DIRECTORY_STRUCTURE   -fno-exceptions -fno-rtti -fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wmissing-format-attribute -Woverloaded-virtual -pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings   -DHAVE_CONFIG_H -I. -I. -I../../gcc -I../../gcc/. -I../../gcc/../include -I../../gcc/../libcpp/include -I/home/ubuntu/OpenVoiceOS/buildroot/output/host/include -I/home/ubuntu/OpenVoiceOS/buildroot/output/host/include -I/home/ubuntu/OpenVoiceOS/buildroot/output/host/include  -I../../gcc/../libdecnumber -I../../gcc/../libdecnumber/dpd -I../libdecnumber -I../../gcc/../libbacktrace   -o calls.o -MT calls.o -MMD -MP -MF ./.deps/calls.TPo ../../gcc/calls.c
g++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
make[4]: *** [Makefile:1117: insn-emit.o] Error 1
make[4]: *** Waiting for unfinished jobs....
rm gcc.pod
make[3]: *** [Makefile:4376: all-gcc] Error 2
make[2]: *** [package/pkg-generic.mk:250: /home/ubuntu/OpenVoiceOS/buildroot/output/build/host-gcc-initial-10.2.0/.stamp_built] Error 2
make[1]: *** [Makefile:84: _all] Error 2
make[1]: Leaving directory '/home/ubuntu/OpenVoiceOS/buildroot'

Not sure what dependency i am missing re: warning? Or if I am doing anything obviously wrong?

Thank you!

[ChanceNCounter]

Where'd you find a link? I need to go tell people not to build this for a while.

[ChanceNCounter]

Slightly longer version: wait a few days and we'll provide an image.

[limitless-code]

Where'd you find a link? I need to go tell people not to build this for a while.

OK thanks, I will hold fire.

I followed the instructions in the README of this repo. This link git clone --recurse-submodules https://github.com/OpenVoiceOS/OpenVoiceOS.git.

[ChanceNCounter]

Oh, absolutely. I meant, who linked you to the repo? =P I thought the main place people were finding us was the corresponding thread at the Mycroft forum, where we periodically mention that it's Almost There. But if there's someplace else, where we've left things looking like it should work, I definitely need to go warn those people!

[ChanceNCounter]

Now that I've pinned an issue for everyone else, I can take a bit more time to expand in here ;)

The gist is, the image is building nicely for the 2/5 of our developers who build it. Off the top of my head, I can't remember what distro/s they're running, but I'll ask.

Getting it to run on other distros means, as you discovered, going through dep hell on each of those systems. It's (usually) a one-time problem, apart from individual breakages on individual system updates down the road. But it's a time-consuming problem, and, at this prealpha moment, the manifest is subject to sudden change.

Hence, we might spend a few hours fighting through a chunk of dep hell that won't exist tomorrow, or we might tack on another whole dep tree by this time next week. It's not that we don't wanna help you guys, it's that holy shit could we (and you) end up spending extra time on Schroedinger's Dephell.

[limitless-code]

Many thanks for taking the time to explain. I found myself here through curiosity after seeing OpenVoiceOS/MycroftOS mentioned on a few YouTube videos. I did a search for OpenVoiceOS (not google but Qwant) and that resulted in me ending up here. I understand the dephell issue so no pressure from me. Looks like a fantastic project even in preAlpha from the YouTube videos I have seen. I look forward to testing any releases.

[ChanceNCounter]

Ah! I forgot about the shout-out, didn't think about search engines. Makes sense.

For what it's worth, currently in numbered pre/alpha are OVOS Skill Manager and several of the skills in our Skills Store (available via OSM). OSM is standalone, and should work on any Mycroft build or device (testers sought!) and does not conflict with its first-party counterpart.

It offers the same functionality as MSM, the skill manager that comes with Mycroft, but it also provides access to several repositories that aren't supported by MycroftAI, Inc. (nor, apart from the OVOS Skill Store, by us.)

This includes the repos for other Mycroft-based projects, Pling, and a use-at-your-even-more-than-usual-peril interface for Andlo's Skill List, which the eponymous community member produced by crawling GitHub for everything that looks like a Mycroft skill.

[ccoreilly]

I could successfully build OVOS in gitlab CI using a builder docker image. You can try building it locally from within the docker container, it contains all necessary dependencies.

[NeonDaniel]

Where'd you find a link? I need to go tell people not to build this for a while.

OK thanks, I will hold fire.

I followed the instructions in the README of this repo. This link git clone --recurse-submodules https://github.com/OpenVoiceOS/OpenVoiceOS.git.

If you still want to do this and don't mind a little troubleshooting, my suspicion is a system dependency j1nx and I forgot to document because we already had it installed.. You might try installing the Mycroft apt dependencies and re-running.

https://github.com/MycroftAI/mycroft-core/blob/170ebc44ecf5cf3ceeaf44f282c992754b1bbc1b/dev_setup.sh#L342-L346

Also, want to make sure you ran ./scripts/br-patches.sh per the setup instructions.

[limitless-code]

Many thanks for the suggestions. Will try this over the weekend as I am still getting the same error with the docker container suggestion using builder.Dockerfile. All suggestions are appreciated regardless of outcome. :)

Get Outlook for Androidhttps://aka.ms/AAb9ysg

---

From: Daniel McKnight @.> Sent: Thursday, August 5, 2021 8:26:59 PM To: OpenVoiceOS/OpenVoiceOS @.> Cc: limitless-code @.>; Author @.> Subject: Re: [OpenVoiceOS/OpenVoiceOS] Build errors (#58)

Where'd you find a link? I need to go tell people not to build this for a while.

OK thanks, I will hold fire.

I followed the instructions in the README of this repo. This link git clone --recurse-submodules https://github.com/OpenVoiceOS/OpenVoiceOS.git.

If you still want to do this and don't mind a little troubleshooting, my suspicion is a system dependency j1nx and I forgot to document because we already had it installed.. You might try installing the Mycroft apt dependencies and re-running.

https://github.com/MycroftAI/mycroft-core/blob/170ebc44ecf5cf3ceeaf44f282c992754b1bbc1b/dev_setup.sh#L342-L346

Also, want to make sure you ran ./scripts/br-patches.sh per the setup instructions.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/OpenVoiceOS/OpenVoiceOS/issues/58#issuecomment-893725768, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ALX6KGMFPJSPODD6UBPQXQTT3LQYHANCNFSM5BLIMVHA. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email.

[limitless-code]

oh my giddy I/O! finally got past the first dependency warning (not managed to compile it fully so can't test) and found the reason for the other error.

/home/ovos/OpenVoiceOS/buildroot/output/host/bin/ccache /usr/bin/g++ -fno-PIE -c   -O2 -I/home/ovos/OpenVoiceOS/buildroot/output/host/include -DIN_GCC  -DCROSS_DIRECTORY_STRUCTURE   -fno-exceptions -fno-rtti -fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wmissing-format-attribute -Woverloaded-virtual -pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings   -DHAVE_CONFIG_H -I. -I. -I../../gcc -I../../gcc/. -I../../gcc/../include -I../../gcc/../libcpp/include -I/home/ovos/OpenVoiceOS/buildroot/output/host/include -I/home/ovos/OpenVoiceOS/buildroot/output/host/include -I/home/ovos/OpenVoiceOS/buildroot/output/host/include  -I../../gcc/../libdecnumber -I../../gcc/../libdecnumber/dpd -I../libdecnumber -I../../gcc/../libbacktrace   -o bb-reorder.o -MT bb-reorder.o -MMD -MP -MF ./.deps/bb-reorder.TPo ../../gcc/bb-reorder.c
`g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
Makefile:1117: recipe for target 'insn-emit.o' failed
make[3]: *** [insn-emit.o] Error 4
make[3]: *** Waiting for unfinished jobs....
rm gcc.pod
make[3]: Leaving directory '/home/ovos/OpenVoiceOS/buildroot/output/build/host-gcc-initial-10.2.0/build/gcc'
Makefile:4376: recipe for target 'all-gcc' failed
make[2]: *** [all-gcc] Error 2
make[2]: Leaving directory '/home/ovos/OpenVoiceOS/buildroot/output/build/host-gcc-initial-10.2.0/build'
package/pkg-generic.mk:247: recipe for target '/home/ovos/OpenVoiceOS/buildroot/output/build/host-gcc-initial-10.2.0/.stamp_built' failed
make[1]: *** [/home/ovos/OpenVoiceOS/buildroot/output/build/host-gcc-initial-10.2.0/.stamp_built] Error 2
make[1]: Leaving directory '/home/ovos/OpenVoiceOS/buildroot'
cp -f buildroot/output/images/sdcard.img release/OpenVoiceOS_rpi4_64-gui.img
cp: cannot stat 'buildroot/output/images/sdcard.img': No such file or directory
Makefile:24: recipe for target 'rpi4_64-gui' failed
make: *** [rpi4_64-gui] Error 1

in dmesg I find what looks to be the cause the problem!

[ 9065.579559] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/,task=cc1plus,pid=262417,uid=1001
[ 9065.579601] Out of memory: Killed process 262417 (cc1plus) total-vm:588296kB, anon-rss:531756kB, file-rss:0kB, shmem-rss:0kB, UID:1001 pgtables:1152kB oom_score_adj:0
[ 9065.668545] oom_reaper: reaped process 262417 (cc1plus), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB

any ideas as to how much memory I should have to compile/ make?

as i said I am running the compilation/ make on a Raspberry pi 4 2 GB. I can't upgrade the memory on this, is there a way to limit what "cc1plus" uses so it doesn't hang the machine?

EDIT: I think something like make -j 4 or ``make -jnproc` may work. Currently trying and will update here.

or increasing swap size may help? or some other issue causing it in the make process? Not sure if anyone has running the compilation on an RPI 4 before?

Thank you.

[NeonDaniel]

oh my giddy I/O! finally got past the first dependency warning (not managed to compile it fully so can't test) and found the reason for the other error.

/home/ovos/OpenVoiceOS/buildroot/output/host/bin/ccache /usr/bin/g++ -fno-PIE -c   -O2 -I/home/ovos/OpenVoiceOS/buildroot/output/host/include -DIN_GCC  -DCROSS_DIRECTORY_STRUCTURE   -fno-exceptions -fno-rtti -fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wmissing-format-attribute -Woverloaded-virtual -pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings   -DHAVE_CONFIG_H -I. -I. -I../../gcc -I../../gcc/. -I../../gcc/../include -I../../gcc/../libcpp/include -I/home/ovos/OpenVoiceOS/buildroot/output/host/include -I/home/ovos/OpenVoiceOS/buildroot/output/host/include -I/home/ovos/OpenVoiceOS/buildroot/output/host/include  -I../../gcc/../libdecnumber -I../../gcc/../libdecnumber/dpd -I../libdecnumber -I../../gcc/../libbacktrace   -o bb-reorder.o -MT bb-reorder.o -MMD -MP -MF ./.deps/bb-reorder.TPo ../../gcc/bb-reorder.c
`g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
Makefile:1117: recipe for target 'insn-emit.o' failed
make[3]: *** [insn-emit.o] Error 4
make[3]: *** Waiting for unfinished jobs....
rm gcc.pod
make[3]: Leaving directory '/home/ovos/OpenVoiceOS/buildroot/output/build/host-gcc-initial-10.2.0/build/gcc'
Makefile:4376: recipe for target 'all-gcc' failed
make[2]: *** [all-gcc] Error 2
make[2]: Leaving directory '/home/ovos/OpenVoiceOS/buildroot/output/build/host-gcc-initial-10.2.0/build'
package/pkg-generic.mk:247: recipe for target '/home/ovos/OpenVoiceOS/buildroot/output/build/host-gcc-initial-10.2.0/.stamp_built' failed
make[1]: *** [/home/ovos/OpenVoiceOS/buildroot/output/build/host-gcc-initial-10.2.0/.stamp_built] Error 2
make[1]: Leaving directory '/home/ovos/OpenVoiceOS/buildroot'
cp -f buildroot/output/images/sdcard.img release/OpenVoiceOS_rpi4_64-gui.img
cp: cannot stat 'buildroot/output/images/sdcard.img': No such file or directory
Makefile:24: recipe for target 'rpi4_64-gui' failed
make: *** [rpi4_64-gui] Error 1

in dmesg I find what looks to be the cause the problem!

[ 9065.579559] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/,task=cc1plus,pid=262417,uid=1001
[ 9065.579601] Out of memory: Killed process 262417 (cc1plus) total-vm:588296kB, anon-rss:531756kB, file-rss:0kB, shmem-rss:0kB, UID:1001 pgtables:1152kB oom_score_adj:0
[ 9065.668545] oom_reaper: reaped process 262417 (cc1plus), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB

any ideas as to how much memory I should have to compile/ make?

as i said I am running the compilation/ make on a Raspberry pi 4 2 GB. I can't upgrade the memory on this, is there a way to limit what "cc1plus" uses so it doesn't hang the machine?

EDIT: I think something like make -j 4 or ``make -jnproc` may work. Currently trying and will update here.

or increasing swap size may help? or some other issue causing it in the make process? Not sure if anyone has running the compilation on an RPI 4 before?

Thank you.

Not sure about running on a Pi; increasing swap might fix OOM errors, though it will take several days to build (for reference I think it took about 2 days in a VM with 4 Zen2 cores and 16GB RAM allocated. Also, make sure you have about 70GB available disk space where you run this (68GB used as of 4/28/2021). (Not sure if this all made it into docs)

[limitless-code]

Thanks. That's the info I was after, I have made some progress regarding some of my issues. The OOM errors are most probably a red herring as i ditched my rp4 2GB version for my rpi4 4Gb version but usage hardly went over 1GB RAM and found that my docker container which I am running in interactive "bash" mode to troubleshoot is running on an attached USB SSD not the SD card(rpi4 4GB version). This is now seeming performing fine. Conclusion: running the build process on an SD card (rpi4 2GB version) caused the jobs(etc) to queue up in physical memory as SD cards are too slow to write/read to and eventually caused the host kernel to sacrifice the docker container I was using to build the image (just a theory!).

Edit:: My SD card is too slow/ bad and therefore caused the above . I tried a new one and it is working fine and fast.

That being said and getting further.

After running _make rpi464-gui-config I am still getting:

WARNING: unmet direct dependencies detected for BR2_PACKAGE_PYTHON_PROTOBUF
  Depends on [n]: (BR2_PACKAGE_PYTHON [=n] || BR2_PACKAGE_PYTHON3 [=y]) && (BR2_HOSTARCH [=aarch64]=x86_64 || BR2_HOSTARCH [=aarch64]=x86)
  Selected by [y]:
  - BR2_PACKAGE_PYTHON_PYCHROMECAST [=y]

WARNING: unmet direct dependencies detected for BR2_PACKAGE_PYTHON_PROTOBUF
  Depends on [n]: (BR2_PACKAGE_PYTHON [=n] || BR2_PACKAGE_PYTHON3 [=y]) && (BR2_HOSTARCH [=aarch64]=x86_64 || BR2_HOSTARCH [=aarch64]=x86)
  Selected by [y]:
  - BR2_PACKAGE_PYTHON_PYCHROMECAST [=y]

I cannot seem to fix this no matter what python packages I install including the suggested packages from dev_setup.sh

I tried just ignoring this warning and ran make _rpi4_64-gui_

but after ~7 hrs it runs through some python tests and fails 2 tests, the codec test and another one (I'll edit this comment with more detail after it has finished another build attempt).

then it exits ~1hr later with the following:

/home/ovos/OpenVoiceOS/buildroot/output/build/host-python-2.7.18/Modules/_ctypes/libffi/src/arm/sysv.S:410: Error: undefined symbol FFI_TYPE_STRUCT_VFP_FLOAT
 used as an immediate value
/home/ovos/OpenVoiceOS/buildroot/output/build/host-python-2.7.18/Modules/_ctypes/libffi/src/arm/sysv.S:411: Error: undefined symbol FFI_TYPE_STRUCT_VFP_DOUBL
E used as an immediate value
/home/ovos/OpenVoiceOS/buildroot/output/build/host-python-2.7.18/Modules/_ctypes/libffi/src/arm/sysv.S:450: Error: undefined symbol FFI_TYPE_STRUCT_VFP_FLOAT
 used as an immediate value
/home/ovos/OpenVoiceOS/buildroot/output/build/host-python-2.7.18/Modules/_ctypes/libffi/src/arm/sysv.S:453: Error: undefined symbol FFI_TYPE_STRUCT_VFP_DOUBL
E used as an immediate value

Python build finished, but the necessary bits to build these modules were not found:
_bsddb             _sqlite3           _tkinter
bsddb185           bz2                dbm
gdbm               readline           sunaudiodev
To find the necessary bits, look in setup.py in detect_modules() for the module's name.

Failed to build these modules:
_ctypes

Makefile:544: recipe for target 'sharedmods' failed
make[2]: *** [sharedmods] Error 1
make[2]: Leaving directory '/home/ovos/OpenVoiceOS/buildroot/output/build/host-python-2.7.18'
package/pkg-generic.mk:247: recipe for target '/home/ovos/OpenVoiceOS/buildroot/output/build/host-python-2.7.18/.stamp_built' failed
make[1]: *** [/home/ovos/OpenVoiceOS/buildroot/output/build/host-python-2.7.18/.stamp_built] Error 2
make[1]: Leaving directory '/home/ovos/OpenVoiceOS/buildroot'

Not sure how far into the build it is before exiting (but it is getting much further than before) and not sure if this is a problem with packages not installed in my build environment or something else? Pretty sure my hardware issues are resolved now though and I have over 80GB available on my USB attached SSD so that should be fine.

[NeonDaniel]

make[2]: Leaving directory '/home/ovos/OpenVoiceOS/buildroot/output/build/host-python-2.7.18'

Are you building with python2.7? 3.8 is what worked for me last; you can check with:

python --version

[limitless-code]

You are correct I am. I will try the version 3.8

Sent from Mailhttps://go.microsoft.com/fwlink/?LinkId=550986 for Windows

From: Daniel @.> Sent: 10 August 2021 01:34 To: @.> Cc: @.>; @.> Subject: Re: [OpenVoiceOS/OpenVoiceOS] Build errors (#58)

make[2]: Leaving directory '/home/ovos/OpenVoiceOS/buildroot/output/build/host-python-2.7.18'

Are you building with python2.7? 3.8 is what worked for me last; you can check with:

python --version

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/OpenVoiceOS/OpenVoiceOS/issues/58#issuecomment-895642029, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ALX6KGNG547FKHN3M32SZS3T4BXXRANCNFSM5BLIMVHA. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email.

[NeonDaniel]

You are correct I am. I will try the version 3.8

If you're keeping track of fixes as you go, adding notes to the README would be much appreciated! :)

[limitless-code]

will do.. however LOL:

I have built another docker image... I am sure it doesn't have python version 2.X but still the same warning and error for make rpi4_64-gui-config and make rpi4_64-gui .

I am in my own python env this time too.

(my_env) ovos@c884047e7572:~/OpenVoiceOS$ python3
Python 3.8.10 (default, Jun  2 2021, 10:49:15)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

Note: Ubuntu latest is

here is the Dockerfile

FROM ubuntu:latest

ENV TERM linux
ENV DEBIAN_FRONTEND noninteractive

RUN set -x \
    && sed -i 's/# \(.*multiverse$\)/\1/g' /etc/apt/sources.list \
    && apt-get update \
    && apt-get -y install --no-install-recommends \
    vim less nano git subversion gcc qttools5-dev qttools5-dev-tools python3 python3-dev libncurses-dev \
    patch make file g++ wget cpio unzip rsync bc git sudo ca-certificates xz-utils curl \
    qtbase5-dev qtdeclarative5-dev bzip2 gzip tar\
    python3-setuptools libtool \
    libffi-dev libssl-dev autoconf automake bison swig libglib2.0-dev \
    portaudio19-dev mpg123 screen flac curl libicu-dev pkg-config \
    libjpeg-dev libfann-dev build-essential jq pulseaudio pulseaudio-utils \
    && apt-get -y autoremove \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

RUN addgroup --gid 1001 --system ovos && \
    adduser --system --uid 1001 --gid 1001 ovos

USER ovos

I am running it in bash (entry point via command line) in the docker container the other thing I changed from the dockerfile was running this as root.

apt install python3.8-venv

could it be that buildroot/ make is downloading python-2.7.18 and is separate from my OS build environment?

[limitless-code]

OK. A quick look further up buildroot_rpi4_64-gui_output.txt

rm -rf /home/ovos/OpenVoiceOS/buildroot/output/target/usr/share/glib-2.0/gdb
/usr/bin/sed -i -e 's%${bindir}/%%g' /home/ovos/OpenVoiceOS/buildroot/output/host/aarch64-buildroot-linux-gnu/sysroot/usr/lib/pkgconfig/glib-2.0.pc /home/ovo
s/OpenVoiceOS/buildroot/output/host/aarch64-buildroot-linux-gnu/sysroot/usr/lib/pkgconfig/gio-2.0.pc
ESC[7m>>> host-python 2.7.18 DownloadingESC[27m
--2021-08-11 14:18:06--  https://python.org/ftp/python/2.7.18/Python-2.7.18.tar.xz
Resolving python.org (python.org)... 138.197.63.241
Connecting to python.org (python.org)|138.197.63.241|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://www.python.org/ftp/python/2.7.18/Python-2.7.18.tar.xz [following]
--2021-08-11 14:18:07--  https://www.python.org/ftp/python/2.7.18/Python-2.7.18.tar.xz
Resolving www.python.org (www.python.org)... 151.101.12.223, 2a04:4e42:3::223
Connecting to www.python.org (www.python.org)|151.101.12.223|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 12854736 (12M) [application/octet-stream]
Saving to: '/home/ovos/OpenVoiceOS/buildroot/output/build/.Python-2.7.18.tar.xz.IcEh92/output'

So I guess it's not my OS build environment, but something is missing?

Although it doesn't explain the warning from rpi4_64-gui-config

[limitless-code]

I am most likely wrong here as I don't understand how buildroot works or what the expected behaviour is.

however the modules it could not find are modules that a patch told it to disable so i guess this part where it can't find modules is expected behaviour.

e.g

Applying 0019-Add-option-to-disable-the-sqlite3-module.patch using patch:

so I guess it's this error:

Failed to build these modules:
_ctypes

Makefile:544: recipe for target 'sharedmods' failed
make[2]: *** [sharedmods] Error 1

that's pertinent to it failing, as it mentions python-2.7.18 during this process up to failing which is something that is separate to the OS build environment?

[j1nx]

What build machine you are doing all this?

[limitless-code]

1. Raspberry pi 4
2. 4GB RAM
3. Raspbian buster
4. Running Docker
5. USB attached SSD for docker container

running docker container interactively with ubuntu 20.04 as per Dockerfile in above comments

sudo docker exec -it <container-name> bash

Then followed the README.

[j1nx]

Right! Buildroot is a cross compiling system. You do not run it on a rpi but on a normal x86_64 system.

[limitless-code]

Thank you for ending my hell! This makes total sense. I shall find a spare x86_64 system or use my laptop. I looked into emulating an x86_64 system in docker using qemu on the rpi 4 but decided not to, saving the little sanity I have left in the process as that still may not currently be possible on arm based architecture anyway (some emulation does work but not x86_64 currently, only manged to get i686 emulation working in docker)

So to conclude: Only use x86_64 based architecture/OS

[NeonDaniel]

So to conclude: Only use x86_64 based architecture/OS

If you could, please add a note to the README. Totally missed this in my troubleshooting.

[limitless-code]

Sure, I will hold fire on conclusion 'til I receive new SSD so i can boot to it, test on x86_64 architecture and ensure nothing else is missed.

That won't happen 'till the after weekend.

[limitless-code]

ftp://ftp.alsa-project.org/pub/plugins/alsa-plugins-1.2.2.tar.bz2

FTP PASV not usually allowed by default as it requires port 21 and other random ports to be opened on the firewall.

or it wil failover and work via http so long as the file is there which it is not:

--2021-08-15 16:14:19--  http://sources.buildroot.net/alsa-plugins/alsa-plugins-1.2.2.tar.bz2
Resolving sources.buildroot.net (sources.buildroot.net)... 104.26.1.37, 104.26.0.37, 172.67.72.56, ...
Connecting to sources.buildroot.net (sources.buildroot.net)|104.26.1.37|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2021-08-15 16:14:19 ERROR 404: Not Found.

--2021-08-15 16:14:19--  http://sources.buildroot.net/alsa-plugins-1.2.2.tar.bz2
Resolving sources.buildroot.net (sources.buildroot.net)... 172.67.72.56, 104.26.0.37, 104.26.1.37, ...
Connecting to sources.buildroot.net (sources.buildroot.net)|172.67.72.56|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2021-08-15 16:14:19 ERROR 404: Not Found.

http://sources.buildroot.net/alsa-plugins/alsa-plugins-1.2.5.tar.bz2

is available though.

for now I have downloaded alsa-plugins-1.2.2.tar.bz2 from else where ("buildroot" does a checksum compare so it's safe to do this) and pointed sources.buildroot.net to my machine to get round the issue.

tcp 9418 (git) needs to be allowed on the firewall.

[limitless-code]

OK my build environment is: -

hardware: x86_64 architecture OS: Ubuntu 20.04 desktop Extra packages installed via apt in addition to the packages in the README:

1. git
2. make
3. g++
4. curl

build errors out after ~15rs

>>> host-kf5-ki18n 5.75.0 Configuring
(mkdir -p /home/ovos/OpenVoiceOS/buildroot/output/build/host-kf5-ki18n-5.75.0//buildroot-build && cd /home/ovos/OpenVoiceOS/buildroot/output/build/host-kf5-ki18n-5.75.0//buildroot-build && rm -f CMakeCache.txt && PATH="/home/ovos/OpenVoiceOS/buildroot/output/host/bin:/home/ovos/OpenVoiceOS/buildroot/output/host/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin" PKG_CONFIG="/home/ovos/OpenVoiceOS/buildroot/output/host/bin/pkg-config" PKG_CONFIG_SYSROOT_DIR="/" PKG_CONFIG_LIBDIR="/home/ovos/OpenVoiceOS/buildroot/output/host/lib/pkgconfig:/home/ovos/OpenVoiceOS/buildroot/output/host/share/pkgconfig" PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1 PKG_CONFIG_ALLOW_SYSTEM_LIBS=1  /home/ovos/OpenVoiceOS/buildroot/output/host/bin/cmake /home/ovos/OpenVoiceOS/buildroot/output/build/host-kf5-ki18n-5.75.0/ -DCMAKE_INSTALL_SO_NO_EXE=0 -DCMAKE_FIND_ROOT_PATH="/home/ovos/OpenVoiceOS/buildroot/output/host" -DCMAKE_FIND_ROOT_PATH_MODE_PROGRAM="BOTH" -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY="BOTH" -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE="BOTH" -DCMAKE_INSTALL_PREFIX="/home/ovos/OpenVoiceOS/buildroot/output/host" -DCMAKE_C_FLAGS="-O2 -I/home/ovos/OpenVoiceOS/buildroot/output/host/include" -DCMAKE_CXX_FLAGS="-O2 -I/home/ovos/OpenVoiceOS/buildroot/output/host/include" -DCMAKE_EXE_LINKER_FLAGS="-L/home/ovos/OpenVoiceOS/buildroot/output/host/lib -Wl,-rpath,/home/ovos/OpenVoiceOS/buildroot/output/host/lib" -DCMAKE_SHARED_LINKER_FLAGS="-L/home/ovos/OpenVoiceOS/buildroot/output/host/lib -Wl,-rpath,/home/ovos/OpenVoiceOS/buildroot/output/host/lib" -DCMAKE_ASM_COMPILER="/usr/bin/as" -DCMAKE_C_COMPILER="/usr/bin/gcc" -DCMAKE_CXX_COMPILER="/usr/bin/g++"  -DCMAKE_C_COMPILER_LAUNCHER="/home/ovos/OpenVoiceOS/buildroot/output/host/bin/ccache" -DCMAKE_CXX_COMPILER_LAUNCHER="/home/ovos/OpenVoiceOS/buildroot/output/host/bin/ccache"  -DCMAKE_COLOR_MAKEFILE=OFF -DBUILD_DOC=OFF -DBUILD_DOCS=OFF -DBUILD_EXAMPLE=OFF -DBUILD_EXAMPLES=OFF -DBUILD_TEST=OFF -DBUILD_TESTS=OFF -DBUILD_TESTING=OFF -DBUILD_SHARED_LIBS=ON  -DCMAKE_CXX_FLAGS="-O2 -I/home/ovos/OpenVoiceOS/buildroot/output/host/include" )
-- The C compiler identification is GNU 9.3.0
-- The CXX compiler identification is GNU 9.3.0
-- Check for working C compiler: /usr/bin/gcc
-- Check for working C compiler: /usr/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/g++
-- Check for working CXX compiler: /usr/bin/g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- 

Installing in /home/ovos/OpenVoiceOS/buildroot/output/host. Run /home/ovos/OpenVoiceOS/buildroot/output/build/host-kf5-ki18n-5.75.0/buildroot-build/prefix.sh to set the environment for KI18n.
-- Looking for __GLIBC__
-- Looking for __GLIBC__ - found
-- Performing Test _OFFT_IS_64BIT
-- Performing Test _OFFT_IS_64BIT - Success
-- Performing Test HAVE_DATE_TIME
-- Performing Test HAVE_DATE_TIME - Success
CMake Error at CMakeLists.txt:42 (find_package):
  Could not find a package configuration file provided by "Qt5Qml" (requested
  version 5.12.0) with any of the following names:

    Qt5QmlConfig.cmake
    qt5qml-config.cmake

  Add the installation prefix of "Qt5Qml" to CMAKE_PREFIX_PATH or set
  "Qt5Qml_DIR" to a directory containing one of the above files.  If "Qt5Qml"
  provides a separate development package or SDK, be sure it has been
  installed.

-- Configuring incomplete, errors occurred!
See also "/home/ovos/OpenVoiceOS/buildroot/output/build/host-kf5-ki18n-5.75.0/buildroot-build/CMakeFiles/CMakeOutput.log".
make[2]: *** [package/pkg-generic.mk:240: /home/ovos/OpenVoiceOS/buildroot/output/build/host-kf5-ki18n-5.75.0/.stamp_configured] Error 1
make[1]: *** [Makefile:84: _all] Error 2
make[1]: Leaving directory '/home/ovos/OpenVoiceOS/buildroot'
cp -f buildroot/output/images/sdcard.img release/OpenVoiceOS_rpi4_64-gui.img
cp: cannot stat 'buildroot/output/images/sdcard.img': No such file or directory
make: *** [Makefile:26: rpi4_64-gui] Error 1

The build environment shows "libqt5qml5 is already the newest version (5.12.8-0ubuntu1)."

[limitless-code]

sudo apt-get install qtdeclarative5-dev fixes the above error.

[limitless-code]

Build fails Package no longer there/ moved?

pying jarbas_stt_plugin_chromium.egg-info to /home/ovos/OpenVoiceOS/buildroot/output/target/usr/lib/python3.8/site-packages/jarbas_stt_plugin_chromium-0.1.3-py3.8.egg-info
running install_scripts
ESC[7m>>> python-jarbas-wake-word-plugin-precise ad3ced5e4fec11530fc4d065bd446c689e39987a DownloadingESC[27m
--2021-08-18 16:30:29--  https://github.com/JarbasLingua/jarbas-wake-word-plugin-precise/archive/ad3ced5e4fec11530fc4d065bd446c689e39987a/python-jarbas-wake-word-plugin-precise-ad3ced5e4fec11530fc4d065bd446c689e39987a.tar.gz
Resolving github.com (github.com)... 140.82.121.4
Connecting to github.com (github.com)|140.82.121.4|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2021-08-18 16:30:30 ERROR 404: Not Found.

--2021-08-18 16:30:30--  http://sources.buildroot.net/python-jarbas-wake-word-plugin-precise/python-jarbas-wake-word-plugin-precise-ad3ced5e4fec11530fc4d065bd446c689e39987a.tar.gz
Resolving sources.buildroot.net (sources.buildroot.net)... 104.26.1.37, 104.26.0.37, 172.67.72.56, ...
Connecting to sources.buildroot.net (sources.buildroot.net)|104.26.1.37|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2021-08-18 16:30:31 ERROR 404: Not Found.

--2021-08-18 16:30:31--  http://sources.buildroot.net/python-jarbas-wake-word-plugin-precise-ad3ced5e4fec11530fc4d065bd446c689e39987a.tar.gz
Resolving sources.buildroot.net (sources.buildroot.net)... 172.67.72.56, 104.26.0.37, 104.26.1.37, ...
Connecting to sources.buildroot.net (sources.buildroot.net)|172.67.72.56|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2021-08-18 16:30:31 ERROR 404: Not Found.

make[2]: *** [package/pkg-generic.mk:148: /home/ovos/OpenVoiceOS/buildroot/output/build/python-jarbas-wake-word-plugin-precise-ad3ced5e4fec11530fc4d065bd446c689e39987a/.stamp_downloaded] Error 1
make[1]: *** [Makefile:84: _all] Error 2
make[1]: Leaving directory '/home/ovos/OpenVoiceOS/buildroot'

EDIT:

In file:

 buildroot/output/target/usr/lib/python3.8/site-packages/jarbas_stt_plugin_chromium-0.1.3-py3.8.egg-info/PKG-INFO

Home-page: https://github.com/JarbasLingua/jarbas-stt-plugin-chromium

redirects to here:

https://github.com/OpenVoiceOS/ovos-stt-plugin-chromium

but isn't reflected in the logs.

[ChanceNCounter]

On it.

[ChanceNCounter]

Re: moved packages, #76

[limitless-code]

Thanks all for your input and patience. I have an image loaded (4.1GB?) and connected via WiFi, although there was some issue with the WiFi connection which may be due to either the length of my passphrase or more likely due to it having a few special characters (dashes - ) in the passphrase?

currently it is on the "Starting Up" Loading ... screen

My build environment is: -

• hardware: x86_64 architecture
• OS: Ubuntu 20.04 desktop

Extra packages installed via apt in addition to the packages in the README:

git
make
g++
curl
wget
qtdeclarative5-dev

It took ~22hrs to build the image using Intel Core i5-6200U Processor with 8GB Ram and USB2 attached 240GB SSD

[limitless-code]

It seems that Mycroft loaded some of my skills before pairing? Only noticed this when I opened my front door home assistant told Mycroft (OVOS) and Mycroft spoke that my "front door is open", didn't think it should do this before pairing. It was stuck on the Mycroft/ Local Backend choice for a while and rebooted a few times so didn't get a chance to get the pairing code in order to pair.

Have a Home assistant skill setup in home.mycroft.ai from previous picroft installation, not sure how it knew to download "my profile" before pairing (mac address? just a guess as I'm pretty sure my public ip address has change since picroft install).

[JarbasAl]

the local backend selection path is not yet fully implemented, it will seem to get stuck and never finish loading

but the whole point of running locally is not needing to pair or use mycroft servers at all, so I'm not sure if i misunderstood or what you were expecting to happen?

[limitless-code]

the local backend selection path is not yet fully implemented, it will seem to get stuck and never finish loading

but the whole point of running locally is not needing to pair or use mycroft servers at all, so I'm not sure if i misunderstood or what you were expecting to happen?

Thanks, re: backend.

I see now, Mycroft is integrated on Home assistant and doesn't need a "Long lived" token for HA to talk to Mycroft in order for it to do TTS for messages which is why it worked (so long as the "mycroft device" ip address hadn't changed which it hadn't). I thought it was using a skill but that's to push from Mycroft to HA which does require the Long Lived token and as far as i can tell so as you say it's working as expected.

Are there default credentials so i can SSH to OVOS and troubleshoot etc? Thanks

[ChanceNCounter]

mycroft/mycroft, and sshd should be running

[limitless-code]

Someone asked me to update the README. I don't think I would have access to update it? If anyone wishes to update the README I left a summary of the build environment and extra packages required a few comments up. Thanks!

[limitless-code]

closing issue

[ChanceNCounter]

@limitless-code I think that was directed at @j1nx, and somebody will probably get around to it. However, the question belies a certain unfamiliarity that I love to fix, as most git tutorials are WAY too advanced and explain too much at once. I've never tried to write it in a single GitHub comment, so here goes.

(Edit: I explain nothing, quite deliberately. Just do the stuff.)

Fork this repo on GitHub, clone your fork, and PR the changes. If you already cloned it from our GitHub, you don't have to clone it again. One command will switch to your fork:

git remote set-url origin [your_fork_url]

Next, make a new branch so you can switch between your changes and the "main" version.

git checkout -b update-readme (outputs "switched to new branch: update-readme")

You are now working on your own branch of this repo, separate from the development and production branches that we build and ship. You named it update-readme because I told you to =P but git doesn't care what you name it.

You can make your changes to the readme now. When you're finished,

git add README.md
git status (verify the file is staged, in green, and nothing else is green)
git commit -m "update readme"

You have now added a commit to your feature branch, which you can submit to us. If you want to verify that it worked, you can type git log. It will open in your pager. Your new commit should be at the top, with the message "update readme". That's how it will look in this repo, too. When you're finished, type

git push origin update-readme

GitHub may ask for your credentials at this point. Then it will push your changes to GitHub, on your fork, in a branch called update-readme.

GitHub should send back a link, which will be near the bottom of the terminal output. That link will open a pull request back here. Open the link and hit "submit."

Congratulations. You have just used git to submit changes to open source software. You don't need to know why you did any of that. Half of it is self explanatory. The rest will come with time. Nuking and starting over is usually the wrong answer. Tuts just suck, so nobody ever gets that far, and most end up thinking of git as "that thing that makes me nuke my code once in a while."

[limitless-code]

the local backend selection path is not yet fully implemented, it will seem to get stuck and never finish loading

Just a quick note (even though I closed this): - I found that the problem with the backend options for me was a problem using a RPI4 with 1GB RAM and it using all the swap partition (even once setup). My RPI4 with 2GB RAM (same sd card) performed seamlessly especially when choosing the mycroft backend option. Perhaps even more RAM (4GB) is required when playing video (on a big screen TV), but so far not seen any swapping using 2GB RPI4.

[limitless-code]

@ChanceNCounter, Thanks for the write up of the "just do the stuff" re: git! I have used git to make changes/ updates to my own repositories (all be it a year ago), so your detailed but simplified write up adds some needed knowledge on how use git to submit changes to open source software and is much appreciated! I will give it a go at some point this week.