dkbuilder buildlv2 error

[MaxPayne86]

Hello, I started today to look into the nice dkbuilder tool, first of all thanks for realizing this tool. I've installed Docker.io and fired the following commands

1. host: $ docker run -it --name guitarix-dkbuilder -v /localdisk/massimo/Work/Git/brummer10/guitarix:/mnt/data ubuntu bash
2. guest: $ apt-get update
3. guest: $ apt-get install git geda geda-gattrib geda-gnetlist geda-gschem geda-gsymcheck geda-symbols python-dev python-virtualenv python-tk virtualenv virtualenvwrapper faust libeigen3-dev libcminpack-dev libsundials-dev libserial-dev maxima libslicot0
4. guest: $ dpkg --list | grep faust # Need to be = 0.9.90, is actually 0.9.95~repack1-2
5. guest: $ cd /mnt/data/trunk/tools/ampsim/DK
6. guest: $ source /usr/share/virtualenvwrapper/virtualenvwrapper.sh
7. guest: $ mkvirtualenv -r python_venv guitarix
8. guest: $ nano ~/.bashrc
9. guest: $ append this into the ~/.bashrc file

dkbuilder() { source /usr/share/virtualenvwrapper/virtualenvwrapper.sh cd /mnt/data/trunk/tools/ampsim/DK/gschem-schematics workon guitarix }

10. guest: $ source ~/.bashrc
11. guest: $ dkbuilder # Run dkbuilder
12. guest: $ ./build-plug.py -i VoxWahV847.sch --buildlv2 -n VoxWahV847

Input file 1: VoxWahV847.sch module_id: voxwahv847 ['R12', 'RESISTOR', '10k', 'unnamed_net11', 'unnamed_net2'] ['R11', 'RESISTOR', '10k', 'GND', 'unnamed_net10'] ['RL', 'RESISTOR', '60', 'unnamed_net12', 'V6'] ['L1', 'INDUCTOR', '0.5H', 'unnamed_net8', 'unnamed_net12'] ['R3', 'RESISTOR', '22k', 'unnamed_net3', '+9V'] ['Vcc', 'POWER', '9V', '+9V'] ['P1', 'VARIABLE_RESISTOR', 'value=80k,var=Wah,a=3', 'unnamed_net10', 'unnamed_net11', 'unnamed_net9'] ['C1', 'CAPACITOR', '0.01u', 'V2', 'V1'] ['R10', 'RESISTOR', '10k', 'GND', 'unnamed_net6'] ['R5', 'RESISTOR', '100k', 'GND', 'V6'] ['R7', 'RESISTOR', '33k', 'unnamed_net8', 'V6'] ['R6', 'RESISTOR', '470k', 'V6', 'unnamed_net3'] ['R8', 'RESISTOR', '470k', 'unnamed_net5', 'unnamed_net3'] ['R1', 'RESISTOR', '68k', 'V1', 'unnamed_net7'] ['C4', 'CAPACITOR', '0.22u', 'unnamed_net5', 'unnamed_net9'] ['C3', 'CAPACITOR', '0.01u', 'unnamed_net6', 'unnamed_net8'] ['R4', 'RESISTOR', '510', 'GND', 'unnamed_net4'] ['R2', 'RESISTOR', '1.5k', 'unnamed_net8', 'V2'] ['C2', 'POLARIZED_CAPACITOR', '4.7u', 'V6', 'GND'] ['OUT1', 'OUTPUT', 'None', 'unnamed_net2'] ['IN1', 'INPUT', 'None', 'unnamed_net7'] ['T2', 'NPN_TRANSISTOR', 'Vt=26mV,Is=20.3fA,Bf=1430,Br=4', 'unnamed_net1', 'unnamed_net5', 'unnamed_net6'] ['T1', 'NPN_TRANSISTOR', 'Vt=26mV,Is=20.3fA,Bf=1430,Br=4', 'unnamed_net3', 'V2', 'unnamed_net4'] ['C6', 'CAPACITOR', '0.22u', 'unnamed_net2', 'unnamed_net3'] ['R9', 'RESISTOR', '1k', 'unnamed_net1', '+9V']

S = ((R(12), 'u11', 'u2',), (R(11), GND, 'u10',), (R('L'), 'u12', 'V6',), (L(1), 'u8', 'u12',), (R(3), 'u3', '+9V',), (V('cc'), '+9V',), (P(1), 'u10', 'u11', 'u9',), (C(1), 'V2', 'V1',), (R(10), GND, 'u6',), (R(5), GND, 'V6',), (R(7), 'u8', 'V6',), (R(6), 'V6', 'u3',), (R(8), 'u5', 'u3',), (R(1), 'V1', 'u7',), (C(4), 'u5', 'u9',), (C(3), 'u6', 'u8',), (R(4), GND, 'u4',), (R(2), 'u8', 'V2',), (C(2), 'V6', GND,), (T(2), 'u1', 'u5', 'u6',), (T(1), 'u3', 'V2', 'u4',), (C(6), 'u2', 'u3',), (R(9), 'u1', '+9V',), (OUT, 'u2',), (IN, 'u7',), ) V = {C(1): 0.01e-6, C(2): 4.7e-6, C(3): 0.01e-6, C(4): 0.22e-6, C(6): 0.22e-6, L(1): 0.5, P(1): dict(value=80.e3, var='Wah', a=3), R('L'): 60., R(1): 68.e3, R(10): 10.e3, R(11): 10.e3, R(12): 10.e3, R(2): 1.5e3, R(3): 22.e3, R(4): 510., R(5): 100.e3, R(6): 470.e3, R(7): 33.e3, R(8): 470.e3, R(9): 1.e3, T(1): dict(Vt=26e-3, Is=20.3e-15, Bf=1430, Br=4), T(2): dict(Vt=26e-3, Is=20.3e-15, Bf=1430, Br=4), V('cc'): 9., }

personality failure 1

---

Truncated output: ... Traceback (most recent call last): File "./build-plug.py", line 479, in main(sys.argv[1:]) File "./build-plug.py", line 476, in main dk.build() File "./build-plug.py", line 397, in build faustdsp, faustui = c1.get_faust_code(filename=str(dspname)) File "./analog.py", line 780, in get_faust_code b, a = self.sim_filter.get_z_coeffs(samplerate=FS) File "./dk_simulator.py", line 244, in get_z_coeffs expr = self.solve(expr, self.in_mat, self.out_mat) File "./dk_simulator.py", line 334, in solve maxima_error("unexpected output from maxima") File "./dk_simulator.py", line 328, in maxima_error raise ValueError(s) ValueError: unexpected output from maxima`

Any clue?

[brummer10]

Hi

When using a virtual environment, you've already dependency isolation from you main system, so there is no reason to encapsulate it in a docker. Indeed you've to install the pre-requests then on your main system, that may be the reason why you use docker? Other then that, the steps you've made to setup the dkbuilder environment seems alright to me. Beside that faust now could be version 2.15.11 as well, but that have nothing to do with the error message. I'm sorry, but I'm expected docker to be the culprit, so I've no idea how you could get it to run in there. But, I may be wrong, I've no experience with docker at all.

[MaxPayne86]

When using a virtual environment, you've already dependency isolation from you main system, so there is no reason to encapsulate it in a docker. Indeed you've to install the pre-requests then on your main system, that may be the reason why you use docker?

I know that some of the problems involved with tools are with the type of distribution, packages, libraries and so on. Python-virtualenv isolates the python stuff but you're still dependent on system libraries like for example maxima. Regarding the docker itself, I'm using the ubuntu image so it's like having a clean ubuntu 18.04.1 image from a live cd, except I don't have DISPLAY defined so I may not be able to use the --table option.

Which version of maxima are you using? Mine is 5.41.0-3

Cheers,

[brummer10]

I'm using debian/sid with maxima 5.43.2-3, but it is known to work with older versions as well. Now I see it seems you've forgotten to install maxima-share. Try to install that before you try to install a newer maxima version.

[adegert]

Am Di., 14. Apr. 2020 um 15:31 Uhr schrieb Massimo Pennazio notifications@github.com:

Hello, I started today to look into the nice dkbuilder tool, first of all thanks for realizing this tool. I've installed Docker.io and fired the following commands [...] guest: $ ./build-plug.py -i VoxWahV847.sch --buildlv2 -n VoxWahV847 [...] personality failure 1

I think this error message is the key. Maxima seems to query the process personality with the Linux personality syscall. My guess is that the standard docker container security setting blocks this call. Please try if calling docker run with --privileged helps.

(just starting maxima --version should show if it works, since maxima uses these calls on startup: $ strace maxima --version |& grep perso personality(0xffffffff) = 0 (PER_LINUX) personality(PER_LINUX|READ_IMPLIES_EXEC) = 0 (PER_LINUX) personality(PER_LINUX|ADDR_NO_RANDOMIZE) = 0x400000 (PER_LINUX|READ_IMPLIES_EXEC) personality(0xffffffff) = 0x40000 (PER_LINUX|ADDR_NO_RANDOMIZE) personality(0xffffffff) = 0x40000 (PER_LINUX|ADDR_NO_RANDOMIZE) personality(PER_LINUX|ADDR_NO_RANDOMIZE|READ_IMPLIES_EXEC) = 0x40000 (PER_LINUX|ADDR_NO_RANDOMIZE)

you can also use more fine grained methods to modify the security setting of a docker container, but I would first try with --privileged)

[MaxPayne86]

Thanks for the support @adegert @brummer10,

I've enabled the personality syscall in my docker container in this way

1. host: $ docker run -it --name guitarix-dkbuilder -v /localdisk/massimo/Work/Git/brummer10/guitarix:/mnt/data --security-opt seccomp=/home/massimo/Stuff/docker/personality.json ubuntu bash

where personality.json is the original default.json patched in this way

--- default.json    2020-04-15 13:13:08.948020031 +0200
+++ personality.json    2020-04-15 13:13:08.948020031 +0200
@@ -227,6 +227,7 @@
                "open",
                "openat",
                "pause",
+               "personality",
                "pipe",
                "pipe2",
                "poll",

I've then retried build-plug.py

(guitarix) root@a64fa33012ad:/mnt/data/trunk/tools/ampsim/DK/gschem-schematics# ./build-plug.py -i VoxWahV847.sch --buildlv2 -n VoxWahV847

Input file 1: VoxWahV847.sch
module_id: voxwahv847
['R12', 'RESISTOR', '10k', 'unnamed_net11', 'unnamed_net2']
['R11', 'RESISTOR', '10k', 'GND', 'unnamed_net10']
['RL', 'RESISTOR', '60', 'unnamed_net12', 'V6']
['L1', 'INDUCTOR', '0.5H', 'unnamed_net8', 'unnamed_net12']
['R3', 'RESISTOR', '22k', 'unnamed_net3', '+9V']
['Vcc', 'POWER', '9V', '+9V']
['P1', 'VARIABLE_RESISTOR', 'value=80k,var=Wah,a=3', 'unnamed_net10', 'unnamed_net11', 'unnamed_net9']
['C1', 'CAPACITOR', '0.01u', 'V2', 'V1']
['R10', 'RESISTOR', '10k', 'GND', 'unnamed_net6']
['R5', 'RESISTOR', '100k', 'GND', 'V6']
['R7', 'RESISTOR', '33k', 'unnamed_net8', 'V6']
['R6', 'RESISTOR', '470k', 'V6', 'unnamed_net3']
['R8', 'RESISTOR', '470k', 'unnamed_net5', 'unnamed_net3']
['R1', 'RESISTOR', '68k', 'V1', 'unnamed_net7']
['C4', 'CAPACITOR', '0.22u', 'unnamed_net5', 'unnamed_net9']
['C3', 'CAPACITOR', '0.01u', 'unnamed_net6', 'unnamed_net8']
['R4', 'RESISTOR', '510', 'GND', 'unnamed_net4']
['R2', 'RESISTOR', '1.5k', 'unnamed_net8', 'V2']
['C2', 'POLARIZED_CAPACITOR', '4.7u', 'V6', 'GND']
['OUT1', 'OUTPUT', 'None', 'unnamed_net2']
['IN1', 'INPUT', 'None', 'unnamed_net7']
['T2', 'NPN_TRANSISTOR', 'Vt=26mV,Is=20.3fA,Bf=1430,Br=4', 'unnamed_net1', 'unnamed_net5', 'unnamed_net6']
['T1', 'NPN_TRANSISTOR', 'Vt=26mV,Is=20.3fA,Bf=1430,Br=4', 'unnamed_net3', 'V2', 'unnamed_net4']
['C6', 'CAPACITOR', '0.22u', 'unnamed_net2', 'unnamed_net3']
['R9', 'RESISTOR', '1k', 'unnamed_net1', '+9V']

S = ((R(12), 'u11', 'u2',),
     (R(11), GND, 'u10',),
     (R('L'), 'u12', 'V6',),
     (L(1), 'u8', 'u12',),
     (R(3), 'u3', '+9V',),
     (V('cc'), '+9V',),
     (P(1), 'u10', 'u11', 'u9',),
     (C(1), 'V2', 'V1',),
     (R(10), GND, 'u6',),
     (R(5), GND, 'V6',),
     (R(7), 'u8', 'V6',),
     (R(6), 'V6', 'u3',),
     (R(8), 'u5', 'u3',),
     (R(1), 'V1', 'u7',),
     (C(4), 'u5', 'u9',),
     (C(3), 'u6', 'u8',),
     (R(4), GND, 'u4',),
     (R(2), 'u8', 'V2',),
     (C(2), 'V6', GND,),
     (T(2), 'u1', 'u5', 'u6',),
     (T(1), 'u3', 'V2', 'u4',),
     (C(6), 'u2', 'u3',),
     (R(9), 'u1', '+9V',),
     (OUT, 'u2',),
     (IN, 'u7',),
     )
V = {C(1): 0.01e-6,
     C(2): 4.7e-6,
     C(3): 0.01e-6,
     C(4): 0.22e-6,
     C(6): 0.22e-6,
     L(1): 0.5,
     P(1): dict(value=80.e3, var='Wah', a=3),
     R('L'): 60.,
     R(1): 68.e3,
     R(10): 10.e3,
     R(11): 10.e3,
     R(12): 10.e3,
     R(2): 1.5e3,
     R(3): 22.e3,
     R(4): 510.,
     R(5): 100.e3,
     R(6): 470.e3,
     R(7): 33.e3,
     R(8): 470.e3,
     R(9): 1.e3,
     T(1): dict(Vt=26e-3, Is=20.3e-15, Bf=1430, Br=4),
     T(2): dict(Vt=26e-3, Is=20.3e-15, Bf=1430, Br=4),
     V('cc'): 9.,
     } 

build lv2_plugin from: ['VoxWahV847.sch']
VoxWahV847
Please select a Plugin type from the list
 1) DelayPlugin       14) BandpassPlugin    27) FlangerPlugin
 2) ReverbPlugin      15) CombPlugin        28) PhaserPlugin
 3) DistortionPlugin  16) EQPlugin      29) ReverbPlugin
 4) WaveshaperPlugin  17) MultiEQPlugin     30) SimulatorPlugin
 5) DynamicsPlugin    18) ParaEQPlugin      31) ReverbPlugin
 6) AmplifierPlugin   19) HighpassPlugin    32) SpatialPlugin
 7) CompressorPlugin  20) LowpassPlugin     33) SpectralPlugin
 8) EnvelopePlugin    21) GeneratorPlugin   34) PitchPlugin
 9) ExpanderPlugin    22) ConstantPlugin    35) UtilityPlugin
10) GatePlugin        23) InstrumentPlugin  36) AnalyserPlugin
11) LimiterPlugin     24) OscillatorPlugin  37) ConverterPlugin
12) FilterPlugin      25) ModulatorPlugin   38) FunctionPlugin
13) AllpassPlugin     26) ChorusPlugin      39) MixerPlugin
#? 14
Plugin type BandpassPlugin
Directory gx_voxwahv847.lv2 allready exist, should we remove it? Yes/No
1) Yes
2) No
#? 1
generate voxwahv847.cc and copy to gx_voxwahv847.lv2
Traceback (most recent call last):
  File "../../..//dsp2cc", line 2329, in <module>
    main()
  File "../../..//dsp2cc", line 2309, in main
    options)
  File "../../..//dsp2cc", line 579, in __init__
    self.faust_version = headvers.group(3)
AttributeError: 'NoneType' object has no attribute 'group'
Error, see message above

So the problem seems to be in the faust code parser. I've added some prints around line 579 of dsp2cc

type(self) = <class '__main__.Parser'>
type(self.lines) = <type 'generator'>
line0 = //----------------------------------------------------------

line1 = // name: "VoxWahV847"

line2 = //

line3 = // Code generated with Faust 0.9.95 (http://faust.grame.fr)

line4 = //----------------------------------------------------------
...
headvers = None

so it seems every line in self.lines has some data and the string "Code generated with Faust" is present so maybe the problem is in skip_until method?

Thanks

[adegert]

Am Mi., 15. Apr. 2020 um 17:52 Uhr schrieb Massimo Pennazio notifications@github.com:

Thanks for the support @adegert @brummer10,

I've enabled the personality syscall in my docker container in this way

1. host: $ docker run -it --name guitarix-dkbuilder -v /localdisk/massimo/Work/Git/brummer10/guitarix:/mnt/data --security-opt seccomp=/home/massimo/Stuff/docker/personality.json ubuntu bash

where personality.json is the original default.json patched in this way

--- default.json 2020-04-15 13:13:08.948020031 +0200 +++ personality.json 2020-04-15 13:13:08.948020031 +0200 @@ -227,6 +227,7 @@ "open", "openat", "pause",

• "personality", "pipe", "pipe2", "poll",

I've then retried build-plug.py

(guitarix) root@a64fa33012ad:/mnt/data/trunk/tools/ampsim/DK/gschem-schematics# ./build-plug.py -i VoxWahV847.sch --buildlv2 -n VoxWahV847 [...] Traceback (most recent call last): File "../../..//dsp2cc", line 2329, in main() File "../../..//dsp2cc", line 2309, in main options) File "../../..//dsp2cc", line 579, in init self.faust_version = headvers.group(3) AttributeError: 'NoneType' object has no attribute 'group' Error, see message above

So the problem seems to be in the faust code parser. I've added some prints around line 579 of dsp2cc

type(self) = <class 'main.Parser'> type(self.lines) = <type 'generator'> line0 = //----------------------------------------------------------

line1 = // name: "VoxWahV847"

line2 = //

line3 = // Code generated with Faust 0.9.95 (http://faust.grame.fr)

line4 = //---------------------------------------------------------- ... headvers = None

so it seems every line in self.lines has some data and the string "Code generated with Faust" is present so maybe the problem is in skip_until method?

Unfortunately there seem to be some incompatibilities with the Faust version. Easiest solution would be to switch the container to debian:10 as base (this is current Debian stable). And you have to add lv2-dev to your package list.

The Faust version of Ubuntu 18.04 doesn't work any more with the current dsp2cc. I tried to check out an older version of dsp2cc, but that doesn't work with the current skeleton file (missing definition of the faustpower template). Instead of getting all this fixed and support multiple versions, I'd rather document a working container configuration.

[MaxPayne86]

Thank you @adegert, I'm already appending my docker configuration to README.developers, once I get a working setup I'd be happy to send a pull request. Yeah keeping an experimental tool cross-compatible among different distributions and dependencies versions considering everything nowadays is a moving target is time consuming that's why I love containers ;)

[MaxPayne86]

UPDATE: I have tried with debian:10. The version of faust installed now is

2.14.4~repack2-1

however, I still get the error in dsp2cc

Traceback (most recent call last):
  File "../../..//dsp2cc", line 2336, in <module>
    main()
  File "../../..//dsp2cc", line 2316, in main
    options)
  File "../../..//dsp2cc", line 586, in __init__
    self.faust_version = headvers.group(3)
AttributeError: 'NoneType' object has no attribute 'group'

this time in self.lines I can see

line2 = Code generated with Faust 2.14.4 (https://faust.grame.fr)

maybe it's a bug of dsp2cc at this point? I'm using latest guitarix version caae92. Or maybe the version of Faust is still non aligned with the parser dsp2cc? Can we identify the minimal version that is working with this script? Thanks

[brummer10]

current git use faust 2.15.11. Have you checked if the faust code is generated in trunk/tools/ampsim/DK/dkbuild/VoxWahV847/?

[adegert]

Am Do., 16. Apr. 2020 um 13:01 Uhr schrieb Massimo Pennazio notifications@github.com:

UPDATE: I have tried with debian:10. The version of faust installed now is

2.14.4~repack2-1

however, I still get the error in dsp2cc

Traceback (most recent call last): File "../../..//dsp2cc", line 2336, in main() File "../../..//dsp2cc", line 2316, in main options) File "../../..//dsp2cc", line 586, in init self.faust_version = headvers.group(3) AttributeError: 'NoneType' object has no attribute 'group'

this time in self.lines I can see

line2 = Code generated with Faust 2.14.4 (https://faust.grame.fr)

maybe it's a bug of dsp2cc at this point? I'm using latest guitarix version caae92. Or maybe the version of Faust is still non aligned with the parser dsp2cc? Can we identify the minimal version that is working with this script? Thanks

This is strange, because before I wrote my answer I checked that it really works. I used the current gtk3 branch (cdb0278d), and I can't find a commit caae92.

[adegert]

attached Dockerfile and executable script docker-dkbuilder (meant as documentation / demo). Works for me with current master and gtk3 branch. Docker version 19.03.8

[adegert]

please remove .txt extension Dockerfile.txt docker-dkbuilder.txt

[MaxPayne86]

Your Dockerfile file works, thanks a lot.