Open pridhiviraj opened 7 years ago
I'm completely confused on what you ran to hit this. This is from a regular op-build command? What level? Any user-level changes?
No changes. I just build using regular op-build command, Cloned latest op-build master-next branch and tried building PNOR image for firestone platforms
git clone --recursive https://github.com/open-power/op-build --branch master-next . op-build-env op-build firestone_defconfig && op-build
Able to reproduce second time too.
I ran these exact command and it all worked fine. What OS/machinetype are you running on?
Attach these files from your build tree: op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/hostboot_build_images/temporary_hb.hb.xml op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/openpower_mrw_scratch/FIRESTONE_hb.mrw.xml op-build/output/build/firestone-xml-2494a430656504d014780000adbc2c2e7cfc54ab/firestone.xml
uname -a Linux localhost.localdomain 4.6.7-300.fc24.x86_64 #1 SMP Wed Aug 17 18:48:43 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
I am running on my Laptop having fedora 24 on x86_64 architecture.
@stewart-ibm - Is this the distro that had a broken PERL XML parser that we had to work around?
Patrick Williams notifications@github.com writes:
@stewart-ibm - Is this the distro that had a broken PERL XML parser that we had to work around?
It's possible. I think it was also broken in Fedora 24. An update has gone out, but I'm not sure if it's gone to Fedora 24 also (or just 25) or if it has been applied.
-- Stewart Smith OPAL Architect, IBM.
I also see this error on Ubuntu 17.04 . DISTRIB_DESCRIPTION="Ubuntu 17.04" NAME="Ubuntu" VERSION="17.04 (Zesty Zapus)" ID=ubuntu Following are the xml parser versions which are installed : libxml-libxml-perl/zesty,now 2.0128+dfsg-1build1 amd64 [installed,automatic] libxml-namespacesupport-perl/zesty,zesty,now 1.11-1 all [installed,automatic] libxml-parser-perl/zesty,now 2.44-2build2 amd64 [installed,automatic] libxml-sax-base-perl/zesty,zesty,now 1.07-1 all [installed,automatic] What is the correct xml parser version which should be used?
Apparently this might be occurring on Debian too - mentioned on IRC that someone ran into this.
I've hit this too trying to build for palmetto.
Can we get an answer to which XML parser we need?
All I did was:
$ git clone --recursive https://github.com/open-power/op-build
$ . op-build-env
$ op-build palmetto_defconfig && op-build
What I pulled:
$ git show
commit cbac09a858a7c8773605ab1a36bd1ad1a576fa65 (HEAD -> master, origin/master, origin/HEAD)
Merge: c980f52 00dd65f
Author: Stewart Smith <stewart@linux.vnet.ibm.com>
Date: Wed Nov 8 14:23:08 2017 +1100
Merge branch 'release' with op-build v1.20
Build system details:
$ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
NAME="Debian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
$ uname -a
Linux ka1 4.9.0-0.bpo.2-amd64 #1 SMP Debian 4.9.18-1~bpo8+1 (2017-04-10) x86_64 GNU/Linux
For what it worth this doesn't look like a XML parsing bug. Without delving too deeply into the code, it looks like getEnumerationType()
wants the XML to have:
<enumerationType>
<id>IBSCOM_MCS_BASE_ADDR</id>
....
</enumerationType>
Which I cannot find anywhere...
The problem is that it is falling down an else leg. Basically we can't interpret the value of the attribute as a number so we assume it must be an enumerated constant. In this case it is not an enumeration so we fail.
This is what I see in palmetto.xml
So something in our xml parsing is turning that array value into a big string instead. This attribute is not defined as an array in attribute_types.xml so there is a disconnect between the system mrw and what hostboot expects. We handle this inside of processMrw.pl like this.
my ($base,$node_offset,$proc_offset,$offset) = split(/,/,
$targetObj->getAttribute($target,"IBSCOM_MCS_BASE_ADDR"));
my $i_base = Math::BigInt->new($base);
my $i_node_offset = Math::BigInt->new($node_offset);
my $i_proc_offset = Math::BigInt->new($proc_offset);
my $i_offset = Math::BigInt->new($offset);
my $mcs = $targetObj->getAttribute($target, "MCS_NUM");
#Note: Hex convert method avoids overflow on 32bit machines
my $mcsStr=sprintf("0x%016s",substr((
$i_base+$i_node_offset*$node+
$i_proc_offset*$proc+$i_offset*$mcs)->as_hex(),2));
$targetObj->setAttribute($target, "IBSCOM_MCS_BASE_ADDR", $mcsStr);
So something in a perl update broke this logic such that it is no longer generating valid HB xml. Can you look in your output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/openpower_mrw_scratch/PALMETTO_hb.mrw.xml to see what xml was actually generated? That file is the output from processMrw.pl.
github doesn't like "xml" so "txt" it is...
This is the problem:
<attribute>
<id>IBSCOM_MCS_BASE_ADDR</id>
<default>0x000000000000000N</default>
</attribute>
Should I send a patch to remove those lines? Orrr?
Those lines are generated, there is nothing to patch. The problem is in the processMrw.pl somewhere where it is incorrectly processing the MRW xml, most likely something in the code snippet I pasted above.
I had the same issue and changing line
my $i_offset = Math::BigInt->new($offset);
into
my $i_offset = hex($offset);
helped
It turned out that sometimes i_offset is NaN
If someone has a working patch, feel free to make a pull request
I see this too. Latest op-build on Ubuntu 17.10 ppc64le
perl --version
This is perl 5, version 26, subversion 0 (v5.26.0) built for powerpc64le-linux-gnu-thread-multi
op-build$ grep -r 0x000000000000000N
output/host/powerpc64le-buildroot-linux-gnu/sysroot/hostboot_build_images/temporary_hb.hb.xml: <default>0x000000000000000N</default>
output/host/powerpc64le-buildroot-linux-gnu/sysroot/hostboot_build_images/temporary_hb.hb.xml: <default>0x000000000000000N</default>
output/host/powerpc64le-buildroot-linux-gnu/sysroot/hostboot_build_images/temporary_hb.hb.xml: <default>0x000000000000000N</default>
output/host/powerpc64le-buildroot-linux-gnu/sysroot/hostboot_build_images/temporary_hb.hb.xml: <default>0x000000000000000N</default>
Just a note, that it’s still present with Ubuntu 18.04 LTS.
copy the firestone xml where the common lives
bash -c 'mkdir -p /home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/openpower_mrw_scratch && cp -r /home/pridhiviraj/op-build/output/build/firestone-xml-2494a430656504d014780000adbc2c2e7cfc54ab/* /home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/openpower_mrw_scratch'
generate the system mrw xml
perl -I /home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/hostboot_build_images /home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/hostboot_build_images/processMrw.pl -x /home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/openpower_mrw_scratch/firestone.xml Loading MRW XML: /home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/openpower_mrw_scratch/firestone.xml Use of uninitialized value in new at /home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/hostboot_build_images/processMrw.pl line 664. Use of uninitialized value in new at /home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/hostboot_build_images/processMrw.pl line 665. Use of uninitialized value in new at /home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/hostboot_build_images/processMrw.pl line 666. Use of uninitialized value in new at /home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/hostboot_build_images/processMrw.pl line 664. Use of uninitialized value in new at /home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/hostboot_build_images/processMrw.pl line 665. Use of uninitialized value in new at /home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/hostboot_build_images/processMrw.pl line 666. Creating XML: /home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/openpower_mrw_scratch/FIRESTONE_hb.mrw.xml MRW created successfully!
merge in any system specific attributes, hostboot attributes
/home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/hostboot_build_images/mergexml.sh /home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/openpower_mrw_scratch/"FIRESTONE_hb.system.xml" /home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/hostboot_build_images/attribute_types.xml /home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/hostboot_build_images/attribute_types_hb.xml /home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/hostboot_build_images/target_types_merged.xml /home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/hostboot_build_images/target_types_hb.xml /home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/openpower_mrw_scratch/"FIRESTONE_hb.mrw.xml" > /home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/hostboot_build_images/temporary_hb.hb.xml;
creating the targeting binary
/home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/hostboot_build_images/xmltohb.pl --hb-xml-file=/home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/hostboot_build_images/temporary_hb.hb.xml --fapi-attributes-xml-file=/home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/hostboot_build_images/fapiattrs.xml --src-output-dir=none --img-output-dir=/home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/hostboot_build_images/ --vmm-consts-file=/home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/hostboot_build_images/vmmconst.h --noshort-enums --bios-xml-file=/home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/openpower_mrw_scratch/"FIRESTONE_bios.xml" --bios-schema-file=/home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/hostboot_build_images/bios.xsd --bios-output-file=/home/pridhiviraj/op-build/output/host/usr/powerpc64le-buildroot-linux-gnu/sysroot/hostboot_build_images/"firestone"_bios_metadata.xml [FATAL!] Could not find enumeration with ID of IBSCOM_MCS_BASE_ADDR
package/pkg-generic.mk:216: recipe for target '/home/pridhiviraj/op-build/output/build/firestone-xml-2494a430656504d014780000adbc2c2e7cfc54ab/.stamp_built' failed make[1]: [/home/pridhiviraj/op-build/output/build/firestone-xml-2494a430656504d014780000adbc2c2e7cfc54ab/.stamp_built] Error 1 Makefile:76: recipe for target '_all' failed make: [_all] Error 2 make: Leaving directory '/home/pridhiviraj/op-build/buildroot'