search

openSUSE / osc

The Command Line Interface to work with an Open Build Service
http://openbuildservice.org/
GNU General Public License v2.0
169 stars 185 forks source link

Unproccessed node error with projects that contain build disable #1605

Open Thaodan opened 3 months ago

Thaodan commented 3 months ago

Describe the bug A clear and concise description of what the bug is.

To Reproduce Steps to reproduce the behavior:

  1. Have OBS project containing a setting in _meta such as:
<build>
  <disable package="foobar"/>
</build>
  1. Run command osc --debug -H build <repo> <arch>
  2. See error

Expected behavior A clear and concise description of what you expected to happen.

Screenshots, console outputs If applicable, add screenshots or console outputs to help explain your problem.

Additional context

> osc --debug -H build latest_i486 i586 hostarch: x86_64 arg_arch: i586 arg_repository: latest_i486 arg_descr: None DEBUG: makeurl: /source/home:thaodan:devel:hw:sony:edo/_meta? DEBUG: ---------------------------------------- DEBUG: GET example.com/source/home:thaodan:devel:hw:sony:edo/_meta DEBUG: send: b'GET /source/home:thaodan:devel:hw:sony:edo/_meta HTTP/1.1\r\nHost: build.example.com\r\nAccept-Encoding: identity\r\nuser-agent: osc/1.8.3\r\nAccept: application/xml\r\n\r\n' DEBUG: reply: 'HTTP/1.1 401 Unauthorized\r\n' DEBUG: header: Server: nginx DEBUG: header: Date: Thu, 08 Aug 2024 07:46:18 GMT DEBUG: header: Content-Type: text/html DEBUG: header: Content-Length: 172 DEBUG: header: Connection: keep-alive DEBUG: header: WWW-Authenticate: Basic realm="Please enter build.example.com account details" DEBUG: send: b'GET /source/home:thaodan:devel:hw:sony:edo/_meta HTTP/1.1\r\nHost: build.example.com\r\nAccept-Encoding: identity\r\nuser-agent: osc/1.8.3\r\nAccept: application/xml\r\nAuthorization: Basic YmJpZGFyOmlmKE5VTEwpcGFuaWM5OSgpOw==\r\n\r\n' DEBUG: reply: 'HTTP/1.1 200 OK\r\n' DEBUG: header: Server: nginx DEBUG: header: Date: Thu, 08 Aug 2024 07:46:18 GMT DEBUG: header: Content-Type: application/xml; charset=utf-8 DEBUG: header: Transfer-Encoding: chunked DEBUG: header: Connection: keep-alive DEBUG: header: Cache-Control: max-age=0, private, must-revalidate DEBUG: header: Vary: Accept-Encoding DEBUG: header: X-XSS-Protection: 1; mode=block DEBUG: header: X-Opensuse-Runtimes: {"view":0.0720880925655365,"db":1.6176399999999997,"backend":0,"xml":0} DEBUG: header: X-Request-Id: 6112adb2-2996-435a-8c41-777706185de2 DEBUG: header: X-Frame-Options: SAMEORIGIN DEBUG: header: X-Runtime: 0.014011 DEBUG: header: X-Content-Type-Options: nosniff DEBUG: header: ETag: W/"df87986fb6aba6c979720d73e06cd2b6" DEBUG: header: Status: 200 OK DEBUG: header: Cache-Control: public DEBUG: header: Strict-Transport-Security: max-age=15768000 Traceback (most recent call last): File "/usr/bin/osc", line 33, in sys.exit(load_entry_point('osc==1.8.3', 'console_scripts', 'osc')()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/osc/babysitter.py", line 227, in main sys.exit(run(commandline.OscMainCommand())) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/osc/babysitter.py", line 70, in run prg.main(argv) File "/usr/lib/python3.11/site-packages/osc/commandline.py", line 565, in main exit_code = cmd.run(args) ^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/osc/commandline.py", line 249, in run return cmd.run(args) ^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/osc/commandline.py", line 509, in run return self.func(args.command, args, *args.positional_args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/osc/commandline.py", line 7557, in do_build args = self.parse_repoarchdescr(args, opts.noinit or opts.offline, opts.alternative_project, False, opts.vm_type, opts.multibuild_package) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/osc/commandline.py", line 7168, in parse_repoarchdescr repositories = list(get_repos_of_project(apiurl, project)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/osc/core.py", line 4011, in get_repos_of_project project_obj = obs_api.Project.from_api(apiurl, prj) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/osc/obs_api/project.py", line 115, in from_api return cls.from_file(response, apiurl=apiurl) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/osc/util/models.py", line 579, in from_file return cls.from_xml(root, apiurl=apiurl) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/osc/util/models.py", line 719, in from_xml entry = klass.from_xml(node, apiurl=apiurl) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/site-packages/osc/util/models.py", line 772, in from_xml cls._remove_processed_node(None, root) File "/usr/lib/python3.11/site-packages/osc/util/models.py", line 632, in _remove_processed_node raise RuntimeError(f"Node {node} contains unprocessed attributes {node.attrib}") RuntimeError: Node contains unprocessed attributes {'package': 'droidmedia'}
dmach commented 2 months ago

The schema in OBS says that package attribute is not allowed (only arch or repository are):

project.rng

        <optional>
          <element ns="" name="build">
            <ref name="flag-element"/>
          </element>
        </optional>

obs.rng

  <define ns="" name="flag-switch">
    <optional>
      <attribute name="repository">
        <data type="string" />
      </attribute>
    </optional>
    <optional>
      <attribute name="arch">
        <ref name="build-arch" />
      </attribute>
    </optional>
  </define>

  <define ns="" name="flag-element">
    <zeroOrMore>
      <optional>
        <element name="enable">
          <ref name="flag-switch"/>
        </element>
      </optional>
      <optional>
        <element name="disable">
          <ref name="flag-switch"/>
        </element>
      </optional>
    </zeroOrMore>
    <empty/>
  </define>

Let me check with the OBS developers if the schema doesn't need an update...

dmach commented 2 months ago

I'm even unable to add such xml chunk to project meta in the OBS webui (the latest snapshot from git). Are you really sure that this is a valid attribute, everything indicates that most likely it's not.

keto commented 2 months ago

The per package build flags in project meta are a custom feature in our OBS, so not exactly a bug in osc.

Though it seems that the new XML model parsing and validation in osc is a bit problematic in that sense. Isn't backwards and forwards compatibility with the server and client quite difficult if this kind of things are hard coded on the client side?

dmach commented 2 months ago

I may need to re-think the object model, maybe only manipulate the underlying XML instead of parsing it and assembling from scratch when finished.

Didn't you think about submitting your change upstream? That would close the gap a little.

keto commented 2 months ago

We are quite a bit behind upstream OBS, on 2.9, and haven't had time to update and rebase the patches. That is the main reason for not submitting them upstream. But also for this particular per package build flag feature, I'm not sure how useful it really is. The reasons why we have it can be handled in other ways, which probably would also make things easier in general.

Thaodan commented 2 months ago

Personally I think enforcing a strict XML model is wrong, I agree with Pami on that.

About the per package disable flags inside the prjconf: Since they could also be added to the package itself I'm not sure how much difference it does make in the end to put them in the prjconf. But if the patch is useful to upstream we could submit it.

dmach commented 2 months ago

The easiest and fastest fix is probably extending osc to support your use case. I'll document that the attribute is a private extension and may not validate on the server.

Working with XML differently requires a lot of time and effort and is not on my priority list at the moment. If this becomes a common problem, I'll return to it in the future.

dmach commented 2 months ago

@Thaodan @keto I spoke to the OBS developers and they would prefer you to use the existing OBS features:

Would any of these options work for you? If not, why?

Thaodan commented 2 months ago

Daniel Mach @.***> writes:

@Thaodan @keto I spoke to the OBS developers and they would prefer you to use the existing OBS features:

  • Disable package builds in package meta
  • The latest version of OBS supports project config directive BuildFlags: onlybuild:<package>

Would any of these options work for you? If not, why?

The first option could work. We have to discuss this internally and report back.

Thanks for helping so far.