siemens / kas

Setup tool for bitbake based projects
MIT License
352 stars 144 forks source link

Clone a repository without using it as a layer #16

Closed rapgenic closed 4 years ago

rapgenic commented 4 years ago

Hi, first of all I wanted to thank you for this tool, it makes maintaing yocto projects a lot easier (a lot better than repo and custom init scripts)!

I am currently building a project which does not use poky, but openembedded-core instead. Everything seems to work fine, with the exception that bitbake needs to be cloned separately (poky includes it, but openembedded-core does not). If I add it to the kas-project.yml the repository is correctly cloned and kas executes normally, however when starting bitbake it complains that the folder "bitbake" (which is the clone destination of bitbake repository) does not contain the layer.conf file.

This is due to the fact that the bitbake repository is not a layer, still it is automatically included by kas in the generated bblayers.conf file.

This is confirmed by removing the bitbake repo from the kas-project.yml file after it has been first cloned: in this case it starts building with no complaining!

Is there a way to simply clone a repository but avoid including it in the bblayers.conf file?

jan-kiszka commented 4 years ago

Good to hear that it's working well!

The pattern you are looking for is

layers:
  .: excluded

I think we should improve to documentation to explicitly describe the bitbake scenario. Patches welcome!

rapgenic commented 4 years ago

Thank you for your quick reply!

Unfortunately not working yet, could you please take a look at the following yml?

header:
  version: 8

machine: stm32mp1

distro: openstlinux-weston

target: st-image-core

local_conf_header:
  meta-protech: |
    USER_CLASSES ?= "buildstats image-mklibs image-prelink"
    PATCHRESOLVE = "noop"
    ACCEPT_EULA_stm32mp1 = "1"

repos:
  # Local layer
  meta-protech:

  # Yocto layers
  openembedded-core:
    url: "http://git.openembedded.org/openembedded-core"
    refspec: cd7cf933b3235560ec71576d8f3836dff736a39f
    layers:
      meta:
  meta-openembedded:
    url: "http://git.openembedded.org/meta-openembedded"
    refspec: 436cf0aa2b2802da706588d4daa1a8240d172df8
    layers:
      meta-oe:
      meta-gnome: 
      meta-xfce: 
      meta-initramfs: 
      meta-multimedia: 
      meta-networking: 
      meta-webserver: 
      meta-filesystems: 
      meta-perl: 
      meta-python: 
  meta-qt5:
    url: "https://github.com/meta-qt5/meta-qt5"
    refspec: 1520d5b2b2beec5e1c3209d3178219e93ef08bca
  meta-timesys:
    url: "https://github.com/TimesysGit/meta-timesys"
    refspec: 31d5742812e2b21bb67b7379f7f6f74689b32074
  bitbake:
    url: "http://git.openembedded.org/bitbake"
    refspec: 6b045e074c6fea97d4e305a5a3c8bf82135d95eb
    layers:
      .: excluded
  meta-st-openstlinux:
    url: "https://github.com/STMicroelectronics/meta-st-openstlinux"
    refspec: f023f39b13fa13f6391be5fbc10fb2bf7369402a
  meta-st-stm32mp:
    url: "https://github.com/STMicroelectronics/meta-st-stm32mp"
    refspec: 8055ad11c92144e4a147a00834de53a2de21e42d
  meta-st-stm32mp-addons:
    url: "https://github.com/STMicroelectronics/meta-st-stm32mp-addons"
    refspec: ef975cf09260b1e30a21384d2b1e751e6729d219

This is the error message:

ERROR: Unable to parse /home/andrea/Devel/stm32mp1/protech/bitbake/conf/layer.conf: [Errno 2] file /home/andrea/Devel/stm32mp1/protech/bitbake/conf/layer.conf not found
rapgenic commented 4 years ago

Status update: digging through the code I found that this behaviour seems to be due to a bug in the source that prevents

layers:
  .: excluded

from being effective. Here I see that when reading the repo layers and the _layers variable is empty it returns the path of the repository as the single layer https://github.com/siemens/kas/blob/a48ae6a7fa8d9cc56f4dfca1b6a75b48e77e4f17/kas/repos.py#L52-L56

And when the instance of the Repo class is created by Repo.factory from the function ctx.config.get_repos() called in the WriteBBConfig command it obviously gets an empty layers variable:

https://github.com/siemens/kas/blob/a48ae6a7fa8d9cc56f4dfca1b6a75b48e77e4f17/kas/repos.py#L83-L91

Because the only defined layer . is excluded thus leaving the list empty, and returning the full path of the repository as a layer.

jan-kiszka commented 4 years ago

I could have sworn someone modeled that before, but I can confirm you analysis. Let's carry this to kas-devel@googlegroups.com so that we have the full audience, also when discussing a solution.

jan-kiszka commented 4 years ago

Ha, looks like the solution is simple:

diff --git a/kas/repos.py b/kas/repos.py
index 56bc54f..0031814 100644
--- a/kas/repos.py
+++ b/kas/repos.py
@@ -51,9 +51,7 @@ class Repo:

     def __getattr__(self, item):
         if item == 'layers':
-            if not self._layers:
-                return [self.path]
-            return [self.path + '/' + layer for layer in self._layers]
+            return [os.path.join(self.path, layer) for layer in self._layers]
         elif item == 'qualified_name':
             url = urlparse(self.url)
             return ('{url.netloc}{url.path}'
@@ -84,7 +82,7 @@ class Repo:
         """
             Returns a Repo instance depending on params.
         """
-        layers_dict = repo_config.get('layers', {})
+        layers_dict = repo_config.get('layers', {'': None})
         layers = list(filter(lambda x, laydict=layers_dict:
                              str(laydict[x]).lower() not in
                              ['disabled', 'excluded', 'n', 'no', '0', 'false'],
jan-kiszka commented 4 years ago

See also https://groups.google.com/forum/#!topic/kas-devel/vHHO_UcqNKY. Please drop a tested-by there if things work for you.