Give projects the ability to set default parent directory

[merlijn-sebrechts]

Eclipse Che version 5.1.2.

I want to set the default "parent directory" of a project/template/example (I'm a bit confused by the naming). I thought the path property of a project specified the default parent directory, but this doesn't seem to be working.

Example project:

[
  {
    "name":"reactive-layer",
    "displayName":"Reactive Layer",
    "path":"/charms/layers",
    "description":"A Basic reactive layer",
    "projectType":"blank",
    "mixins":[],
    "attributes":{},
    "modules":[],
    "problems":[],
    "source":{
      "type":"git",
      "location":"https://github.com/juju-solutions/template-reactive-python.git",
      "parameters":{}
    },
    "commands":[],
    "links":[],
    "category":"Samples",
    "tags":["python","juju"]
  }
]

I'm working on a Juju stack for eclipse Che. This is what the projects directory should look like.

/projects
└── charms
    ├── layers
    │  ├── layer-a
    │  └── layer-b
    ├── interfaces
    │  └── interface-x
    └── builds
        ├── charm-a
        └── charm-b

When creating a "layer" project, it should be put in /projects/charms/layers. When creating an interface project, it should be put in /projects/charms/interfaces etc.

Is is possible to customize Che to have this behavior?

Note that each layer and interface should have its own git repo, has different build commands etc, so they really are different projects.

[TylerJewell]

Hi @galgalesh - thanks for the clear explanation and graphics. Not all issue posters are that clear, so it helps with the diagnosis.

In the Che concept, each "project" has a 1:1 mapping to a repository. A single workspace can have many projects, each mapped to different repositories. I can talk to the engineers, but I am fairly certain tha teach project (repo) has to be bound to the root of the /projects folder.

We do support the concept of multi-module maven projects, which I don't think fits your world. Each folder can be set as a maven module and then be given specific build commands and treated as a folder "with context".

I am not too familiar wtih JuJu but reading about it now. Can you elaborate a bit more on why deep sub-folders are mapped to separate git repositories each? This is the first time we have a seen a single project that has such an elaborate repository mapping. It's not hte first time we have seen a need for flexible command assignment to different contexts.

BTW, please do look up Intelligent Commands in the epic backlog. We are 75% there and you will like it - it will be a huge help to this scenario.

[merlijn-sebrechts]

Hi @TylerJewell Intelligent Commands will indeed be a huge help to this scenario.

From my tests, it seems that Che can handle "projects in projects". You can do this by changing the "parent directory" of a project when you create it. In the Juju case, the "parent directory" of a project depends on the project type (layer or interface), so I'd like the "create project" window to suggest the correct "parent directory" by default.

Short explanation: each Charm, interface and layer is a separate project, but everybody needs a combination of these. It's very common to work on multiple layers and interfaces at the same time but everybody works on a different combination.

More explanation:

• Charms are packages of operations code. A bunch of charms work together to manage your infrastructure, for example, they manage a Kubernetes cluster.
• Charms are created by pulling a bunch of layers and interfaces together. Two Charms with the same interface can communicate with each other and work together.
• Layers specify their dependencies (which can be other layers and interfaces). By building a layer, you pull together the layer and all its dependencies to create a Charm. Dependencies that aren't available locally are pulled from an online repository.
• Each layer and interface has its own git repo, and is created and maintained separately.
• We can't put everything in / because you can have a charm, layer and interface with the same name and the build tools expect separate directories for each.

Your combination of Charms, layers and interfaces matters because that specific combination is what's needed to manage your infrastructure. But each charm, layer and interface is a separate project and other people might use a different combination to power their infrastructure.

PS: If this structure seems alien to you, it's probably because it is the result of an organically grown build system created by sysadmins who have had limited exposure to IDE's and build systems.

[TylerJewell]

Thank you for the detailed explanation. You are correct that this approach to organization is a bit alien to me. We do a decent amount of work with Puppet and Chef, and those systems don't have an approach like this as well!

With the project in project scenario - are you asking for an enhancement? Or did you find a setting in the project creation flow that lets you define the right parent?

[merlijn-sebrechts]

I'm asking for a way that a project can propose a "default" parent directory during project creation. Currently the "parent directory" in the project creation window is always set to /.

It seems like the path field in a json project definition should allow you to do that, but that approach doesn't seem to be working. So either I misinterpret the docs and this is a feature request or the feature is broken and this is a bug report..

[TylerJewell]

@bmicklea @JamesDrummond - for investigation - is this a misunderstanding in the docs, or is this a feature request?

[bmicklea]

I believe this is an enhancement request and I'm labelling it as such for now. My reasoning is that the request is for the default "Create Project" screen to interrogate a projects type and structure and suggest locations for new projects (perhaps by interrogating those projects as well). The only way I think you could do this would be by running some kind of logic after the project had been imported. Regardless it's a new capability beyond what we offer today.

@galgalesh please correct me if I've misinterpreted the request.

[merlijn-sebrechts]

I think you misinterpreted this. I simply want to change the default "parent directory" of a project sample/template.

So when I register the project template, I want to specify what "parent directory" will be suggested when a user creates a project based on that project template. This is currently "/" and I want to change it to "/charms/layers/"

Can you please tell me what the role is of the "path" variable in a sample object? According to the docs[0], this seems to be what I need. However, registering a project template using a sample object with a different "path" doesn't seem to have any effect.

The terminology and inconsistent use in the docs isn't helping here. Note that the project sample docs[1] use "project sample", "project template", "code sample", "wizard-driven template" and simply "template". Then it goes on about "sample object" and "template object".

I understand that Che is moving very fast and things like this aren't the N1 priority, but I feel like I'm swimming against the current here while all I want to do is add support for a new framework...

[0] https://www.eclipse.org/che/docs/workspace/data-model-samples/index.html [1] https://www.eclipse.org/che/docs/workspace/samples/

[JamesDrummond]

@galgalesh This can be done using the IDE project > create project > path .... Recipes that are located in a subdirectory of /projects need an existing folder. So one option I might suggest is creating a git project that defines only the structure.

/projects
└── charms
    ├── layers
    │  ├── layer-a
    │  └── layer-b
    ├── interfaces
    │  └── interface-x
    └── builds
        ├── charm-a
        └── charm-b

Then have all your other projects and define the path like /charms/layers/layer-a in the recipe. Create a .gitignore file for subproject content such as /charms/layers/layer-a/*. The following is a simple example I put together.

{
  "environments": {
    "default": {
      "machines": {
        "dev-machine": {
          "attributes": {
            "memoryLimitBytes": "2147483648"
          },
          "servers": {},
          "agents": [
            "org.eclipse.che.terminal",
            "org.eclipse.che.ws-agent",
            "org.eclipse.che.ssh"
          ]
        }
      },
      "recipe": {
        "location": "eclipse/ubuntu_jdk8",
        "type": "dockerimage"
      }
    }
  },
  "defaultEnv": "default",
  "projects": [
    {
      "links": [],
      "name": "blank",
      "attributes": {},
      "type": "blank",
      "source": {
        "location": "https://github.com/che-samples/aspnet-web-simple.git",
        "type": "git",
        "parameters": {}
      },
      "path": "/blank",
      "description": "",
      "problems": [],
      "mixins": []
    },
    {
      "links": [],
      "name": "test",
      "attributes": {},
      "type": "blank",
      "source": {
        "location": "https://github.com/che-samples/aspnet-web-simple.git",
        "type": "git",
        "parameters": {}
      },
      "path": "/blank/test",
      "description": "",
      "problems": [],
      "mixins": []
    },
    {
      "links": [],
      "name": "test2",
      "attributes": {},
      "type": "blank",
      "source": {
        "location": "https://github.com/che-samples/aspnet-web-simple.git",
        "type": "git",
        "parameters": {}
      },
      "path": "/blank/test/test2",
      "description": "",
      "problems": [],
      "mixins": []
    }
  ],
  "name": "issue-3931",
  "commands": [
    {
      "commandLine": "cd /projects/blank && dotnet restore",
      "name": "blank: update dependencies",
      "attributes": {
        "previewUrl": ""
      },
      "type": "custom"
    },
    {
      "commandLine": "cd /projects/blank && dotnet run",
      "name": "blank: run",
      "attributes": {
        "previewUrl": "http://${server.port.5000}"
      },
      "type": "custom"
    }
  ],
  "links": []
}

Notice that /blank/test/test2 does not work but /blank/test does. This because I first created /blank project and then created a folder inside called /blank/test/.

Ideally you should look into https://git-scm.com/book/en/v2/Git-Tools-Submodules

[JamesDrummond]

@galgalesh Yes were are updating all of our docs still. Thanks for pointing out the template/sample issue. I linked to a new github issue. If you see any documentation you feel needs improvements feel free to post on the eclipse/che-docs repo or even submit a PR :) .

[TylerJewell]

Clever work & nice job, @JamesDrummond - that post is worth of a blog post. Why don't you have Josh add that content as a best practice to an upcoming blog post. Or definitely include that in the project creation docs.

[JamesDrummond]

@TylerJewell Might need to delve in deeper on how https://git-scm.com/book/en/v2/Git-Tools-Submodules work. I'll create an issue in che-docs repo to pick it up and link back here.

@galgalesh I hope this encompasses what you were after. If not, please let us know. We could maybe ask one of our developers who maybe familiar with php juju charm projects.

[merlijn-sebrechts]

Hi @JamesDrummond

This is what I need! However, I can't seem to get to work.

If I copy-paste your example into Workspaces > add Workspace > Workspace Config, the project appears in the IDE, but it doesn't exist on disk and it has no contents. I have that behavior with a clean install of both :stable and :nightly.

[ghost]

@galgalesh just creating a workspace with projects defined in its config isn't enough. You need to physically import them.