ibek develop and permissions

gilesknap commented 11 months ago

Working with docker you do not have permission to write to /epics as its make by root and you are 'vscode'.

This is kind of nice as it gives us the 'prod' view of support modules built in the container.

When attempting to manually do a ibek support develop (as yet not implemented) in order to work on adding new PVI definitions I did this.

sudo mv /epics/support/ADCore /repos/ADCore
sudo chown vscode /repos/ADCore
sudo rm /epics/support/ADCore
sudo ln -s /repos/ADCore /epics/support/ADCore

Now I want to do /epics/ioc-adaravis/ibek-support/ADCore/install after generating some new PVI defs in /epics/ioc-adaravis/ibek-support/ADCore`.

However there are still the following problems:

the install script tries to do apt installs and is not root
the install script needs write access to
- /epics/support/configure/*
- /epics folder to re-create ibek-defs folder
- /epics/ibek-defs/ (to re-create symlinks into ibek-support subfolders)

All the above were fixed with some chmods but the apt install one required an edit to install.sh.

simply running install.sh under sudo seems like the easiest option but root would then need the venv activated (or the python in its path to be precise).

This is just some notes to remind me what ibek support develop might need to do. Also note that once we implement read-only 'prod' in the container some of this will also affect podman.

TODO: make up a nice tidy way of dealing with all this fiddlyness.

gilesknap commented 11 months ago

For the purpose of working on screens only (which only updates ibek-support) I have over-complicated things because having added new *.pvi.device.yaml we need only do ibek support generate-links ADCore.

However, the above still stands if you want to make changes to a support module and compile it.

GDYendell commented 11 months ago

This isn't entirely relevant, but do we have to assume where the developer's editable mount is, or could it be part of the devcontainer config? For example, for a while I have mounted things at /workspace inside devcontainers (I am not particularly wedded to that name, I think just I copied it from here), so this wouldn't work for me currently because it uses /repos.

I think the answer is probably yes we do have to assume where they will be, but it would be nice to not be opinionated and let the developer have some control over this.

GDYendell commented 11 months ago

Ah, you mention some of the difficulties around this in #134 ...

gilesknap commented 11 months ago

I am currently a fan of /repos because:

it means that every generic IOC has an identical devcontainer.json - I have tried to make as many of the files in a generic IOC repo agnostic of which repo they are in - that way, when I make useful updates to the ioc-template 'source of truth' repo, it is trivial to propagate the new things to other generic IOC repos
its nice and straight-forward, and makes the tutorials and documentation easy to write see https://epics-containers.github.io/main/user/tutorials/dev_container.html#id1
it makes other peoples ioc-XXX easy to work on.
By enforcing a single root for all project folders we avoid having to restart the container to remount new host folders
typing cd /repos/my-module is easier than /scratch/defid/work/mymodule or whatever

Despite all these things, there is a conflict of interest when it comes to publishing bob files and their links outside of the container as per #134. Now I'm hoping to come up with some clever way of supporting #134 needs and still keeping /repos/

gilesknap commented 11 months ago

in fact I think I need to re-ephasize point 3. above.

3. it makes other peoples ioc-XXX easy to work on.

When you make a choice about where your extra projects sit in your host file system and you want to use them inside the dev-container you are required to:

stop the devcontainer
edit the devcontainer.json to add a new mount point in.
restart the devcontainer
remember not to check that above change in because it will be broken for everyone else.

Conversely /repos says - you can add folders that are peers to your generic IOC repo into your workspace as you like - that's it. It's restricted to peer folders but its way easier and also probably how most people work anyway.

GDYendell commented 11 months ago

Yeah I think these are all good reasons and it is important to make it as easy as possible to be able to work with each others projects. I think I don't find /repos very descriptive of what it does, which is to allow you to mount things from your host into the container. The directories don't necessarily have to be repos, and it doesn't include all repos.

coretl commented 11 months ago

Explicit mount in devcontainer had the benefit of flagging a change in git. Is there a similar change if we mount in all of repost?

GDYendell commented 11 months ago

If you add folder to workspace a directory in the mount while inside the devcontainer you still get the git integration.

gilesknap commented 11 months ago

@coretl what @GDYendell said. I'm using generic IOC devcontainers all the time now and just add what I want from /repos like you and I always do with /workspace. git integration fully works and it's a tidy workflow.

coretl commented 11 months ago

I mean how do we remember that we're "running from work". If you have to edit .devcontainer then you dirty the repo, which is a good reminder. Do we get the same with /repos? Do we care?

coretl commented 11 months ago

Yeah I think these are all good reasons and it is important to make it as easy as possible to be able to work with each others projects. I think I don't find /repos very descriptive of what it does, which is to allow you to mount things from your host into the container. The directories don't necessarily have to be repos, and it doesn't include all repos.

how abou /workspace?

GDYendell commented 11 months ago

I mean how do we remember that we're "running from work".

Ah. I wasn't thinking so much about knowing that we are running from work as I was that the changes are saved on the host file system if you do forget. I am not sure at what point in the development cycle it would become apparent that you are relying on local changes to a support module and need to push them somewhere. I guess the generic ioc build will fail in CI? Is that late enough that it would be annoying or is that fine?

gilesknap commented 11 months ago

I mean how do we remember that we're "running from work".

We have a solution for this. ibek dev support (todo in #123) will:

copy /epics/support/XXX to /workspace/XXX
or optionally use an existing folder in /workspace
update /epics/ioc-xxx/RELEASES.local (location to be finalised) to point at /workspace/XXX
make sure /epics/ioc-xxx/RELEASES.local is included in /epics/support/configure/RELEASE (the global RELEASE file)

Thus a pushed 'work' version would fail an tell you its looking for the support module in /workspace

(assumes I have changes /repos to /workspace which is clearly a good thing to do - I wanted to dissociate from the vscode default but on reflection that is pointless and a bad idea)

coretl commented 11 months ago

Ok, this is probably fine then

gilesknap commented 7 months ago

I believe this has been resolved with change to /workspaces for peer folders and additional work to set ownership when using docker.

epics-containers / ibek

ibek develop and permissions #142