Designing a standard distribution file path

riking commented 4 years ago

Because so many of the relevant file formats use absolute paths, we should come up with a canonical distribution / install path so that these files are more portable across installations.

Proposals:

First, define a standard lookup path of /usr/share/pdks/$tech plus /usr/local/share/pdks/$tech plus $XDG_DATA_HOME/pdks/$tech plus $HOME/.local/share/pdks/$tech.
Tools should define their own per-tech configuration file layout below this directory, with liberty/LEF+DEF/GDS on a standard path
... This doesn't actually solve the absolute path problem as stated, but it does open up the path to doing so.

Do not action this issue until after November 30 :)

RTimothyEdwards commented 4 years ago

Roger that. I have also discussed this briefly with @mithro ; we need a standardized install path that the open source EDA tools can agree upon. /usr/share/pdks/ is a reasonable choice.

RTimothyEdwards commented 3 years ago

Now that the first MPW is off and running, I have worked on normalizing open_pdks around a default location of /usr/share/pdks and reworking some of the tools to make use of that. With the update I made to magic today, magic can automatically determine how to bring up the whole PDK from the technology name in a .mag file as it reads it in on startup.

The order of precedence that I'm working with is: (1) Environment variable or Tcl variable PDK_PATH (or PDKPATH) that points to one specific technology (e.g., sky130A) (2) Environment variable or Tcl variable PDK_ROOT (or PDKROOT) that points to directory that has one or more PDKs installed somewhere in the directory tree below it, and (3) The open_pdks default install path /usr/share/pdks/.

Currently I am not intending to support some hidden (or otherwise) subdirectory of $HOME other than what can be done by setting PDK_ROOT or PDK_PATH in the environment.

mithro commented 3 years ago

Do you want to put the proposal in a public Google doc somewhere? That way we can have something which describes the current proposal?

riking commented 3 years ago

and (3) The open_pdks default install path /usr/share/pdks/.

Currently I am not intending to support some hidden (or otherwise) subdirectory of $HOME other than what can be done by setting PDK_ROOT or PDK_PATH in the environment.

Having something under $HOME is important for users that do not have local root access on the system -- the use of $XDG_DATA_HOME and $HOME/.local/share as local overrides for /usr/share has been established in the XDG Base Directory Specification for over a decade and is generally recognized as a Good Idea™.

Don't currently have the energy to write this up in a doc -- it's possible I'll do it over the next few days but don't wait too long for me.

olofk commented 2 years ago

I'm looking at making PDKs relocatable as part of https://github.com/fusesoc/docker-openlane-sky130/issues/1 together with @donn and @proppy If absolute paths are required, then I think a single env var is the way to go and enforcing that to be set

Just did a look through one of the built libraries and came to the following conclusions and proposals

libs.tech/magic/sky130A.magicrc sets n overridable PDKPATH. Leave as is

[ ] libs.tech/qflow/*.sh contains absolute paths. Use env var here instead
[ ] libs.tech/xcircuit/sky130A.xcircuitrc sets PDK_ROOT. Don't do that and instead infer from $PDKPATH (or vice versa)
[ ] libs.tech/xschem/xschemrc uses absolute path in XSCHEM_LIBRARY_PATH. Use env var instead
[ ] libs.tech/magic/sky130A.tcl uses an absolute path for TECHPATH. Use env var instead

I might very well have missed something but those were the ones I found and they seem very fixable to me

RTimothyEdwards commented 2 years ago

@olofk : The magicrc file does what I prefer, which is to set a default value based on the user's installation, but allow an override from an environment variable.

olofk commented 2 years ago

@RTimothyEdwards Fair enough. I'd say that's up to you and it's possible to override. Let's drop that from the list and focus on the other ones. Any known ones I missed?

Also forgot to ask. There's PDK_ROOT, PDKPATH and TECH.. something. Could we have one and derive the others from that?

RTimothyEdwards commented 2 years ago

@olofk : I do make a distinction between PDKROOT and PDKPATH, the first one being the parent directory of the second one. For historical reasons, magic supports alternative forms with underscores. Because PDKPATH is more specific than PDKROOT, it is always preferred. PDKROOT can always be derived from PDKPATH but not vice versa.

My recommendation is that all tools support PDKPATH and any setup files that use other versions of the variables should be corrected. There are certain instances where PDKROOT is appropriate, though; for example, qflow will generally want to go searching for all PDKs available, since it works with verilog as source and can potentially target any PDK whatsoever as long as it can find it. Openlane/OpenROAD would presumably be the same. Tools like xschem/magic/netgen/etc. that have to start up assuming one and only one PDK will normally just look at PDKPATH.

mithro commented 2 years ago

We should write this down somewhere and publish it.

We can also make the PDK install into PDKROOT directory by default?

olofk commented 2 years ago

@RTimothyEdwards Got it. Thanks for the clarifications. Can qflow meaningfully work with several PDKs in the same run? Or would it be easier for consistency to just launch it multiple times with different PDKPATH like we do with the other tools if we want to use different PDKs?

Also, I added TECHPATH to the list of hardcoded paths to fix

@mithro Not sure I understand what you mean bu installing into PDKROOT by default? The thing I'm looking for here is to replace all hard coded path references in the generated PDK with something relocatable (likely depending on PDKROOT, PDKPATH or both being set.

mithro commented 2 years ago

If I understand correctly,

PDKROOT == /usr/local/share/pdk -- IE Place where all the PDKs are installed.
PDKPATH == /usr/local/share/pdk/sky130 -- IE A specific PDK installed under PDKROOT

olofk commented 2 years ago

Yes, that's the correct relation between PDKROOT and PDKPATH.

mithro commented 2 years ago

@olofk - So by default a PDK should install itself under the PDKROOT directory then?

olofk commented 2 years ago

Edited because I got it wrong first

@mithro Yes. Which means we will have two dirs $PDKROOT/sky130A/libs.ref and $PDKROOT/sky130A/libs.tech in this case

RTimothyEdwards commented 2 years ago

@olofk : It is case-sensitive and it's sky130A and sky130B in the path. Otherwise, that's correct.

RTimothyEdwards commented 2 years ago

@mithro : I'm okay with modifying the configuration script such that in the absence of any command-line option for the install location, if PDKROOT is defined in the environment, then it will configure automatically to install there.

olofk commented 2 years ago

Just to make sure we're not talking past each other, I would like to clarify that installation location and making it relocatable are two different things for me.

Making it relocatable means that any reference to a file path in the generated PDK files will be $PDKPATH/some/dir instead of an absolute path. When we solve that, it doesn't really matter (to me) where it is being installed since the user will just define $PDKPATH to where the files are located on the system. And for users of the PDK manager I'm looking to build, they will just query the pdk-manager to get the correct path, e.g. PDKPATH=$(pdk-config sky130A)

donn commented 2 years ago

Just stressing, Olof: sky130A. Capital A. We start mixing cases and the situation becomes even worse.

olofk commented 2 years ago

@donn Yes, you're right. This is not the time to be sloppy with those things. I updated my original comment as well to avoid anyone from picking up the wrong case

RTimothyEdwards / open_pdks

Designing a standard distribution file path #60