Get Xyce and the PDK playing nice together

mithro commented 3 years ago

Expected Behavior

You are able to use Xyce for doing analog spice simulations inside the skywater-pdk

Actual Behavior

It doesn't work

Comments

I'm creating this bug to track the various issues people have discovered when trying to use Xyce with the skywater-pdk.

pepijndevos commented 3 years ago

From my understanding the blocking issue is that Xyce does not support the scale option. I've seen some talk about removing it, but it appears a tough nut to crack. The scale option only applies to physical sizes, but not to other physical units like capacitance. This, combined with the deep hierarchy makes it very tricky to rewrite all the equations without the scale option. People who know what they are talking about said automated solutions are tricky because of the way it applies in only some contexts, so simply replacing W with W*1e-6 will break calculations in non-scaled contexts.

The other option would be to add support for the scale option to Xyce, but it is thought that they probably have little interest in this admittedly stupid option. Maybe @peshola could comment on this?

Worth noting that Magic also uses some hack where everything is essentially unscaled by 1e6 so that the final result comes out right. So in this sense removing the scale option would improve tool support across the board. It's just a tricky thing to do.

Maybe ngspice itself could be used as a library to do AST transformations? Because it understands the scale option, I imagine it has a bigger chance of success than regex based approaches.

adrianfreed commented 3 years ago

It might be interesting to provisionally add the ngspice scale option to Xyce. This would make it possible to evaluate Xyce compatibility in other areas and in particular evaluate the benefits of Xyce's parallel solver. Both the recent FOSSI talks (OSU cells, OpenRAM) emphasized characterization and simulation bottlenecks.

adrianfreed commented 3 years ago

Here is an excerpt of some of the errors I see after trying to load the library with a .lib TT card. There are issues with relative include file names. I think Xyce might require absolute path names. Their documentation talks of paths relative to the directory of the executable.

Netlist error in file corners/tt/nonfet.spice at or near line 24 Equal sign (=) required between parameter and value in .PARAM statement

Netlist error: Could not find include file

Netlist error in file parasitics/sky130_fd_prmodelparasitic__diode_ps2nw.model.spice at or near line 16 Parameter $ not formatted correctly in .model SKY130_FD_PRMODELPARASITIC__DIODE_PS2NW

sky130_fd_prnfet_01v8_lvttt.pm3.spice

Netlist warning in file all.spice at or near line 2 Unrecognized dot line will be ignored

.ENDL encountered without library name, currently inside .LIB TT

adrianfreed commented 3 years ago

Xyce doesn't support $ end of line comments which are peppered in many of the .param cards of sky130.

This probably causes the "$ not formatted correctly error" and "= sign required" errors above.

adrianfreed commented 3 years ago

The "unrecognized dot line will be ignored error" is the infamous scale line: .option scale=1.0u

adrianfreed commented 3 years ago

The ".ENDL encountered without library name, currently inside .LIB TT" error can be fixed by making sure all the .endl cards are followed by the name of the library they terminate. This fix is compatible with ngspice

adrianfreed commented 3 years ago

It looks like the .option problem is the significant show stopper in terms of the sky130 spice device library.

For the spice decks in the tests folders the problem is that they use the ngspice interactive mode language (the cards without "." in front of them) . This isn't available in xyce.

peshola commented 3 years ago

This statement above, "Xyce doesn't support $ end of line comments", is correct. Xyce uses ; (semicolon) as its in-line comment character.

Karlsefni2012 commented 3 years ago

We have the "scale" option as a feature request in our internal issue tracker. That is something we can expedite.

Karlsefni2012 commented 3 years ago

Also, note that we (Xyce) have a file translation tool (XDM) that can handle some file compatibility issues. In Xyce we use ".options " instead of .option.

Karlsefni2012 commented 3 years ago

I am pleased to announce that I managed to add "option scale" to Xyce as of today. It is still running thru regression testing, but hopefully (knock on wood) will be in the public git repo for Xyce in the near future, possibly next week.

As Xyce uses a slightly different specification for options, the netlist command for "scale" is in this format:

.options parser scale=1.0e-6

Converting it to that format from ".option scale=1.0e-6" is the kind of thing our XDM tool can do, at least in theory.

mkghub commented 3 years ago

I have did some experiments on the spice files without 'option scale'. And have done some test simulations. Manual editing is required for some files, as is stated above. So some regression tests are needed to have a confidence.

There are a few other compatibility issues, after solving the option scale problem. Maybe developers of Xyce can give hints for best solutions?

1. Temperature variable 'temp' variable cannot be used in the equations. A temporal solution is modifying 'temp' to 'temper' in the equations and defining 'temper' as a parameter on the input file. This method is OK for doing simulations on a fixed temperature but not OK for temperature sweep.
1. voltage dependent resistor models Node voltages cannot be used in equations. In XDM manual, it is said that the solution for this problem is using behavioral current source models. For now I don't know how to incorporate both temperature dependence and voltage dependence. If we can neglect the voltage dependence of poly resistors it is OK.
1. Diode model don't support level 3 model In diode model files, level = 3.0 is defined. For Xyce there's only level 1 and 2.
1. Environment variable in include path statement For hspice and specter, an environment variable can be used in include path statements. For Xyce I don't know how to do it. For example .inclue "$sky130_fd_pr/models....." works for hspice and specter but Xyce refuses it. A temporary solution which I am using for now is defining symbolic link under the simulation directory such as 'sky130_fd_pr' and using it in the spice files. .include "sky130_fd_pr/models..."
1. 'vt' variable This is a trivial problem. In one file, 'vt' is used as a parameter variable. This needs to be modified to something like 'vt0'.
1. In line Comments Changing '$' to ';' is OK for ngspice, Xyce, and hspice, etc.
1. Model Binning This is not an issue. The following statement is needed in the input file for using binning models. .options parser model_binning=true

I am doing the tests and modifications for personal interests with the files on the repository https://github.com/mkghub/skywater130_fd_pr_models

Karlsefni2012 commented 3 years ago

Regarding the use of temperature (TEMP or TEMPER) in equations, that is being worked on.

In the meantime, I think it is good to clarify one aspect of equations in Xyce. They are used in a lot of contexts, of course. In some contexts, TEMP is allowed, and in some others it is not. Here are two examples.

In Xyce .paramand .global_param are treated very differently. .param are basically static and cannot contain anything (user variables, voltage nodes, special variables, etc) that might change during a simulation. .global_param, on the other hand, are allowed to change during a simulation. So, you can think of .paramas being kind of like a parser-level string substitution, and a .global_param as being a true variable. A .param is set up during parsing and then not tracked after that.

So, this is legal:

.global_param gp1={2*TEMP}

And this is not:

.param p1={2*TEMP}

Also this is not legal, because temp is a reserved simulator variable:

.param temp=23.0

Behavioral sources (which use equations) can use temperature. For example, in our test suite, see this input file: Xyce_Regression/Netlists/Certification_Tests/BUG_606_SON/bug606son.cir.

Netlist to demonstrate Vt bug
**************************************************************
* This netlist tests SON Bug 606, 654 and 1033.  SON Bug 606
* was about making VT work within expressions.  SON Bugs
* 654 and 1033 are then about making TEMP and VT work
* correctly as "dependent parameters" for all Xyce devices.
**************************************************************

B2 3 0 V={TEMP+273.15}
R3 3 0 1

B3 4 0 V={Vt}
R4 4 0 1
.OP

.step TEMP 27 30 1
.PRINT DC V(3) {TEMP+273.15} {(TEMP+273.15)*(1.3806226e-23/1.6021918e-19)} V(4) {Vt}
.END

This ties into the second issue you raise, about voltage dependent resistor models. As you note the XDM guide suggests replacing something like this:

R1   1  2   ’P1*v(in,out)’

with this:

B1 1 2 I={(V(2)-V(1)/(P1*v(in,out))}’

If you needed to incorporate TEMP into the equation for B1, that should work. It just can't come thru a .param

I should note that we've been doing a LOT of work on expressions in Xyce. We are about to release 7.2, in which the Xyce expression library has been completely replaced with a new one. The new one addresses many, many long standing expression-related issues, including the use of complex numbers.

The new expression in 7.2 partially addresses the difference between .param and .global_param. The new library doesn't treat .param as constants. So, going forward we should ultimately be able to make .param behave the same as .global_param. That isn't quite done yet, as while the new expression library treats .param as non-constant, the rest of the Xyce source hasn't yet been updated to reflect this change. But I expect that this will be fully propagated thru the rest of Xyce over the coming months.

peshola commented 3 years ago

To add to the previous comment, if you look in the Xyce Regression test suite then the tests in Netlists/Certification_Tests/BUG_1203_SON show various ways that the TEMP (or TEMPER in the upcoming Xyce 7.2 release) special variable can used in a .GLOBAL_PARAM in an expression for a device instance parameter. If those examples don't cover your usages then please let us know, as the Xyce Team has been fixing the use of special variables over the past few release cycles.

Issue 4 ("Environment variable in include path statement") is something that we are aware of. I hope to investigate that feature request in the near future, to at least scope out a path for adding it (or something like it) to Xyce.

mkghub commented 3 years ago

Thank you for your response. I have read the above Regression test suite and did test again with Xyce 7.2. Now, I am in the same state as before. Maybe I am missing some points.

For example, the device subckt definition has the parameter .param vc1_end = {vc1_raw_end(1+tc1_voltco(temp-30))}

Since, 'temp' cannot be used, one can use 'temper' instead, as shown below .param vc1_end = {vc1_raw_end(1+tc1_voltco(temper-30))}

In the input file, I have tried the following,

.global_param temper = {temp}

It gives the same error as using 'temp' directly.

How to make the 'temper' variable follow 'temp' exactly, even when we are doing temperature sweep?

peshola commented 3 years ago

Does this syntax work in your netlist?

.global_param vc1_end = {vc1_raw_end(1+tc1_voltco(temp-30))}

As Eric alluded to above, for historical reasons that are being slowly corrected, .PARAM and .GLOBAL_PARAM are not the same in Xyce. Also, .PARAM in Xyce still acts differently than .PARAM in HSPICE. This is also being slowly fixed.

On another note, we are working towards making TEMPER and TEMP into synonyms in Xyce. However, that work is on the most recent version of our master branch available at github, and is not in Xyce 7.2.

mkghub commented 3 years ago

If I use .global_param inside of subcircuit statements, I get the following error. "Attempt to assign global_param inside of subcircuit"

peshola commented 3 years ago

And I assume that it's inconvenient to move the .global_param statement outside of the .subckt statement because some element of the expression vc1_end = {vc1_raw_end(1+tc1_voltco(temp-30))} may be different in different instantiations of the subcircuit in your netlist?

Karlsefni2012 commented 3 years ago

Hello everyone, I wanted to give an update on this issue.

Recently (past few weeks) a number of changes have been pushed to the official public Xyce git repository, which should impact the use of Xyce on the Skywater PDK. There is still more to do, but several of the issues mentioned in this thread have been fixed. The various improvements include:

Model binning is now on by default, and doesn't need to be turned on from the netlist.
.param is much more flexible now, and is allowed to be set to "special" variables like TEMP. Previously only .global_params were allowed to do this, for historical reasons. Most distinctions between .param and .global_param are now gone. (long story there, not really worth getting into here)
.LIB and .INC now support relative paths.

peshola commented 3 years ago

To clarify the relative path fix, various Spicen actually appear to handle this relative path issue in different ways. My research back around Thanksgiving showed that the following was true. Define:

P1 = path relative to the directory of the file with the .INC ( or .LIB) line P2 = path relative to the directory of the top-level netlist.

Then this appears to be true, where "fallback" means that P2 is used if P1 does not exist. Absolute paths are also supported by all four simulators. (I should be verifying what Spectre does, shortly.)

Xyce should now handle paths for .INC and .LIB files like HSPICE does. The fix should work on Linux, OSX and Windows, and was pushed to github last weekend. If folks still have concerns about this feature request then please post a comment here. If we need to make Xyce support both the HSPICE and ngspice behaviors then that should be doable, but it might require a command-line switch.

GlenHertz commented 3 years ago

FYI. If my memory serves me right HSPICE will search for:

Relative to dir of file
Relative to dir of top netlist
Relative to dir of where it was launched from

It will also look for .gz extensions at each step if they exist (so .inc foo.sp will look for foo.sp and then foo.sp.gz).

Karlsefni2012 commented 3 years ago

@GlenHertz thanks! that is helpful. We'll double-check those use cases for HSPICE. What is currently in the development branch may not do all of that yet.

peshola commented 3 years ago

@GlenHertz You were correct.

Both "relative to dir of top netlist" and "relative to dir of where it was launched from" work in HSPICE. The relative priority, between options (1), (2) and (3), is also correct in your comment. I did not test the .gz extension feature.

Xyce 7.2, and earlier, makes an implicit assumption, when handling .LIB and .INC file paths, that the "dir of the top netlist" is the same as the "dir of where Xyce was launched from". So, your options (2) and (3) don't always work correctly in Xyce 7.2, if that assumption is false.

Both (2) and (3) in your comment should now work with what is currently in the development branch of Xyce available on github. This should be true even if the "launch directory" and the "dir of the top netlist" are different. However, I'm still working on test cases to verify that is true in general. (For the internal Sandia readers, I may also change the new function handleIncludeFilePath() in the file N_IO_ParsingHelpers.C to more explicitly handle (3).)

GlenHertz commented 3 years ago

Oh, good to hear my memory is still OK. A quick tip to figure out what a program does is to run in on Linux with:

strace -f -e open,stat ngspice path/to/netlist.sp

And just use a missing file for the .lib or .include and it will print out everywhere it looks so you don't have to test it a bunch for different scenarios and guess. If you want to see more output then remove the "-e open,stat" part.

peshola commented 3 years ago

@GlenHertz . Thanks for the pointer to strace. A useful tool.

My comment from a few days ago was wrong. The version of Xyce currently available at github only looks in (1) "Relative to dir of file" and (3) "Relative to dir of where Xyce was launched from". I have a version of Xyce, on a local branch, that also looks in (2) "Relative to dir of top netlist". So, that feature might should also be available, on github, in a few weeks after some more refinement and testing.

Adding a command-line switch to mimic any differences, in the file search path behavior, between HSPICE and ngspice is doable. But I'm not currently planning to do that unless there's a clear need for it.

peshola commented 3 years ago

Based on the output from strace, it appears that ngspice's search order (for ngspice-33) is:

"Relative to dir of where ngspice was launched from"
/install/path/share/ngspice/scripts
Relative to dir of include file

So, that is actually quite different from HSPICE.

umarcor commented 3 years ago

A few days ago, I added Xyce to hdl.github.io/containers, so there is now a container that users can try for testing/using Xyce. It's available as hdlc/xyce or ghcr.io/hdl/debian-buster/xyce.

# docker run --rm -it hdlc/xyce Xyce -v
Xyce Release 6.12-opensource

As you can see, it is currently based on the versions I used in december 2019, when testing co-simulation betwen GHDL and Xyce (https://ghdl.github.io/ghdl-cosim/vhpidirect/examples/vffi_user.html#xyce): https://github.com/hdl/containers/blob/main/debian-buster/xyce.dockerfile#L51-L54. However, since all other tools in hdl/containers are built from main/master/head branches, I'd like to update Xyce too.

@peshola, since Xyce is very dependent on Trilinos, what procedure do you suggest? Trilinos 12.12.1 is still recommended in https://xyce.sandia.gov/documentation/BuildingGuide.html#instTrilinos. Should I stick to that and pick master of Xyce only?

peshola commented 3 years ago

@umarcor I'm not the best Xyce person to discuss building Xyce within docker. However, I will ask someone on the team to engage with you, We will be releasing Xyce 7.3 in the next month, or so. So, it may be a good time to think about this topic.

umarcor commented 3 years ago

@peshola, in this case, building within docker is not different from building it on a Debian host: https://github.com/hdl/containers/blob/main/debian-buster/xyce.dockerfile#L27-L118. I might convert that to a regular shell script, should someone on the team want to have a look at it. The issue after bumping to 7.2 is that some dependency/tool seems to be missing. Note /bin/bash: no: command not found in the following log (https://github.com/hdl/containers/runs/2320141138):

2021-04-12T03:24:00.3673607Z #9 1257.0 Making install in ExpressionSrc
2021-04-12T03:24:00.3704422Z #9 1257.0 make[4]: Entering directory '/tmp/build/xyce-build/src/UtilityPKG/ExpressionSrc'
2021-04-12T03:24:00.3706362Z #9 1257.0 no -oExpressionParser.cxx ../../../../Xyce/src/UtilityPKG/ExpressionSrc/ExpressionParser.yxx
2021-04-12T03:24:00.3712665Z #9 1257.0 no  -i -+ --full  --prefix=exp  -oExpressionLexer.cxx ../../../../Xyce/src/UtilityPKG/ExpressionSrc/ExpressionLexer.l
2021-04-12T03:24:00.3715787Z #9 1257.0 /bin/bash: no: command not found
2021-04-12T03:24:00.3716895Z #9 1257.0 make[4]: *** [Makefile:759: ExpressionParser.cxx] Error 127
2021-04-12T03:24:00.3718075Z #9 1257.0 make[4]: *** Waiting for unfinished jobs....
2021-04-12T03:24:00.3719869Z #9 1257.0 /bin/bash: no: command not found
2021-04-12T03:24:00.3721477Z #9 1257.0 make[4]: Leaving directory '/tmp/build/xyce-build/src/UtilityPKG/ExpressionSrc'
2021-04-12T03:24:00.3722823Z #9 1257.0 make[4]: *** [Makefile:762: ExpressionLexer.cxx] Error 127
2021-04-12T03:24:00.3724353Z #9 1257.0 make[3]: Leaving directory '/tmp/build/xyce-build/src/UtilityPKG'
2021-04-12T03:24:00.3726019Z #9 1257.0 make[3]: *** [Makefile:781: install-recursive] Error 1
2021-04-12T03:24:00.3727013Z #9 1257.0 make[2]: *** [Makefile:938: install] Error 2
2021-04-12T03:24:00.3728393Z #9 1257.0 make[2]: Leaving directory '/tmp/build/xyce-build/src/UtilityPKG'
2021-04-12T03:24:00.3729948Z #9 1257.0 make[1]: *** [Makefile:886: install-recursive] Error 1
2021-04-12T03:24:00.3731292Z #9 1257.0 make[1]: Leaving directory '/tmp/build/xyce-build/src'

Anyway, no rush at all. I just provided this info so you can forward it to whoever. Ping me at your own will.

TBird2001 commented 3 years ago

@umarcor, your inquiry comes at an interesting, perhaps opportune, point. We are in the process of moving from autotools to CMake for our primary build system. In the upcoming 7.3 release of Xyce, CMake build instructions will be provided for the first time. We've tried to make it easier than the autotools approach, so it may be simpler to create the Debian container using CMake. Also, CMake is the future for Xyce, so you will have to move to it at some point. Nevertheless, since it is the first time anyone other than us will be using the CMake system, there is the possibility of bugs (hopefully minor ones).

If you decide to try the CMake, I'd be happy to help you troubleshoot as needed (as it would be good to get some external feedback). The instructions will be in a new INSTALL.md file, which should get pushed to GitHub around next Monday (the 19th).

As to your Trilinos question, we will be using 12.12.1 at least through the 7.3 release. Being better compatible with Trilinos is actually one of the drivers for the move to CMake, as they will be dropping their autotools support soon. Also, an upcoming Trilinos release will have a solver that we hope to leverage. As such, we will eventually start tracking Trilinos more closely, at least in terms of their releases. For the time being, though, you should be able to use the same build of Trilinos 12.12.1 for the next six months, at least.

Regarding the install issue in your second comment, I have no idea what is going on. I've asked for some help from others in our group. Hopefully we will have something soon.

tvrusso commented 3 years ago

The issue after bumping to 7.2 is that some dependency/tool seems to be missing. Note /bin/bash: no: command not found in the following log (https://github.com/hdl/containers/runs/2320141138):


2021-04-12T03:24:00.3673607Z #9 1257.0 Making install in ExpressionSrc
2021-04-12T03:24:00.3704422Z #9 1257.0 make[4]: Entering directory '/tmp/build/xyce-build/src/UtilityPKG/ExpressionSrc'
2021-04-12T03:24:00.3706362Z #9 1257.0 no -oExpressionParser.cxx ../../../../Xyce/src/UtilityPKG/ExpressionSrc/ExpressionParser.yxx
2021-04-12T03:24:00.3712665Z #9 1257.0 no  -i -+ --full  --prefix=exp  -oExpressionLexer.cxx ../../../../Xyce/src/UtilityPKG/ExpressionSrc/ExpressionLexer.l

This suggests that for some reason configure is not finding a usable version of Bison, so the value of "$BISON" is set to "no". It is supposed to exit with a fatal error at configure time in that case and not create any Makefiles, so it should not even be possible to run "make" after that. You are clearly running make after configure only if configure succeeds, so this is fairly puzzling. Somehow it's getting past the failing probe for bison and using "no" as the program name.

Could you please attach your "config.log" file that is produced by configure and which should be present in the build directory where this is happening? (you will have to rename it to have a suffix that Github doesn't refuse to attach, like txt).

tvrusso commented 3 years ago

you please attach your "config.log" file that is produced by configure and which should be present in the build directory where this is happening? (you will have to rename it to have a suffix that Github doesn't refuse to attach, like txt).

I see what the problem is from looking at the output of configure in your link.

There are several things wrong here --- first is that Xyce 7.2 was incorrectly wrapping the configure probe for Bison inside its test for whether the reaction parser was enabled. In versions prior to 7.2 bison was only used for that parser, so that was appropriate, but in Xyce 7.2 a new, mandatory dependence on Bison was introduced and the configure script was not updated to make it a mandatory check. That was fixed in master some time ago.

The result of this mistake in 7.2 is that you cannot build Xyce with the "--disable-reaction_parser" option in that version. You MUST remove that. You can put it back for 7.3 or master if you really don't want the feature.

There is another problem, in that you are not using the "ARCHDIR" feature in the way that our building guide recommends, and seem to be trying to do it by setting an environment variable prior to invoking configure. This sort of works, but prevents the build system from finding some useful files other than libraries and headers that Trilinos creates (this is why you're getting all sorts of warnings about not being able to find "Makefile.export" files). You are setting an environment variable ARCHDIR to the $XYCE_OUTDIR, but you are not passing this to Xyce's configure in a way that lets it work.

In order to build Xyce 7.2 as we recommend, you will need to remove "--disable-reaction_parser" from your configure line, and add "ARCHDIR=$XYCE_OUTDIR" to the configure line (not just setting it via an environment variable):

../Xyce/configure \
      CXXFLAGS="-O3 -std=c++11" \
      LDFLAGS="-Wl,-rpath=$xyceBuildDir/utils/XyceCInterface -Wl,-rpath=$xyceBuildDir/lib" \
      CPPFLAGS="-I/usr/include/suitesparse" \
      ADMS_CXXFLAGS="-O1" \
      ARCHDIR=$XYCE_OUTDIR \
      --disable-verbose_linear \
      --disable-verbose_nonlinear \
      --disable-verbose_time \
      --enable-shared \
      --enable-xyce-shareable

There is no real downside to enabling the reaction parser even though you're not using it. There is also no real point to adding the various explicit "disable" options for verbosity, as they are disabled by default.

It will be optional again in 7.3 and is already optional again in master.'

Finally, the use of ADMS_CXXFLAGS="-O1" should no longer be necessary except on really memory-starved systems, as the ADMS code has been completely revamped as of Xyce 7.1 to generate code that no longer causes compilers to have memory hemmorages.

tvrusso commented 3 years ago

FWIW, the bison/flex issue was fixed in November in commit Xyce/Xyce@44c9788. So any master branch checkout after that commit will not have the issue you report.

I recommend just going right to building Xyce master branch rather than the 7.2 or even 7.3 releases (when the latter comes out in a few weeks). We are pushing the code to github on a mostly weekly basis, and the code only gets pushed to github master branch when it is stable and passing all tests (we skip a week if there are any problems).

Many new fixes targeted directly at Skywater PDK are only present on the master branch at the moment.

As @TBird2001 says, we're still recommending Trilinos 12.12.1 and definitely NOT Trilinos's master branch.

tvrusso commented 3 years ago

I will note also that your docker build is using a Trilinos CMake invocation that we recommended back at release 6.12, but which has not stayed the same, and will change again with the release of 7.3.

Notably, Xyce now requires C++11 support in the compiler and both Trilinos and Xyce must be built with that support. Xyce's master branch configure forces the correct options into the compiler, as will Trilinos if you don't specify otherwise.

Secondly, our recommendations used to suggest building Trilinos with Isorropia and Zoltan support even in serial, and this is pointless as they aren't used by Xyce in serial. We no longer recommend this.

UMFPACK is no longer required for Xyce.

Several new features can be enabled if Stokhos and Amesos2 are built into Trilinos.

The new recommendations are not yet published on our building guide, they will be published at the same time we release 7.3 in a few weeks.

The Trilinos build options we will be recommending for a serial build starting with release 7.3 will be:

cmake \
-G "Unix Makefiles" \
-DCMAKE_C_COMPILER=gcc \
-DCMAKE_CXX_COMPILER=g++ \
-DCMAKE_Fortran_COMPILER=gfortran \
-DCMAKE_CXX_FLAGS="$FLAGS" \
-DCMAKE_C_FLAGS="$FLAGS" \
-DCMAKE_Fortran_FLAGS="$FLAGS" \
-DCMAKE_INSTALL_PREFIX=$ARCHDIR \
-DCMAKE_MAKE_PROGRAM="make" \
-DTrilinos_ENABLE_NOX=ON \
  -DNOX_ENABLE_LOCA=ON \
-DTrilinos_ENABLE_EpetraExt=ON \
  -DEpetraExt_BUILD_BTF=ON \
  -DEpetraExt_BUILD_EXPERIMENTAL=ON \
  -DEpetraExt_BUILD_GRAPH_REORDERINGS=ON \
-DTrilinos_ENABLE_TrilinosCouplings=ON \
-DTrilinos_ENABLE_Ifpack=ON \
-DTrilinos_ENABLE_AztecOO=ON \
-DTrilinos_ENABLE_Belos=ON \
-DTrilinos_ENABLE_Teuchos=ON \
  -DTeuchos_ENABLE_COMPLEX=ON \
-DTrilinos_ENABLE_Amesos=ON \
  -DAmesos_ENABLE_KLU=ON \
-DTrilinos_ENABLE_Amesos2=ON \
 -DAmesos2_ENABLE_KLU2=ON \
 -DAmesos2_ENABLE_Basker=ON \
-DTrilinos_ENABLE_Sacado=ON \
-DTrilinos_ENABLE_Stokhos=ON \
-DTrilinos_ENABLE_Kokkos=ON \
-DTrilinos_ENABLE_ALL_OPTIONAL_PACKAGES=OFF \
-DTrilinos_ENABLE_CXX11=ON \
-DTPL_ENABLE_AMD=ON \
-DAMD_LIBRARY_DIRS="/usr/lib" \
-DTPL_AMD_INCLUDE_DIRS="/usr/include/suitesparse" \
-DTPL_ENABLE_BLAS=ON \
-DTPL_ENABLE_LAPACK=ON \
$SRCDIR

This builds fewer unneeded components and adds the few new ones that enable more useful features in Xyce.

The new guidance for building Xyce 7.3 (which assumes Trilinos built as above and which works with Xyce master branch) will be:

/path/to/Xyce/configure \
CXXFLAGS="-O3" \
ARCHDIR="$HOME/XyceLibs/Serial" \
CPPFLAGS="-I/usr/include/suitesparse" \
--enable-stokhos \
--enable-amesos2 \
--prefix=$HOME/XyceInstall/Serial

Adjust prefix and ARCHDIR to match your needs, and add additional options such as the shared/shareable options to taste.

GlenHertz commented 3 years ago

FYI. We were able to generate binaries for Xyce for Linux. Here are the build scripts: https://github.com/JuliaPackaging/Yggdrasil/tree/master/X/Xyce

It is meant to be installed with Julia (Pkg.add("Xyce_jll")). There is a standalone installer called ygg (https://github.com/fredrikekre/ygg) but is kind of alpha. If you want to use ygg and run into trouble let me know and I can help you out.

peshola commented 3 years ago

The version of Xyce that was pushed to github last weekend now does this for file paths on .INC and .LIB lines. This should be the final change to that feature for the upcoming Xyce 7.3 release.

Relative to dir of file
Relative to dir of top netlist
Relative to dir of where it was launched from

@Karlsefni2012 has made many other improvements to Xyce for Skywater compatibility. We might not summarize them here, until the Xyce 7.3 release candidate is finished this week, or next.

mithro commented 3 years ago

@peshola - I would welcome a release announcement post to the SkyWater mailing lists and IRC channel about Xyce support for SkyWater!

Karlsefni2012 commented 3 years ago

@mithro Xyce 7.3 should be complete in next few weeks. We're finishing up documentation and final approvals. We'll be sure to post the announcement in as many Skywater-aligned places as we can. :)

mithro commented 3 years ago

@Karlsefni2012 - Very supportive of doing that!

dan-fritchman commented 2 years ago

At present getting this combo to work together appears to require (at least) the Perl-fu suggested by @mguthaus here: https://github.com/RTimothyEdwards/open_pdks/issues/124

umarcor commented 2 years ago

FTR, container image gcr.io/hdl-containers/xyce, hdlc/xyce or ghcr.io/hdl/xyce:latest (same content uploaded to three registries) is now following Xyce's master branch. @tvrusso thanks for the very detailed explanations above!

pepijndevos commented 2 years ago

This is great news! Where does the source code for these images live? Would be great to add some more metadata to the Docker hub page so people know where it comes from. I assume it's this one? Took me a while to find. https://github.com/hdl/containers/blob/main/debian-bullseye/xyce.dockerfile

umarcor commented 2 years ago

The documentation is available at hdl.github.io/containers.

I recommend using gcr.io or ghcr.io, instead of hub.docker.com. Docker's hub does not support using / in image names, so not all of the images are/will be available there. Particularly, images for architectures other than amd64 are not available.

Overall, we use the registry API to push the images to the three registries, so we don't handle metadata related to the frontend/host. Nevertheless, I will add some URLs to https://hub.docker.com/u/hdlc.

umarcor commented 2 years ago

The descriptions of the images in hub.docker.com/u/hdlc now have refs to hdl.github.io/containers.

proppy commented 2 years ago

fyi, xyce is also available as a spack package.

proppy commented 2 years ago

fyi, xyce is also available as a spack package.

proppy commented 2 years ago

fyi, xyce is also available as a spack package.

google / skywater-pdk