ghdl / ghdl

VHDL 2008/93/87 simulator
GNU General Public License v2.0
2.4k stars 366 forks source link

Digital/Analog simulation #1052

Open umarcor opened 4 years ago

umarcor commented 4 years ago

A comment in Gitter (November 30, 2019 10:35 AM) lead to an interesting chat about Digital/Analog simulation with GHDL. There are two related but different use cases:

Ideally, it would be possible to combine both approaches, so that it is possible to describe a SoC with VHDL and VHDL-AMS and then simulate it with external analog peripherals.


Currently, GHDL supports two mechanisms to interface foreign languages:

However, for the first use case the difference is trivial, since analog modules are solved through callbacks (in C/C++/Python). It has already been considered or prototyped with multiple analog simulators:

nghdl uses VHPIDIRECT. Xyce might use either VHPIDIRECT or VPI; examples have been reported for the latter only, using cocotb. I am not aware of any implementation with Gnucap yet.

Either of these approaches allow to define black boxes in VHDL which are solved through callbacks. Hence, it is possible to describe SoCs with digital and analog pieces, but VHDL and netlist are required, instead of VHDL and VHDL-AMS. In other words, it is not a matter of hierarchy, but about the language(s).


Precisely, the initial focus of the discussion was on VHDL-AMS, which AFAIK has not been implemented yet. With recent advances in synthesis and integration with Yosys, I think that a similar approach might work with Xyce:

This approach is similar to using ADMS to convert Verilog-A to C (December 1, 2019 7:32 PM),

I think that the complexity lays in the conversion between GHDL AST for VHDL-AMS and the format/API that Xyce supports. As said, I don't think Xyce provides an API to build netlists, but this is probably because the netlist format itself can be considered an API:


Overall, it seems that it would be useful as a starting point to have a cannonical VHDL package where analog types are defined. A very naive approach would be:

type analog_t is record
  voltage: integer;
  current: integer;
  electrical_ref: integer;
end;

function digital(a: analog_t) return std_logic_vector;
function analog(d: std_logic_vector; current: integer; ref: integer) return analog_t;
entity analog_module is
  port (
    i: in std_logic_vector;
    o: out std_logic_vector
  );
end;

architecture dad of analog_module is

  alias dac is analog[std_logic_vector,integer,integer return analog_t];
  alias adc is digital[analog_t return std_logic_vector];

  signal a_in, a_out: analog_t;

begin

  a_in <= dac(i, 0, 0); 

  -- Model

  o <= adc(a_out);

end;

Such a solution would allow to implement multiple architectures:

It is up to the developers to use dac and/or adc, or to describe their own.

Note that analog_t above is based on ieee.electrical_systems.electrical: https://standards.ieee.org/content/dam/ieee-standards/standards/web/download/1076.1-2017_downloads.zip. An alternative is to add the zip to libraries.

Last, but not least, there is a full VHDL-AMS testsuite in the repo already (https://github.com/ghdl/ghdl/tree/master/testsuite/vests/vhdl-ams) but I believe that it is not being used. Some of the components/tests need the specific IEEE packages, but some other can already work (for example https://github.com/ghdl/ghdl/tree/master/testsuite/vests/vhdl-ams/ashenden/compliant/digital-modeling).


/cc @tmeissner @pepijndevos @cmarqu @peshola

umarcor commented 4 years ago

the SAND Report you reference above is a bit dated. It was updated for Xyce 6.11, in June 2019, as SAND Report SAND2019-6653 ("Application Note: Mixed Signal Simulation with Xyce 6.11 "). There were various bug fixes done for Xyce 6.11. In addition, some of the function signatures in the XyceClnterface class, and the corresponding Python methods, were modified to make them more compliant with ANSI-C.

@peshola, I think I get an idea of how XyceCInterface works, even with the outdated reports; hence, I guess I will be ok with the sources. Actually, I'd like to have a look at some working example. Just the netlist of any simple design being simulated in Xyce will work. I found section 'Simulation Examples with Xyce' in https://xyce.sandia.gov/downloads/_assets/documents/Users_Guide.pdf. Nonetheless, it'd be easier to clone a repo which is known to work. I tried https://gitlab.sandia.gov/amsmith/pycat_FSM (ref [5] in https://www.osti.gov/servlets/purl/1488489), but it seems not to be public.

If you're interested in the details then I can work out where to post that newer SAND Report so that it's publicly accessible. (The OSTI site is often several months behind.)

I think it would be useful to have the sources of those reports, which are so tightly related to Xyce, available either in https://github.com/Xyce/Xyce/tree/master/doc (say a new subdir Reports) or in some specific repo (say Xyce/Reports or any other location/domain). It would be easier for us users to find up to date docs, even if the official distribution channel is slightly slow. Anyway, I think that the main advantage would be that working examples can be hosted with the sources of the reports. On the one hand, it would allow users to simulate and see how it works, to better understand the report. On the other hand, since LaTeX seems to be used, it would be possible to import sources through listings. Last, if GitHub Actions (or any other CI service) is used, documents can be built automatically after each push and examples can be simulated to spot regression issues. Should there be no licensing issues and should you be interested on enabling a CI service, please let me know.

If you look at the the Xyce repos on Github, the source code for the XyceCInterface class is in Xyce/Xyce:utils/XyceCInterface@master . Those functions call methods in the Simulator class defined in Xyce/Xyce:src/CircuitPKG/N_CIR_Xyce.h@master .

It seems that the python source is not documented (https://github.com/Xyce/Xyce/blob/master/utils/XyceCInterface/xyce_interface.py) but XyceCInterface (which is equivalent) can be processed with doxygen. Is there any make target to do it?

The directory Xyce/Xyce:utils/XyceCInterface@master has some simple examples of invoking the interface. There are more examples in Xyce/Xyce_Regression:Netlists/MIXED_SIGNAL@master . Unfortunately, those VPI examples use Icarus rather than GHDL.

That's exactly what I was looking for! https://github.com/Xyce/Xyce/blob/master/utils/XyceCInterface/Python_examples/runCircuitWithDACs/runCircuitWithDACs.py seems to fit well with the naive prototype above. I guess that a simple integration with GHDL would consist on:

I foresee that such a solution can be much shorter and easier to understand than https://github.com/Xyce/Xyce/blob/master/utils/XyceCInterface/VPI_examples/runXyceWithDAC/runXyceWithDAC.c.

The next step would be to define the netlist as a VHDL string (or load it from a file from VHDL). In this context, the netlist would not be known before executing ghdl_main. Therefore, runXyceWithDAC_call would also need to be bind through VHPIDIRECT. This is similar to @bradleyharden's proposal in VUnit/vunit#603, but opposite. He is proposing to write C sources which are one-by-one equivalent to some VHDL packages. Here, I am proposing to write VHDL sources to bind the existing C API of Xyce. The advantage in this case is that bodies need not to be reimplemented. Hence, as commented in VUnit/vunit#603, this same approach might work with other simulators that support interfaces similar to VHPIDIRECT (say FLI).

Such an example can be built with a shell script or with makefiles. Nevertheless, VUnit can also be used (see https://github.com/VUnit/vunit/blob/master/examples/vhdl/external_buffer/run.py) as a reference. The relevant snippets are building main.c, and passing the object and grt.ver as cli args to GHDL (https://github.com/VUnit/vunit/blob/master/examples/vhdl/external_buffer/run.py#L69). Precisely, we should write a grt.ver file which includes all the functions from Xyce's API that are to be used from VHDL.

Let me know if you have specific questions. We can discuss them here, or on the Xyce Google Group.

With the approach above, there would still be a limitation: I don't think that running a separate isolated simulation for each analog_module is optimal at all. Is it possible to merge multiple otherwise independent netlist in a single simulation?

Regarding the VHDL-AMS to Xyce/SPICE netlist conversion, do you have any thoughts/feelings? I guess that the parsing part is solved by GHDL, but there might be some missing bits to create a complete netlist which is valid for simulation.

tgingold commented 4 years ago

I don't want to spend time of this now (I prefer to focus on synthesis). Just 2 points:

peshola commented 4 years ago

@umarcor You're correct. This repo (https://gitlab.sandia.gov/amsmith/pycat_FSM) is not publicly available. That's not likely to change in the near future.

This morning, I did make the latest version of the Mixed Signal App Note (as updated for Xyce 6.11) available at https://xyce.sandia.gov/downloads/_assets/documents/AppNote-MixedSignal_6.11.pdf. Our current policy is to only post the .pdf versions of the various Xyce App Notes. We don't push their LaTex sources to GitHub.

You might peruse Sections 5 ("Device Models for Mixed Signal Simulation") and 7 ("Conclusions and Future Work") of the updated App Note. They cover most of the known issues with the existing Xyce implementation.

Another note is that the Python interface to Xyce was intended as a stepping-stone to either a VHPI- or VPI-based interface to GHDL. So, this conversation is useful to the Xyce Team. With that said, improvements to the Xyce Mixed Signal Interface are not currently a high priority for us but they should see some development work in 2020 especially if there's active interest from the GHDL project.

umarcor commented 4 years ago

I don't want to spend time of this now (I prefer to focus on synthesis).

@tgingold, good.

there is a small support of AMS in ghdl, just use -fams

Is it enough to use https://standards.ieee.org/content/dam/ieee-standards/standards/web/download/1076.1-2017_downloads.zip or is it untested?

you cannot statically create a spice netlist from vhdl-ams, as the set of equation to solve is not fixed.

I do neither know the details of Verilog-A vs VHDL-AMS, nor do I hace enough knowledge of the underlying complexity. It seems that Xyce is quite tested with Verilog-A, so I'll just suggest interested users (@pepijndevos, that's you :wink:) to have a look at sections 'Reduce the optimization level of models derived from Verilog-A sources' and 'Turn off analytic sensitivity derivative calculation for Verilog-A-derived devices' from https://xyce.sandia.gov/documentation/BuildingGuide.html.


@peshola, I was just coming to report that I have successfully adapted the first two Python/VPI examples to GHDL using VHPIDIRECT and VUnit. I had a really hard time installing Trilinos and Xyce, until I found the proper configuration options. But I have built a docker image based on ghdl/vunit:llvm (Debian Buster) and everything seems to work as expected!

VUnit is not strictly necessary, but it makes it easier to build the executable combining GHDL and Xyce; and to handle parameters and assertions in the VHDL testbench.

This is the most basic example:

library vunit_lib;
context vunit_lib.vunit_context;

entity tb_xyce is
  generic (
    runner_cfg : string;
    tb_path    : string
  );
end entity;

architecture tb of tb_xyce is

  function xyce(cir: string) return integer is
  begin
    assert false report "VHPIDIRECT xyce" severity failure;
  end;
    attribute foreign of xyce : function is "VHPIDIRECT xyce";

begin

  main: process
    constant cir: string := tb_path & "../circuit.cir";
  begin
    test_runner_setup(runner, runner_cfg);
      check_equal(
        xyce(cir),
        1,
        "return code of 'xyce_init'"
      );
    test_runner_cleanup(runner);
    wait;
  end process;

end architecture;
#include <stdlib.h>
#include <stdint.h>
#include <stdio.h>
#include <assert.h>

#include <N_CIR_XyceCInterface.h>

uint32_t xyce(char** cir) {
    printf("VHPIDIRECT: entering xyce %s\n", *cir);

    void** p = (void **) malloc( sizeof(void* [1]) );

    char *argList[] = {
        (char*)("Xyce"),
        (char*)(*cir),
    };

    int status;

    xyce_open(p);

    status = xyce_initialize(p, sizeof(argList)/sizeof(argList[0]), argList);
    assert( status == 1);
    assert( p != NULL );

    status = xyce_runSimulation(p);
    assert( status == 1);

    xyce_close(p);
    free(p);

    printf("VHPIDIRECT: exiting xyce\n");
    return status;
}
from vunit import VUnit
from os import popen
from os.path import join, dirname

vu = VUnit.from_argv(vhdl_standard="2008")

src_path = join(dirname(__file__), "src", "runACircuit")

c_obj = join(src_path, "main.o")

print(popen(" ".join([
    "gcc",
    "-fPIC",
    "-I" + src_path,
    "-c", join(src_path, "main.c"),
    "-o", c_obj,
])).read())

vu.add_library("lib").add_source_files(join(src_path, "*.vhd"))
vu.set_sim_option("ghdl.elab_flags", ["-Wl," + c_obj, "-Wl,-lxycecinterface"])

vu.main()

This morning, I did make the latest version of the Mixed Signal App Note (as updated for Xyce 6.11) available at xyce.sandia.gov/downloads/_assets/documents/AppNote-MixedSignal_6.11.pdf.

Thanks! Figures 7-2 and 7-3 look very interesting. And there is a lot of new details about ADCs/DACs (the complexity of the third example, and a requirement for a any practical design).

Another note is that the Python interface to Xyce was intended as a stepping-stone to either a VHPI- or VPI-based interface to GHDL. So, this conversation is useful to the Xyce Team.

Note that I did not use the Python interface at all. As you see in the code blocks above, I'm writing a simplified API in C, and binding it to VHDL. This is the API that I am using for the third example (work in progress), as seen in VHDL:

  function xyce_init(
    id      : string;
    circuit : string
  ) return integer;
    attribute foreign of xyce_init : function is "VHPIDIRECT xhdl_init";

  function xyce_run(
    id      : string;
    t       : real;
    arrTime : real_vector(0 to 8);
    arrVolt : real_vector(0 to 8);
  ) return integer;
    attribute foreign of xyce_run : function is "VHPIDIRECT xhdl_run";

  function xyce_close(
    id : string
  ) return integer;
    attribute foreign of xyce_close : function is "VHPIDIRECT xhdl_close";

So, the flow of the co-simulation is managed in the VHDL testbench and/or sources. I implemented a storage, to hold pointers, ids and ADC/DAC names. When xyce_init is executed from VHDL, an item is created and an instance of the simulator is started. Then, VHDL calls xyce_run (which is xyce_updateTimeVoltagePairs + xyce_simulateUntil in a single function call) in a for loop (ADCs/DACs are expected to be synchronous). Last, executing xyce_close VHDL terminates the instance and removes the item from the storage.

This approach should allow to instantiate multiple analog modules, each defined with the same or a different netlist file. However, as commented above, I fear that starting an instance of Xyce for each netlist can be suboptimal.

Note that the storage logic is loosely based on @bradleyharden's work in VUnit/vunit#603, which is a port to C of some of VUnit's internal VHDL types. Hence, I might reuse his implementation, once it is merged in VUnit.

With that said, improvements to the Xyce Mixed Signal Interface are not currently a high priority for us but they should see some development work in 2020 especially if there's active interest from the GHDL project.

Some ideas so far:

A. Is it possible to provide an alternative to xyce_initialize that accepts the netlist as a pointer to string/stream/array_of_bytes? Shall I use stdin? The target is to allow writing netlists as constant strings in the VHDL sources.

B. I think it would be desirable to have some official VHPIDIRECT API for GHDL-Xyce (which is composed of a VHDL package and a C header file). With official, I mean some reviews/hints from developers of both tools. IMHO, the storage, which allows to identify instances with strings rather than (probably difficult to bind) pointers, should be part of this API. Currently, I'm using these types:

struct xhdl_t {
 char     *  id;
 void     ** ptr;
 double   *  time;
 xhdl_c_t *  ADC;
 xhdl_c_t *  DAC;
};

struct xhdl_c_t {
  char ** names;
  uint *  num;
};

Should you know of equivalent types in Xyces codebase, please, let me know. It'd would be much easier to maintain.

Moreover, it should be possible to avoid writing an intermediate API. I.e., the 16 functions in XyceCInterface might be mapped one-by-one to VHDL. However, GHDL has some issues with unconstrained arrays, and most of the arguments in XyceCInterface are unconstrained.

I don't know if these sources should fit in this repo, in Xyce's codebase or as a separate repo. On the one hand, I believe that the issue in Xyce might be the license. On the other hand, I think that ghdl does not include any sources to interact with specific tools.

C. Binaries generated with ghdl -e cannot (should not) be dynamically loaded from e.g. Python (see details in https://github.com/VUnit/vunit/issues/603#issuecomment-560391588). That's why I didn't use the Python interface. Furthermore, it would have been unnecessarily complex:

VHDL call -> VHPIDIRECT -> C -> ctypes -> Python -> Xyce.so

instead of

VHDL call -> VHPIDIRECT -> C -> Xyce.so

However, I see that it can be useful for some users to hide C/C++ almost completely. Fortunately, binaries combining GHDL and Xyce can be built with GCC, instead of using ghdl -e; and it should work on any platform (not only GNU/Linux). Therefore, I believe it is possible to use ctypes to forward C callbacks from the VHPIDIRECT C API to Python functions. This would allow to use VHDL, Xyce and Python only.

D. I believe that using VPI is a very different path. As commented in https://github.com/VUnit/vunit/issues/603#issuecomment-560391588, VPI support in GHDL is much better than VHPIDIRECT. cocotb is a known project that allows co-simulation with GHDL and VPI, and you have already prototyped it. Should you want to support not only Digital/Analog co-simulation but also low-level tinkering with any signal/clock in the VHDL design, I think that VPI and cocotb are a better fit.

Nonetheless, VPI might be overkill for the three existing examples, as it provides many built-in features that are not used at all. Conversely, VHPIDIRECT, does not provide built-in features to inspect the design, but it provides direct matches between VHDL types and functions, which is all we need, since we want to interact with an existing C lib (Xyce).

Summarizing cocotb/VPI are a better fit for users that want to write most of the tests in Python; while VUnit/VHPIDIRECT are meant for HDL developers that want to write the tests in VHDL.

Fatsie commented 4 years ago

Good to have this discussion, but I would also like to know the use case for Mixed signal simulation people have in mind. Depending on the use case the way you want to do the simulation will be different.

umarcor commented 4 years ago

Good to have this discussion, but I would also like to know the use case for Mixed signal simulation people have in mind. Depending on the use case the way you want to do the simulation will be different.

From this point of view, the example above (based on Python_examples/runACircuit or VPI_examples/runXyce), is almost useless. It is to test that we can build and bind both tools through VHPIDIRECT. As said, I think that ADCs/DACs are required for any practical use case.

Moreover, Xyce provides multiple analysis algorithms and allows to define models in C/C++ (that's how ADCs/DACs are implemented). Hence, I believe that almost any use case can be supported. Yet, I think that having so many options can be overwhelming. I feel that it is much easier for someone with Analog/Xyce background to be productive than for Digital/VHDL designers.

Do you have any specific use case or simulation type in mind?

peshola commented 4 years ago

@umarcor I need to try to duplicate some of your work. What versions of ghdl and vtest are you using?

Your thought (to paraphrase) of "not using the Python interface at all; instead writing a simplified API in C, and binding it to VHDL" as discussed in Comment C above seems like a promising approach. The Python interface does add complexity and performance overhead.

Also, for Comment B above, having some sort of well-tested API for GHDL-to-Xyce seems like a good idea. Once we figure out what that API is then we can decide on what parts of it should live in a ghdl repo vs. a Xyce repo. One complicating factor may be that Xyce is open-source, but we can't easily accept source contributions from outside of Sandia. So, that may argue for putting most of the interface into a gdhl repo.

We're also open to modifying some of the existing Xyce APIs. We just need to define what those changes are. For example, we have had other requests for "accepting the netlist as a pointer to string/stream/array_of_bytes".

peshola commented 4 years ago

@fatsie One use case of interest is single-event upsets. An overview of that use case is given by https://www.osti.gov/biblio/1488489 It may be one of the simplest possible use cases, as posed in that paper.

umarcor commented 4 years ago

@umarcor I need to try to duplicate some of your work. What versions of ghdl and vtest are you using?

@peshola, I'm working on subdir examples/vhdl/xyce from branch xyce-dev of dbhi/vunit (a fork of VUnit/vunit): https://github.com/dbhi/vunit/tree/xyce-dev/examples/vhdl/xyce However, bear in mind that I am force-pushing frequently. I will keep it untouched for a couple of days, so that you can get a copy. In the current status, runACircuit and runBCircuit should work. The former is basically the example above, and the latter includes the storage mechanism to separate xyce_init and xyce_run. runWithDACs is a mess.

The environment I am using is this Dockerfile: https://github.com/dbhi/vunit/blob/xyce-dev/examples/vhdl/xyce/Dockerfile So, it is Debian Buster with almost the latest GHDL from master (precisely 0.37-dev (v0.36-1338-g0b847007)), Python 3.7.3, latest release of VUnit, Trilinos 12-12-1 and Xyce 6.12. I built it with:

DOCKER_BUILDKIT=1 docker build -t ghdl/xyce - < Dockerfile

I then start a container from examples/vhdl/xyce. Note that VUnit is already installed in the container, so you don't need all the repo.

docker run --rm -itv $(pwd):/src -w /src ghdl/xyce bash
root@91f3210f228f:/src# ls
Dockerfile  run.py  src  vunit_out
root@91f3210f228f:/src# python3 run.py -v
...
MINV1 = -9.999481e-01 at time = 7.516221e-01
Measure Start Time= 0.000000e+00        Measure End Time= 1.000000e+00

***** Total Simulation Solvers Run Time: 0.052141 seconds
***** Total Elapsed Run Time:            0.057384 seconds
*****
***** End of Xyce(TM) Simulation
*****

Timing summary of 1 processor
                 Stats                   Count       CPU Time              Wall Time
---------------------------------------- ----- --------------------- ---------------------
Xyce                                         1        0.010 (100.0%)        0.059 (100.0%)
  Analysis                                   1        0.010 (100.0%)        0.044 (75.11%)
    Transient                                1        0.010 (100.0%)        0.044 (74.96%)
      Nonlinear Solve                      206        0.010 (100.0%)        0.013 (21.95%)
        Residual                           412        0.010 (100.0%)        0.004 ( 7.03%)
        Jacobian                           206        0.000 ( 0.00%)        0.002 ( 3.10%)
        Linear Solve                       206        0.000 ( 0.00%)        0.003 ( 5.56%)
      Successful DCOP Steps                  1        0.000 ( 0.00%)        0.003 ( 4.64%)
      Successful Step                      204        0.000 ( 0.00%)        0.006 (10.17%)
      Failed Steps                           1        0.000 ( 0.00%)        0.000 (<0.01%)
  Netlist Import                             1        0.000 ( 0.00%)        0.003 ( 5.14%)
    Parse Context                            1        0.000 ( 0.00%)        0.002 ( 3.48%)
    Distribute Devices                       1        0.000 ( 0.00%)        0.000 ( 0.31%)
    Verify Devices                           1        0.000 ( 0.00%)        0.000 (<0.01%)
    Instantiate                              1        0.000 ( 0.00%)        0.000 ( 0.18%)
  Late Initialization                        1        0.000 ( 0.00%)        0.001 ( 1.59%)
    Global Indices                           1        0.000 ( 0.00%)        0.000 ( 0.06%)
  Setup Matrix Structure                     1        0.000 ( 0.00%)        0.000 ( 0.54%)

xyce_close after delete xycePtr
VHPIDIRECT: exiting xyce_run my_circuit
simulation stopped @0ms with status 0
pass (P=2 S=0 F=0 T=2) runBCircuit.tb_xyce.test (0.7 seconds)

==== Summary ====================================
pass runACircuit.tb_xyce.all  (0.7 seconds)
pass runBCircuit.tb_xyce.test (0.7 seconds)
=================================================
pass 2 of 2
=================================================
Total time was 1.3 seconds
Elapsed time was 1.3 seconds
=================================================
All passed!
root@91f3210f228f:/src# exit

Your thought (to paraphrase) of "not using the Python interface at all; instead writing a simplified API in C, and binding it to VHDL" as discussed in Comment C above seems like a promising approach. The Python interface does add complexity and performance overhead.

Also, for Comment B above, having some sort of well-tested API for GHDL-to-Xyce seems like a good idea. Once we figure out what that API is then we can decide on what parts of it should live in a ghdl repo vs. a Xyce repo. One complicating factor may be that Xyce is open-source, but we can't easily accept source contributions from outside of Sandia. So, that may argue for putting most of the interface into a gdhl repo.

Although not fully functional, this is my prototype of that possible simplified API:

The missing pieces are:

I'm just too sleepy to finish it now. It's already been a long sprint to reach here...

We're also open to modifying some of the existing Xyce APIs. We just need to define what those changes are.

Honestly, I'm learning by doing. I've been working with GHDL, VHPIDIRECT and VUnit lately. When I saw your papers a couple of days ago I felt that it might work and I had to try it. I acknowledge that you did a really interesting work already, and despite the installation being painful, the documentation is sparse but very detailed. This is to say that I don't feel entitled to suggest meaningful API changes. I'd rather be warned when you feel that I am simplifying too much.

For example, I merged xyce_simulateUntil and xyce_updateTimeVoltagePairs because in the examples both functions are always used together. However, there might be other use cases which I am not aware about that lead you to have them separated in XyceCInterface.

For example, we have had other requests for "accepting the netlist as a pointer to string/stream/array_of_bytes".

I saw that you are not using GitHub issues, but prefer Google Groups. Although I understand that Groups might be better for general support, Issues (or any other public bug tracker) might be useful to leverage/prioritize technical enhancements. Regarding PRs, note that you can disable them if you have licensing issues. I assume that you do have same internal tracker, but a public service with bug templates and integrated CI can really help get good bug reports and MWEs.

Fatsie commented 4 years ago

Do you have any specific use case or simulation type in mind?

In my previous job I have had experience with mixed signal simulations but I have the feeling that the discussion here is quite theoretical without a practical application in mind. If people have a good use case in mind I could provide some guidance based on experience. For example, if you want to simulate a design with ADC and DAC this is typically done by having Verilog-A models of these blocks. But that thus depends on having the Verilog-A model of the blocks you want to simulate first, which is an art itself. In such situation it is best to look at which models are available and then see how to get these in open source tools.

One use case of interest is single-event upsets.

My previous job was actually in developing radiation hardened libraries and SRAMs and giving support for the analog designers to get the digital files like .lib etc for their IP blocks like ADC/DAC/PLL etc. We did not do mixed signal simulations. Typically the radiation hardness of the digital cells is assessed by pure analog simulations. Also the analog blocks like DAC/ADC are hardened by pure analog simulations. On higher level fault injection is just done in the digital level based on statistics, I am not sure if doing that in mixed signal is efficient as you typically need to do a lot of such simulations because you need derive statistics on the single event upsets with both timing and place of the strike being varied.

cmarqu commented 4 years ago

@Fatsie Unless I'm mistaken, the mentions of ADC and DAC above are referring to what would be called E2L (electrical-to-logic) and L2E (logic-to-electrical) connectmodules/interface elements in commercial simulators.

umarcor commented 4 years ago

In my previous job I have had experience with mixed signal simulations but I have the feeling that the discussion here is quite theoretical without a practical application in mind.

Yes, that is the case. The conversation in Gitter started about VHDL-AMS and whether it would be possible to extend GHDL to support it. Several projects were mentioned and Xyce seems to be the best suited to be plugged into GHDL. At least, one of the authors provided enough info for it to feel easy to implement. One we have a stable interface between GHDL and Xyce is when the fun can begin.

If people have a good use case in mind I could provide some guidance based on experience.

Should you feel that practical and relevant knowledge is being overlooked, please do expand the discussion. If it gets messy, we can always split to some other issue (maybe in Xyce?).

For example, if you want to simulate a design with ADC and DAC this is typically done by having Verilog-A models of these blocks. But that thus depends on having the Verilog-A model of the blocks you want to simulate first, which is an art itself. In such situation it is best to look at which models are available and then see how to get these in open source tools.

I believe that this is already solved in Xyce, since many of the internal models are written in Verilog-A. Precisely, I was experiencing 150min of build time, and I believe it was partially because of how long it took to analyze and optimize the Verilog-A models: https://xyce.sandia.gov/documentation/BuildingGuide.html

Moreover, in the article that Peter uploaded this morning (https://xyce.sandia.gov/downloads/_assets/documents/AppNote-MixedSignal_6.11.pdf), section 5 'Device models for mixed-signal simulation', the details of ADCs/DACs are explained. Figure 5-1 contains the equations for YADC.

From a theoretical perspective, we are providing example sources and build scripts for users that have access to good quality Verilog-A models of the blocks to simulate then along with VHDL 2008 digital designs, using open source tools only. VHDL 2008 provides interesting features to generate complex test sequences composed of multiple concurrent processes. I believe that this can be an interesting feature, even though not mainstread.

I am personally very interested on didactic purposes. EE students that are using simulink and System Generator to simulate VHDL controllers for electromechanical systems can find GHDL + Xyce really appealing.

umarcor commented 4 years ago

@Fatsie Unless I'm mistaken, the mentions of ADC and DAC above are referring to what would be called E2L (electrical-to-logic) and L2E (logic-to-electrical) connectmodules/interface elements in commercial simulators.

According to the descriptions, that's correct. I believe that those were written to implement the XyceCInterface only, not to be accurate. However, should VHDL-AMS be supported in the future, ADCs/DACs might need to be renamed to C2HDL and HDL2C espectively, and there would probably be ADCs/DACs described in VHDL to interface VHDL with VHDL-AMS. Even though Verilog-A is used, I believe that it is as C/C++ models (through verilator?), so there is no HDL suport built-in. Note that support for VHDL-AMS in GHDL is not even in the roadmap, as Tristan commented above.

peshola commented 4 years ago

@umarcor Xyce is not truly an open-source project. It's primary purpose is to support our various Government customers. Those funding sources drive how we're allow to interact with the general public. We currently have permission to push snapshots of Xyce's master branch, as well as tagged releases, to github on a regular basis under GPLv3. However, we're strongly encouraged to consolidate discussions about Xyce with non-Sandia users (bug reports, usage questions, build questions, feature requests) on our Google Group at https://groups.google.com/forum/#!forum/xyce-users. Some of those Google Group discussions then get summarized into our internal issue tracking systems, where the gist of those external requests can be merged with the needs of our internal customers.

We use this approach instead of using git issues at https://github.com/Xyce/Xyce. I agree that this is NOT the optimal way to engage with truly open source projects, but it's what we have.

peshola commented 4 years ago

@fatsie @umarcor Yes, the YADC and YDAC device models in Xyce, discussed in Section 5 of https://xyce.sandia.gov/downloads/_assets/documents/AppNote-MixedSignal_6.11.pdf, are not realistic device models. They gloss over important issues like rise/fall times and drive/sink currents. They were adequate for various internal research customers, but those models do need improvement. Suggestions about that are welcome. We have had requests, from internal customers, to support IBIS device models. Would that approach be relevant to GHDL?

tgingold commented 4 years ago

I think there are 3 different topics that are being discussed:

I have never used xyce/trilinos, so I may be completely wrong.

umarcor commented 4 years ago
  • simulating vhdl and xyce models. The application note is about that. If this was done using VPI, it could also be done with GHDL, but it could be made easier for sure.

What do you think about a package defining VHPIDIRECT functions/procedures that bind to https://github.com/Xyce/Xyce/blob/master/utils/XyceCInterface/N_CIR_XyceCInterface.h directly? You discourage the usage of fat pointers. Is it because they are prone to change because of GHDL internal structure or is it because of portability issues? We can always use an intermediate C file, but I wonder what is your view.

  • Generating xyce models from VHDL-AMS ones, like ADMS. Not sure how useful it is.

It would allow to use VHDL-AMS models in Xyce, just as Verilog-A is supported now. I think that this feature is interesting itself. Moreover, the IBIS models that @peshola mentioned seem to support subcircuits described in Verilog-A or VHDL-AMS. Hence, without VHDL parsing and elaboration some models might not be supported. It is true real IBIS models including VHDL-AMS might not exists.

Another option is to generate SPICE models, which are supported by Xyce. However, it might be limited: https://dokumen.tips/documents/the-implementation-of-a-vhdl-ams-to-spice-converter.html

  • Supporting VHDL-AMS. I am not sure how xyce would be used, as it may not provide a low level interface. I think solvers like trilinos would be at the right level.

Do you mean to build the required solvers into GHDL to support mixed-signal simulation as a built-in feature?

tgingold commented 4 years ago

What do you think about a package defining VHPIDIRECT functions/procedures that bind to https://github.com/Xyce/Xyce/blob/master/utils/XyceCInterface/N_CIR_XyceCInterface.h directly?

Would that be that interesting ? Maybe we should simply provide an equivalent API for GHDL.

You discourage the usage of fat pointers. Is it because they are prone to change because of GHDL internal structure or is it because of portability issues? We can always use an intermediate C file, but I wonder what is your view.

They aren't very intuitive. I think it would be nicer to have a C API for them.

• Generating xyce models from VHDL-AMS ones, like ADMS. Not sure how useful it is. It would allow to use VHDL-AMS models in Xyce, just as Verilog-A is supported now. I think that this feature is interesting itself. Moreover, the IBIS models that @peshola mentioned seem to support subcircuits described in Verilog-A or VHDL-AMS. Hence, without VHDL parsing and elaboration some models might not be supported. It is true real IBIS models including VHDL-AMS might not exists.

Another option is to generate SPICE models, which are supported by Xyce. However, it might be limited: https://dokumen.tips/documents/the-implementation-of-a-vhdl-ams-to-spice-converter.html

That's my point: there is no VHDL-A, only VHDL-AMS.

• Supporting VHDL-AMS. I am not sure how xyce would be used, as it may not provide a low level interface. I think solvers like trilinos would be at the right level. Do you mean to build the required solvers into GHDL to support mixed-signal simulation as a built-in feature?

Yes, that's the idea.

umarcor commented 4 years ago

What do you think about a package defining VHPIDIRECT functions/procedures that bind to Xyce/Xyce:utils/XyceCInterface/N_CIR_XyceCInterface.h@master directly?

Would that be that interesting ? Maybe we should simply provide an equivalent API for GHDL.

Sure! It would allow to use Xyce directly from VHDL, without any C coding at all. Providing an equivalent API for GHDL would require users to write a C body to be the glue between the API of GHDL and the API of Xyce. Note that:

They aren't very intuitive. I think it would be nicer to have a C API for them.

The point is that in this case I don't want/need a C header that allows me to easily create pointers of a specific type to access e.g. GHDL's representation of an unconstrained string. That would be very useful for any of the other cases commented above.

In this case, I need the opposite. We have:

int xyce_getTimeVoltagePairsADC(
  void** ptr,
  int * numADCnames,
  char ** ADCnames,
  int * numPoints,
  double ** timeArray,
  double ** voltageArray
);

And I would like to write a VHDL equivalent:

  type acc_void is access ???;
  type acc_int is access integer;
  type acc_int is access string;
  type real_vector is array (natural range <>) of real;
  type acc_real_vector is access real_vector;

  impure function xyce_getTimeVoltagePairsADC (
    ptr          : acc_void;
    numADCnames  : acc_int;
    ADCnames     : acc_string;
    numPoints    : acc_int;
    timeArray    : acc_real_vector;
    voltageArray : acc_real_vector
  ) return integer;

  attribute foreign of xyce_getTimeVoltagePairsADC : procedure is "VHPIDIRECT xyce_getTimeVoltagePairsADC";

Is it even possible?

Do you mean to build the required solvers into GHDL to support mixed-signal simulation as a built-in feature?

Yes, that's the idea.

That'd be awesome! But it must be a hard task...

I suppose that Xyce has some internal APIs on top of Trilinos that might be a better fit to "solve analog circuits". Certainly, it seems to support many different analysis types. However, I don't know if all of them are useful in the context of VHDL-AMS simulation.

@peshola commented that "Mixed Signal Interface are not currently a high priority for us but they should see some development work in 2020 especially if there's active interest from the GHDL project". Maybe an area of collaboration in the future can be for Xyce to expose an API which GHDL can use as an analog solver. Assuming that most of the parsing and elaboration in GHDL is already usable for VHDL-AMS, the complexity would lie on the synchronization between both domains.

tgingold commented 4 years ago

I fear that the Xyce API is not very VHPIDIRECT friendly. You will need to create an intermediate layer.

umarcor commented 4 years ago

Thanks for confirming. Then, I will complete the intermediate layer. In this context, having a minimal API in C that allows to cleanly access fat pointers created by GHDL would be usable and very useful. However, I feel that I already distracted you enough. I will use constrained arrays, I'll check how Bradley reverse engineered some fat pointers, and we can revisit possible enhancements to GHDL in some weeks/months, after v0.37 is released.

maehne commented 4 years ago

I am delighted to see so much interest for doing mixed-signal simulation with GHDL. I would love to see VHDL-AMS fully supported in a future version of GHDL. I have been working with VHDL-AMS intensively over many years (especially between 2003 and 2011) to model various mixed-signal and multiphysical systems, e.g., a MEMS yaw rate sensor including its driving/readout electronic and an FSK transmitter. This work gave me the occasion to test/evaluate various commercial mixed-signal simulators supporting VHDL-AMS. Those, which supported the more advanced features of VHDL-AMS (e.g., breaking on events to reinitialise the operating point and maybe even changing the set of equations; different tolerance groups for testing for convergence of continuous-time quantities belonging to different physical domains; switched circuit behaviour) rather poorly usually followed the ad-hoc approach outlined in this issue, i.e., implementing some co-simulation interface to an existing in-house SPICE simulator. Each of the VHDL-AMS simulators that I tested over the years (about 5 different simulators from different vendors) had some shortcomings and required usually some tweaking to more advanced VHDL-AMS models to make them work properly with that simulator. This hampered considerably model exchange between users of different simulators.

The strength of VHDL-AMS is that it is a modelling language capable of describing mixed-signal and multiphysical systems not only on the circuit (SPICE) level, but especially on a more abstract behavioural level using sets of differential algebraic equations, which can be even switched at well defined moments, if necessary, when discrete events get triggered either from the digital or analog side ('above). Also the operating points (i.e., continuous-time state) of the simulation can be reinitialised at these moments to take into account discontinuities, improve convergence and precision of the mixed-signal simulation. This is, e.g., demonstrated by the classic bouncing ball model.

Several people in this discussion mentioned that they don't have experience with VHDL-AMS and Verilog-AMS. For them, I can suggest to read the following publications:

The first one is a very compact introduction to the main features of VHDL-AMS. The second one is the most comprehensive book on VHDL-AMS. The third one shows how mixed-signal HDLs can be used to model and simulate RF systems.

Regarding the implementation of a mixed-signal simulator on top of discrete-event simulation kernel, you may check out the following publications by Thomas Uhle and Karsten Einwich:

The AMS extensions for non-linear generalised networks described in these papers for SystemC (a C++-based modeling framework with a discrete-event simulation kernel), implemented the same synchronisation semantics between the discrete-event and continuous-time domains as VHDL-AMS. These very well defined synchronisation semantics are one of the reasons why VHDL-AMS is such a good modeling language for mixed-signal and multiphysical systems. Care should be therefore taken to properly implement them into GHDL. I feel that the proposed ad-hoc approach for integration with Xyce won't be able to enable these close synchronisation semantics. By the way, the solution of the sets of DAEs was implemented using the SUNDIALS IDA solver.

Note, that these remarks are not intended to contradict the usefulness to simulate some SPICE levels together with some top-level VHDL/VHDL-AMS model. Indeed, it can be very useful to be able to map a VHDL-AMS entity/component onto a SPICE-level netlist and co-simulate the whole system. It allows verifying more precisely the circuit implementation by providing more realistic stimuli to the SPICE circuit and verifying its behaviour inside feedback loops. It can also considerably speed up simulation compared to a full SPICE-level simulation.

tgingold commented 4 years ago

Thank you for all the infos. I will have a deeper look at the references.

Do you know if frequency domain is used in vhdl-ams ? That looks like a very particular use of the ams model.

Fatsie commented 4 years ago

I have seen frequency domain being used inside a model of a VCO inside a PLL. This was Verilog-AMS but I would expect VHDL-AMS modeling to use same principle.

tgingold commented 4 years ago

@umarcor: you can compile with gcc backend and debugging to see in gdb the prototype of the functions.

peshola commented 4 years ago

Thanks everyone for the comments and references. I need to read through several of them (especially the VHDL-AMS ones ) before I can comment on which approaches might be easier (or harder) to interface to Xyce. That might not be until January.

I will note that the existing Python ctypes-based interface to Xyce will likely persist, and see some development work early next year. That interface is useful to several of our internal research customers as a quick and easy way to couple Xyce to Python-based prototypes. It was also an easy way to couple CocoTB with GHDL and Xyce, again for mostly research-oriented projects.

umarcor commented 4 years ago

An update since https://github.com/ghdl/ghdl/issues/1052#issuecomment-561859370:

For the most impatient, this is how the VHDL testbench version of runXyceWithDAC.c looks like: https://github.com/umarcor/cosim/blob/master/examples/xyce/src/runWithDACs/tb_xyce.vhd

To do:


@peshola, in *WithDAC* examples from XyceCInterface, setting the timeArray and VoltageArray of DACnames[0] is hardcoded:

In order to support other use cases with more than one DAC, I am considering to enhance the current implementation of the xyce bridge to pass a bidimensional array:

  type arr_t is array (natural range <>, 0 to 1) of real;

or

  type arr_rec_t is record
    time    : real;
    voltage : real;
  end record arr_rec_t;

  type arr_t is array (natural range <>) of arr_rec_t;

and

  type arg_t is array (natural range <>) of arr_t;

  -- example: arg_t(0 to numDACnames-1);

However, I wonder whether some constraint exists regarding the usage of different timeArray vectors for each DAC:

umarcor commented 4 years ago

In practice, the length of both arrays should always match. Is this correct?

https://xyce.sandia.gov/downloads/_assets/documents/AppNote-MixedSignal_6.11.pdf 2.1.9. xyce_updateTimeVoltagePairs

The “error condition” of timeArrayandvoltageArray being of unequal lengths is checked when this method is invoked via the Python ctypes-based interface. It is not checked when xyce_updateTimeVoltagePairs is invoked directly. For direct invocation, it is the responsbility of the calling function to verify that the parameters numPoints, timeArray and voltageArray have consistent values and lengths.

peshola commented 4 years ago

@umarcor Yes, those arrays should have the same length. In general, the App Note attempts to point out where the error handling could still use improvement. Section 3.3 ("Known Limitations and Bugs") of the App Note points out some other issues with the getTimeVoltagePairsADC() and getTimeStatePairsADC() methods.

I have not tried using a different time and voltage array with each DAC. That should work though. If it doesn't that let me know, and I can dig into the underlying Xyce source code to figure out why.

You commented that "the examples in XyceCInterface [and the App note] are artificial or synthetic because there is no synchronization requirements between the loop that handles Xyce in VHDL and other concurrent processes/modules". That statement is correct. Those examples are the equivalent of "Hello World" programs that are intended to help someone get started with the Xyce APIs. They seem to accomplished that goal, for this ongoing discussion at least.

umarcor commented 4 years ago

@umarcor Yes, those arrays should have the same length.

@peshola, thanks. Then, I will use the following types:

type arr_t is array (natural range <>, 0 to 1) of real;
type arg_t is array (natural range <>) of arr_t;

VHDL, the language itself, will ensure that each pair of time and voltage vectors has the same length. At the same time, it will be possible to use a single argument to pass one or multiple pairs of vectors.

Note that it is currently not possible to constrain one of the indexes only. Hence, helper functions (new_arr_t(n: natural) and new_arr_t(t,v: real_vector)) will be provided.

In general, the App Note attempts to point out where the error handling could still use improvement.

I saw later that checks are mostly implemented in the Python interface, but not when using XyceCInterface directly. I think this is actually convenient for our use case, since VHDL can handle it for us.

Section 3.3 ("Known Limitations and Bugs") of the App Note points out some other issues with the getTimeVoltagePairsADC() and getTimeStatePairsADC() methods.

I think that none of those issues applies, since I am not using the Python interface. If I understand it correctly, the limitation of 1000 does not exist when using XyceCInterface, does it?

I have not tried using a different time and voltage array with each DAC. That should work though. If it doesn't that let me know, and I can dig into the underlying Xyce source code to figure out why.

I will try after applying the modifications above and I will let you know.

Those examples are the equivalent of "Hello World" programs that are intended to help someone get started with the Xyce APIs. They seem to accomplished that goal, for this ongoing discussion at least.

I understand it and I think it is a good approach. I did not mean to critisise it, but to ask for help of users that might have some real world examples. IMHO, Xyce's set of examples would be more attractive if synchronization issues were addressed. In order to set timing constraints, a good understanding of whichever design is required. I'm afraid I cannot contribute such an example because I don't know any mixed design well enough to correctly decide the synchronization. All I can think about is adding an RC filter to a PWM output. I will first try to reproduce the listings in section 7.1, about coordinated timestepping issues.

peshola commented 4 years ago

@umarcor "If I understand it correctly, the limitation of 1000 does not exist when using XyceCInterface, does it?". The XyceCInterface itself does not constrain the size of the arrays; but the calling program may because of how that program handles array allocation. The value of 1000 for the Python interface was arbitrarily chosen, and is hard-coded in the xyce_interface.py file. That could likely be improved on.

tgingold commented 4 years ago

It is now possible to analyze an AMS-VHDL file. Almost all the features of AMS-VHDL are handled. It is not well tested and some particular features may not be handled.

Again, it's just analysis and not simulation.

In the future, I plan to plug a DAE solver to be able to simulate AMS-VHDL designs. No performance goals, but just to have a better understanding on the analog requirements.

umarcor commented 4 years ago

@peshola, in the context of [LCS-202x] VHDL DPI/FFI based on GHDL’s implementation of VHPIDIRECT (PDF), I updated the variants of examples using VHPIDIRECT instead of VPI:

For instance, this is the runWithDACs example using an unconstrained multidimensional (2D) array in VHDL: https://github.com/umarcor/ghdl-cosim/blob/develop/vhpidirect/vffi_user/xyce/runWithDACs/tb_xyce_2D.vhd

library vunit_lib;
context vunit_lib.vunit_context;

use work.xyce_pkg.xyce_t;
use work.xyce_pkg.arr2D_t;

entity tb_xyce_eg_with_dacs_2D is
  generic (
    runner_cfg : string;
    tb_path    : string;
    circuit    : string := "../circuit_DACs.cir"
  );
end entity;

architecture tb of tb_xyce_eg_with_dacs_2D is begin

  main: process
    constant steps: natural := 10;
    constant total_sim_time : real := 20.0e-4;

    variable arr2D : arr2D_t(0 to 1, 0 to 8) := (
      (0.0, 0.1e-4, 0.2e-4, 0.4e-4, 0.5e-4, 0.7e-4, 0.8e-4, 1.0e-4, 1.1e-4),
      (0.0, 0.0, 3.0, 3.0, 0.0, 0.0, 3.0, 3.0, 0.0)
    );

    variable xyce : xyce_t;
    variable arrT : real_vector(arr2D'range(2));
    variable response : real;
  begin
    test_runner_setup(runner, runner_cfg);
    xyce.init("run a circuit with dacs (2D)", tb_path & circuit);

    for i in arrT'range loop
      arrT(i) := arr2D(0, i);
    end loop;

    for i in 0 to steps-1 loop
      xyce.run(
        0.0 + real(i+1) * 0.1 * total_sim_time,
        arr2D
      );

      response := xyce.read("YMEMRISTORRES");
      info("response: " & to_string(response));

      -- update time array to repeat pulse
      for j in arrT'range loop
        arr2D(0, j) := arrT(j) + xyce.reqT;
      end loop;
    end loop;

    xyce.close;
    test_runner_cleanup(runner);
    wait;
  end process;

end architecture;

And this is the matching C source: https://github.com/umarcor/ghdl-cosim/blob/develop/vhpidirect/vffi_user/xyce/vffi_xyce.c. Note that xhdl_run and xhdl_run_1D are included for illustration purpose. In practice, xhdl_run_2D should be used. However, vffi_user.h is currently a proposal only (precisely, for it to be introduced in the next revision of the language).

The logs of CI runs are available: https://github.com/umarcor/ghdl-cosim/actions?query=workflow%3Atest


Other refs related to AMS, Xyce, etc.:

@kev-cam ☝🏼 analog_t https://github.com/ghdl/ghdl/issues/1052#issue-531713465