search

zignig / artifex

Declarive solver for stuff
0 stars 0 forks source link

Top level search space for python things #1

Closed zignig closed 1 year ago

zignig commented 3 years ago

Declarative interface for making actual things.

zignig commented 3 years ago

Have a simple declaration that attempts to find a solution. If that does not work ASK for more information. build the thing from what we KNOW.

zignig commented 3 years ago

@jamesbowman @xesscorp @michaelgale

What does a top level python search api look like??

xesscorp commented 3 years ago

Instead of trying to define the system from the top down, try working from the bottom up. Take a simple example (e.g., 48V -> 5V converter). What inputs & outputs are needed at each stage in the design flow? What information has to flow backwards during an iterative design to influence preceding design phases? Once you've done that for a simple design, you can start to ask questions about more complex situations.

zignig commented 3 years ago

I envisage it doing both. Top down for what, bottom up for how.

Take skidl for example. If you had a collection of voltage regulator circuits (linear,switch mode, etc ) you declare at the top that you want some volts and artificial would select one you have used before or raise a query saying that it needs one.

Did you read the GOAP paper? Its a solution planner, these are the things that I have find me a way to what I want.

Need to do more thinking.... :)

xesscorp commented 3 years ago

I'm thinking more that bottom-up is to imagine a specific example and imagine how it would work. Then you would have a concrete implementation to expand upon. If you just try to imagine a general electronics system synthesizer, it's hard to tell where to start. You just keep doing more what-if analysis without doing anything specific to try and see how it works.

michaelgale commented 3 years ago

I agree with @xesscorp , try to "storyboard" an imaginary use case. You could start with some "pseudo code" python statements which someone would use to interact/configure the "solver engine". Then itemize the tasks the solver would perform. And finally itemize the outputs the solver would generate.

From my perspective, I would be happy with a scriptable EDA tool suite. Currently, the workflow of schematic drawing, PCB placement (guided by ratsnest), and finally layout is pretty time consuming. In particular, the schematic drawing phase involves a lot of tedious drawing of net connections (often repetitive and redundant). Thankfully, SKiDL offers us a remedy to this part of the problem.

The next step, is simply to make the PCB placement and layout phase scriptable as well. That way, from one or more python source files, you can completely describe and generate all the assets for making a PCB.

The next extension would be integration of HDL / FPGA synthesis into the model. It would be awesome to close the loop by allowing pin assignments for the FPGA be optimized for the PCB to ease routing congestion and complexity. This way, the entire design process could be globally optimized not only for yielding a better overall design, but also to remove a lot of manual drudgery and tediousness of interacting with different tools, files, workflows, etc.

zignig commented 3 years ago

@michaelgale agreed, My intent is not just EDA but a complete thing pipeline. Like your awesome fxbricks pipeline. I have done similar work with cqparts.

I'm looking at an imperative pipeline. I would like this thing at the top level. It has a toolbox of physical solutions that it will try to solve and return the best guess and a probable path to find the thing

I agree that the 'storyboard' is the best way forward, I'm thinking of starting with an LED and a resistor in a printed box and work up/down from there.

xesscorp commented 3 years ago

Some ideas for a quick & dirty prototype

Instead of working with generic, individual components (resistors, capacitors, ICs, etc.), maybe start with a catalog of prebuilt modules. These could include power supplies (AC-DC, DC-DC of various voltages and current capabilities), processor boards (Teensy, RPi Pico, etc.), FPGA boards (GOWIN Nano or maybe some of those M.2 FPGA modules), displays, analog I/O (ADC, DAC), etc. These would each have a footprint and pinout as well as a 3D model.

The user might start with a general idea for the design such as "sample two analog signals and transmit them to a remote server". The EDA system needs more info on the signals such as amplitudes and frequencies. If these are audio-range signals, then it might select a microcontroller board with integrated ADCs. But if the ADCs don't have enough resolution, it might add an external ADC module. If the ADC module has a parallel interface, the microcontroller may have insufficient I/O so the system moves to an FPGA board. The FPGA board may require 5V, 3.3V and 1.2V supplies, so one or more power supplies would have to be added. If this makes the design too large or expensive, maybe move back to the microcontroller board but use serial ADCs that need less I/O. And so on... The idea is similar to GOAP but with a more complicated space of actions and goals. How to represent these so as to capture the wide range of capabilities and constraints is not immediately clear.

Once the modules are selected, they need to be connected. This isn't trivial as some module I/O is better suited for particular tasks. For example, the I2C ports of the microcontroller should be used to interface to the serial ADCs. But this constraint isn't applicable if something like an FPGA was used. So constraints and capabilities of the I/O also have to be represented.

If the interconnections are specified using something like SKiDL, then that could output a netlist. Then the netlist could be converted into a PCBNEW layout using kinet2pcb. Some crude placement could be done using HierPlace. At that point, the I/O ports for the design might be moved to user-specified points around the edge of the bounding box of the other placed modules. Some mounting holes could also be added. Then the nets are routed using FreeRouting. (All the sensitive wiring should already be done on the modules so FreeRouting will have an easier job.)

With the PCB placement & routing done, the 3D models of the modules can be used to calculate the total 3D bounding box and a simple box with mounting posts and cutouts for the I/O ports can be designed using cadquery/cqparts.

The end result is a system that generates inefficient, limited designs in ugly housings. But it creates a skeleton that can be fleshed-out with better actions/constraints, design-space search algorithms, PCB place & route and mechanical CAD. And those improvements can be done independently because the steps in the design flow are loosely-coupled.

zignig commented 3 years ago

Wow ! thanks for the reply. I've read it twice and my head is still spinning.

SO..

Define some base level modules in skidl + META at the low level.

power: waveform: DC voltage: 5 current: 500mA SKIDL : python stuff

If I ask for power as long and the draw is less than five hundred milli amps , this is a valid construct. The system will try to give me one of these ( thruhole linear with a cap and a resistor and a nice layout). one of these to drop onto a PCB and make sure that it fits into a box.

The box is defined beforehand, it's one you bought for 3.5 DollarBucks and it's two plastic bits and four metal screws. OR it makes a 3d printed object that will contain the pcb and make sure that thermal loading is disipated.

I reckon that a top level document that you elaborate to a secondary file that has UUID and signed datablocks, you could even put it in a blockchain ( but that conversation can only be had over a single malt whiskey ).

Anyway, when I find some time I will prototype, artifex is a crap name ( There are ~200 on github ), I am thinking spoon_bender.

michaelgale commented 3 years ago

To be really honest, I'm not sure I really want a computer to make my things for me? Part of the fun is doing the design. I've always viewed engineering as equal parts creativity and applied skills. Software tools are best used for:

What I find exciting is the "candy store" of tools you folks have helped developed for folks like me. We now how have great tools that once were only available as expensive proprietary commercial offerings. Not only that, we now have two flavours of these types of tools:

  1. Desktop GUI/productivity: e.g. KiCAD + FreeCAD
  2. Scripted/code based: e.g. SKiDL + CadQuery

A concrete example of the power is the fact that our company (Fx Bricks) has designed high precision plastic ABS molds + metal stamping dies for 19x products in less than a year. If this was done by hand, it would have taken 10x longer and would no doubt have more errors and less precision. We also have produced 11x PCBs and can panelize our own PCBs with integrated production test / programming harnesses designed into the panel. Lastly, from our scripted CAD, we can generate all the other good stuff: documentation, photo renders (including HQ Blender renders for e-commerce store assets, catalogs, etc.), animations (for YouTube or production/assembly aids), etc. etc.

From my perspective, this aim to make an intelligent agent for design is best served not necessarily for the actual "designing" but "managing" of the process. For example, if you wanted to make a boxed PCB with certain rough I/O and power requirements, I would like a tool to do this:

  1. Create a set of pre-populated folders and bundle them into a Git repository
  2. The folders would contain templates for python scripts for EDA, Mechanical, renders, etc. as necessary
  3. Some folders would contain pre-fetched data sheets for candidate components
  4. One folder would contain a rough spreadsheet template with aggregated material BOM + costs
  5. A top level README template would be filled with hyperlinks to some of the top level design scripts, data sheets, relevant links to reference articles, etc. The top level requirements would guide how this repo gets populated. Once that task is complete, I can then jump in and get on with the fun part of designing!
xesscorp commented 3 years ago

@michaelgale, a useful design system should include the functions you've described. Many people would like to have some of this functionality. Are you close to starting/completing any of this for Fx Bricks?

The interesting step is #3: pre-fetch data sheets for candidate components. How would you do that? It seems analogous to the phase in my system where modules are selected based on overall design criteria and interactions between modules. This may be the most important phase of design because the more knowledge there is of available modules, the more likely the design can meet or exceed requirements. An automated system with a large and expandable library would be a real asset for design exploration.

I don't really see this as removing the designing part from the engineer because the design phases are loosely coupled and the intermediate results are available for manual modifications:

  1. If you don't like the parts that are selected, just constrain some of the modules with your own selections and throw it back into the system for the remaining design phases.
  2. If you don't like how the modules were connected, just edit the SKiDL and throw it back into the system.
  3. If you don't like the module placement or routing, just re-arrange/re-route it in PCBNEW and throw it back into the system.
  4. If you don't like the housing, just modify it in FreeCAD or cadquery and throw it back into the system.
zignig commented 3 years ago

omg, both of you are correct !!!.

start with a simple heirfile.

name: stuff: key: when:

after that the solver attemps to build any unknown thngs.

michaelgale commented 3 years ago

@xesscorp regarding your question about whether we're close to having a semi/fully automatic system at Fx Bricks--the answer is mostly no. Mechanical CAD is somewhat easier to automate than electrical.

For the 3D model generation of items which are closely related (e.g. our Lego compatible train track), this is easily parameterized/automated. We provide a dictionary or YAML file with specifications, and it procedurally generates a track item satisfying the specification. This is easy because it is rule based around the geometry of how the track should look and how it should conform to the Lego 8 mm stud grid. Automation helps because it removes the repetitive monotony of generating things which are closely related or share many elements in common.

However, for electrical circuit design, it is unlikely have a "cookie cutter" template specification. Beyond having a "toolbox" of sub-circuits, e.g. power supply/regulation, USB interface, serial EEPROMs, etc. every design is likely to be unique in many respects. This is where I see SKiDL being already useful for this problem. I can now have a source code based "toolbox" of circuits that I can draw upon to support a new design. Connecting these elements in a hierarchy is 10x simpler with writing code than drawing it in a Schematic capture tool. In terms of PCB layout, I was unaware of HeirPlace (thanks for the reference!) which can at least help get you started by co-grouping chunks for the component placement phase. Lastly, layout is both an art and science. For some high performance or RF based designs, I would not trust an automated router. There is so much judgement required in terms of the geometry and physical location of elements that influence:

Lastly, I will share one thing that I find myself doing more in design related tasks: use YAML files. I tend to use YAML files to itemize specifications, comments, parameters, build variants, etc. I also use them to store constants, lists, reference data, etc. YAML files are easy to edit and maintain. I design many of my python scripts to ingest these YAML files either into dictionaries or as dotted class attributes. These YAML files can then be shared among both Mechanical and Electrical CAD generator tools so that key dimensions of say mounting holes can be used for the PCB and corresponding mounting bosses on an enclosure for example. This simple "hack" means I don't have to remember to transcribe the coordinates of a mounting hole from one python script to another (and inevitably make a mistake!)

zignig commented 3 years ago

I have been using ansible at work recently and I think that many of the concepts will translate across nicely. The YAML was a bit confusing at first , but makes sense once it gets bigger.

I think a loop that watches a folder for changes and runs the construct across all the files. Solving and running as it goes.

xesscorp commented 3 years ago

@michaelgale , I think you should publish something detailing your use of YAML in your design flow (if you haven't already and if it's not proprietary).

I agree general layout and routing is difficult. That's why I was thinking of using pre-built modules where the all the primitive components are already arranged and the detailed routing is done.

I found this paper about an automated design system that selects components for a software-defined radio system based on user-defined criteria. (This was done about 10 years ago using MS Visual Basic.) That's kinda/sorta what I was thinking of here but maybe based on a constraint satisfier.

A toolbox of SKiDL subcircuits would be very valuable. The lack of one is a major deficit of SKiDL. I've had that on my TODO list for years! Here's a high-level list. Let me know if you have any general or detailed additions. That might inspire me to actually start building it.

zignig commented 3 years ago

Basic package layout started