Sayma_AMC documentation

[jbqubit]

@filipswit is leading effort to write formal documentation of Sinara. When will the documentation effort start? For inspiration see documentation for AFC.

[gkasprow]

He does schematic review right now. It is very time-consuming activity because every single connectivity and component pin need to be verified. He already found over 20 hardware errors in SInara AMC. Documentation is not that critical right now. He will start it in January.

[filipswit]

@jbqubit Thanks for inspiration documents, I think I'm familiar with them. As @gkasprow has written, these days I'm checking schematics. I will take me about 1-2 weeks. So January is good time to start.

[jbqubit]

Thank you for the update @filipswit. :)

[jbqubit]

@gkasprow said

@filipswit started seriously working on Sayma and Metlino documentation. Filip just finished schemtic review of all boards and found several issues. He will use tex templates from AFCK.

That's great! @filipswit Re-open this Issue (or create new ones) if there are questions.

[jbqubit]

In #129 @hartytp said

What do people think about the idea of using the Wiki as the main source of documentation? This could have a few advantages:

Easy for people to edit later on, for example to post measurement results or comment on hardware bugs we find

Allows us to keep the entire Sinara ecosystem documented in a single place, rather than having a tex file for each component

It's already a git repo, which makes version control easy

[jmizrahi]

@hartytp I strongly disagree with this suggestion. A manual needs to be a well-structured, thought-out document that a new user can read through and get a clear picture of the piece of hardware in question, what different design decisions were made and why, or reference when questions come up. Once the hardware is built, it will be mostly static. This is totally different from a free-for-all discussion forum (where one might "post measurement results or comment on hardware bugs"). It is also very different from the distributed, non-hierarchical node structure of a wiki, with fuzzy boundaries on where the manual begins and ends. The manual can certainly be updated with things like lists of known bugs in the future (and should be!), but this should happen in the context of a structured document. The manual should live alongside the more fluid back-and-forth discussion you're talking about.

[hartytp]

A manual needs to be a well-structured, thought-out document that a new user can read through and get a clear picture of the piece of hardware in question, what different design decisions were made and why, or reference when questions come up. Once the hardware is built, it will be mostly static.

This should all apply to the Wiki as well: the bulk of the content should be written during the initial documentation stage, and it should only undergo minor modifications to keep it up to date, or additions when new hardware is added to the ecosystem. When I talk about "measurement results" I mean significant pieces of characterisation, like phase noise spectra that all users want to see, not random things that people have done. I don't think a well-managed Wiki is ever a "free-for-all discussion forum" -- that's what the mailing list is for!

Would it make you less unhappy if I changed my suggestion to: make the documentation a GitHub project written in markdown (rather than Latex)? In that case, it's easy to integrate content from the manual into the Wiki. The major benefit of this as I see it is that we can have a single website which provides documentation for the entirety of Sinara (Metlino, Sayma, Kasli, DACs, etc). This is really helpful for new users who are trying to understand what the hardware ecosystem looks like and how everything fits together (it's much harder to have a stack of independent manuals that don't cross-reference properly).

NB we can restrict who has commit access to the Wiki so it's not a complete free-for all. Also, as the Wiki is a git hub project, we can also publish "releases" of it (e.g. after Sinara round 1), which freeze it -- or sections of it -- a certain points.

[jmizrahi]

The major benefit of this as I see it is that we can have a single website which provides documentation for the entirety of Sinara (Metlino, Sayma, Kasli, DACs, etc).

I fail to see how this cannot be done with a single PDF, if you really think they're that entangled. Who said that there has to be one manual per board?

BTW, I don't agree that they are that entangled at the level of hardware. Each of these boards can operate on its own without the others.

The intended software and firmware makes them part of a larger ecosystem, but software documentation is clearly going to have to be elsewhere, i.e. part of the normal ARTIQ documentation.

[hartytp]

I fail to see how this cannot be done with a single PDF, if you really think they're that entangled. Who said that there has to be one manual per board?

It can all be put into a PDF in principle. But, the plan is to start writing the documentation for Sayma and Metlino now, while the hardware ecosystem is still evolving rapidly. So, the documentation will either have to cover only those boards, and not cover the wider ecosystem (and how they interact with it), or run the risk of being obsoleted rapidly.

Why the preference for tex PDF over markdown files anyway? Markdown files can be saved and opened locally in a browser if desired (rather than over the web). They can also be complied into a PDF if that's important to you.

The intended software and firmware makes them part of a larger ecosystem, but software documentation is clearly going to have to be elsewhere, i.e. part of the normal ARTIQ documentation.

The way this hardware is used in an experiment, and the other hardware it plugs into makes it part of the wider ecosystem.

[hartytp]

I fail to see how this cannot be done with a single PDF,

Also, if you use MD or similar then you can make the documentation easier to navigate by including sidebars etc.

[jmizrahi]

Why the preference for tex PDF over markdown files anyway? Markdown files can be saved and opened locally in a browser if desired (rather than over the web). They can also be complied into a PDF if that's important to you.

If all we're talking about is document formatting platforms, I really don't want to argue about it. I refer to you to the many excellent references on how TeX is a wonderful document formatting system, produces attractive and easy to read documents, and why PDF is a very useful format. I was more talking about the nature and structure of a manual, not how it's rendered.

The way this hardware is used in an experiment, and the other hardware it plugs into makes it part of the wider ecosystem.

You are presuming a level of uniformity in how this hardware is used, which is both unfounded and unnecessary. How you use this hardware in your experiment, and what other hardware you plug it into, is on you to document. It does not belong in a Sayma manual. Sayma (and Metlino etc) are designed extremely flexibly, and can be used in all sorts of ways. I would leave the manuals separate.

[hartytp]

My thinking here is that we can write the documentation for Sayma/Metlino as pages in md, so that they can be viewed as standalone web pages or even compiled into a PDF if desired. However, as they're md, it's also trivial to import them into the Wiki github project. The same pages/files can then be used as part of a general Sinara documentation Wiki, which covers the entire ecosystem.

---

You are presuming a level of uniformity in how this hardware is used, which is both unfounded and unnecessary.

Sure, there will be large differences in how we all use this hardware, but there is a lot of common ground too and it's helpful to make that clear. As things currently stand, the ecosystem will mainly be made of a few different main kinds of things (uTCA boards, Kasli + extensions, etc) which interface together in a small number of different ways (IDC/DRTIO/backplane). IMO, it's worth making that clear in the docs, rather than just documenting the functionality of each module independently without providing any context of why that functionality was provided, or what it is intended to be used for.

e.g. The Metlino VHDCI is designed to interface with the VHDCI carrier. So, it's nice to be able to follow a link from the Metlino docs directly to the VHDCI carrier docs. The VHDCI carrier is designed to interface with all of the extension modules (not just the TTL ones, but also the DACs, ADCs, camera link, etc) so it's useful to be able to directly link from the VHDCI carrier to a list of the available extension modules -- and it's useful for that list to grow as more modules become available. It's also useful to know that Kasli provides an alternate carrier for these same peripherals, and that it can interface with Metlino via DRTIO/fibre. It's also nice to have a high-level introduction to all of this in the beginning of the docs.

[jbqubit]

I agree with @hartytp that it's useful to include cross linking in the documentation. Since @filipswit and @gkasprow are writing the documentation I'm inclined to let them choose the language.

Some observations.

• PDQ2 documentation is written in reStructured text, includes LaTeX inline, is rendered as html using Sphinx and is published on readthedocs.io (link).
• ARTIQ documentation is also written in reStructured text, rendered Sphinx and published by M-Labs.
• Sphinx supports Markdown. I personally prefer writing Markdown over reStructured text.
• A wide range of formats are supported by Sphinx including html, latex and PDF.
• Sphinx provides natural support for cross linking with existing ARTIQ-related docs and source.

[hartytp]

@jbqubit Works for me.

[jasonamini]

@jbqubit I agree. Along those lines, I suggest @filipswit writes a full manual to be delivered with the hardware (e.g. a TeX file or PDF) as the "official" documentation. We can then reformat it to an editable user version that can be posted and edited by the community.

Has anyone here used Pandoc (or similar) for converting between document markups? If so, does it work OK?

[hartytp]

Okay, so to summarise what's been said (as I understand it), the plan is to write two forms of documentation:

1. Board-level documentation of Metlino + Sayma. This won't describe how they fit into the general Sinara ecosystem (inc Kasli etc). It will be "frozen" and not updated to keep up with changes in the Sinara ecosystem. It will take the form of a PDF.
2. Overall Sinara documentation. This has both a high-level overview of the entire ecosystem and board-level documentation for all Sinara-compatible hardware we build (Sayma and Metlino as well as future Kasli extensions, and relevant support hardware like the RF amp we've discussed). It will be kept up do date with changes in the ecosystem, and will be extensively cross-linked. I imagine this looking like a fleshed out version of the current Sinara Wiki. This will link to the more-detailed board level documentation where appropriate.

WUT will write the board-level documentation for Sayma and Metlino as their "official" documentation effort. They will aim to do this in a way that makes it easy to integrate into the overall Sinara documentation.

Questions:

1. Who will be responsible for writing an initial version of the SInara documentation and merging the Metlino/Sayma docs into it?
2. Where will we host the Sinara documentation?

[hartytp]

@jordens How does this fit in with your plans for documenting Kasli + the other hardware you're planning?

[gkasprow]

Anyway, we will write dedicated manuals/datasheets for each board. They are quite universal and can also serve other purposes.

Our referable way is latex because we can nicely publish vector images but I’m open to other forms.

[gkasprow]

I also think that board manuals should be separate documents with all details which are needed to fully explore their possibilities.

Such things like assembly options, diagnostics, testpoints are not very interesting for ARTIQ user. And for such document we need good tool that supports vector graphics, search functions and looks more like manual then webpage.

This is example of such documentation

http://creotech.pl/wp-content/uploads/2015/08/CTI-AFCK_datasheet.pdf

There should be another general document covering certain use cases that embeds interactive links to detailed documentation and here wiki would be better idea since the content will develop intensively while board manuals will be static.

[gkasprow]

@hartytp it seems we came to the same conclusion:)

[jordens]

Pandoc works well.

[jordens]

@hartytp Right now I am just writing an overview about Kasli. I'll do so in reST. Bu I am much more concerned with the content that with the format.

[hartytp]

@jordens: right now I am just writing an overview about Kasli.

Great! If you can cover everything we need to make Kasli compatible extensions (e.g. plans for a Kasli BP if there will be one, etc) in the process, then that will be a really useful resource.

Bu I am much more concerned with the content that with the format.

As am I. Anything that integrates well into a nice, easily navigable webpage works well for me.

[filipswit]

As I understood: @gkasprow , @jasonamini , @hartytp , WUT prepare 3 separate docs (Sayma AMC, Sayma RTM, Metlino).

I prefer to write it in a TeX and export it to PDF, cause I've a good template. For now I'm working on block scheme, I2C map and some specifications of components of Sayma AMC.

[jbqubit]

@filipswit How's the documentation progressing? Please give an update.

[filipswit]

Generally I have cores for all documents. Some pictures are just placeholders for appropriate pics. Signal table for Metlino is incorrect. SaymaRTM.pdf SaymaAMC.pdf Metlino.pdf

Please say what else would you like to see in doc.

[dhslichter]

• [ ] It would be nice to have bus widths shown on the block diagram.
• [ ] More description (a paragraph maybe) in the first section, outlining the capabilities in a little more depth.
• [ ] Listing of valid/available input clock frequencies/types and connections
• [ ] Power draw information, and parameters for supply voltages
• [ ] For Sayma RTM, more accurate description of mezzanine connectors and how they are intended to be bundled

[jbqubit]

Thanks @filipswit! Some comments follow.

Sayma_AMC

• Xilinx Kintex UltraScale

  • [ ] how many transceivers, transceiver type, transceiver maximum data rate for FPGA speed grade

• MGT

  • [ ] define MGT to be Multi-Gigabit Transceiver
  • [ ] new Figure devoted to routing of MGT resources

• Figure 4: General Block Scheme

  • [ ] both DRAMs are missing
  • [ ] MMC is missing

• FPGA bootstrapping

  • [ ] discussion of how this works
  • [ ] how to load FPGA .bit into flash (is there a .bit for bootstrapping FPGA passthrough to SPI flash?)
  • [ ] how to use MMC to load .bit into flash

• FMC

  • group together FMC-related info into a single section
  • to what extent is implementation compliant with VITA FMC HPC?
  • clock and IO resources
  • VADJ voltage and maximum current draw
  • which IO banks are used on FPGA?

• SFP

  • to what extent is this implementation compliant with SFP standards?

• uTCA.4 compliance

  • [ ] discussion of ways board is divergent from uTCA.4
  • [ ] how to disable uTCA features like power interlock checking

• USB-UART

  • [ ] how does this this chip present its multiple UART interfaces to PC?
  • [ ] switch routing addresses
  • [ ] USB driver installation (if any)

• power

  • [ ] list power source options -- card edge vs stand-alone connector (connector name?)
  • [ ] voltage and maximum power consumption
  • [ ] is active cooling required for board?

• JTAG

  • [ ] interface options (UART vs IDC header)
  • [ ] addressing

• Signal Tables

  • [ ] column headers missing
  • [ ] define FP1, FP2 and P2P

Sayma RTM

• [ ] Get additional narrative about Project Description from the wiki.

• [ ] Cite high speed ADC and DAC ICs by name and summarize high-level specifications -- that's what this board is about!

• Figure 4 Block Scheme

  • [ ] SATA connectors
  • [ ] JTAG connector

• AFE mezzanine

  • [ ] short narrative discussing purpose of AFE mezzanines
  • [ ] AFE mezzanine is Analog Front End mezzanine
  • [ ] group together all AFE connectivity in single new section
  • [ ] list resources available to each AFE
  • [ ] list expectations for AFE modules
    • [ ] maximum power consumption
    • [ ] impedance, directionality, voltage ranges, maximum frequency
    • [ ] what to do to avoid damaging ADCs (fast and slow) on Sayma_RTM PCB
  • [ ] standoff length and screw size

• clock distribution mezzanine

  • [ ] short narrative discussing purpose of clock mezzanine
  • [ ] see all above for AFE mezzanine

• housekeeping signals

  • [ ] signals available to AMC-side MMC/IPMI (eg temperature sensors, current, voltage, fan) and addresses
  • [ ] signals available to RTM-side FPGA (eg temperature sensors, current, voltage, fan) and addresses

• clock distribution

  • [ ] include details from Sinara wiki on clock configuration options -- there are many!
  • [ ] use nomenclature for clock names used in wiki and Issues
  • [ ] RF back plane interface

• FPGA bootstrapping

  • [ ] discussion of how this works, say that there are two FPGA, one flash,...
  • [ ] not necessarily ARTIQ-specific
  • [ ] give link to ARTIQ documentation

• Figure 5

  • [ ] use AFE mezzanine not "mezzanine connector"
  • [ ] clock mezzanine is missing

Metlino

• [ ] short discussion of special role of MHC slot in uTCA system
• [ ] short summary of the star topology of uTCA backplane along with cartoon schematic
• [ ] diagram showing how this board stacks with others for minimal MHC operation https://github.com/m-labs/sinara/issues/21#issuecomment-267459815

See comments for Sayma_AMC. Most apply here.

[filipswit]

@jbqubit, @dhslichter Many Thanks for your comments!! It is much easier to write sth. with such a good feedback.

[jbqubit]

Now that I have an Sayma_AMC to play with I can see additional documentation needs.

• JTAG

  • which FTDI chip is used?
  • jtag clock speed
  • SCANSTA112
    • Say what the SCANSTA112 chip does generically.
    • Say how the chip is used in the context of Sayma_AMC -- sitcher mode, right?
    • What is power-on selection?
    • What are all the JTAG busses in the system? What are their SCANSTA addresses?
    • For known ICs with JTAG interfaces list idcode, IR length, human readable description
  • diagram showing how JTAG chain is extended by addition of RTM, FMC, etc. contrast with KC705 approach Figure 1-8 https://www.xilinx.com/support/documentation/boards_and_kits/kc705/ug810_KC705_Eval_Bd.pdf

• Front panel

  • what are all the LEDs? what do their different states indicate? uTCA leds
  • What does LPC1776 do when I press PB3 SW? Does it pull FPGA_RESET?
  • front panel USB port exposes four serial port... what are they connected to?
  • names for SFP connectors
  • SMA connectors

• list of headers on PCB

  • JTAG multiple
  • Cortex programmer plug, i.e. LPCXpresso (MMC_TMS...)

• SW1

  • say its there even if its not yet used
  • FPGA_M0
  • FPGA_M1
  • FPGA_M2

• I2C mux

  • mapping between address and devices
  • discuss how this is shared between FPGA and MCU
  • power on selection

• RGMII

  • discuss how this is shared between FPGA and MCU
  • power on selection
  • mention LD19

• interface between AMC FPGA and MCU uP

  • discuss how this is supposed to work (FPGA_SPI_FLASH_UPD)
  • I2C
  • SPI flash
    • describe mux' scheme
    • how are the different flash used? (N25Q256A13ESF40: IC23, IC24)
  • RGMII

[jbqubit]

@filipswit What's the status of the documentation? The hardware has shipped.

[gkasprow]

Expect documentation update from @filipswit https://github.com/filipswit witnin 2..3 days. He is working hard on it.

[filipswit]

@jbqubit Here you have some update of documentation. I will add here every few days some newer version.

SaymaAMC.pdf

[jbqubit]

It would be best to hold off on posting until you've generated a version you consider ready for feedback. In particular, please implement (or reject) all the suggestions I made for the previous version before reposting. That makes it easier for all the contributors to comment/discuss the same document.

Feedback on what you posted yesterday (Issue Date November 29, 2017).

p. 4 -- lots of the acronyms don't make sense without a short introduction to uTCA.4 p. 4 -- transceiver type, maximum data rate p. 4 -- spell out what MMC is an abbreviation for, give brief synopsis of properties of LPC1776 p. 7 -- back view of PCB connectors p. 7 -- consider illustrating the location and description of the different IC's in the way that iFixIt does p. 13 -- LED call outs look great! p. 18 -- excerpts from schematics are helpful at explaining; more verbose would help

Suggestion: Ask a friend who knows nothing about Sayma to read your documentation and ask questions. This will be a type of feedback that is invaluable as it is from the perspective of a person who will use the documentation to understand what the board does.

[gkasprow]

@filipswit you can add introduction to MTCA - just copy part of my article about it.

[filipswit]

p. 4 -- lots of the acronyms don't make sense without a short introduction to uTCA.4

Would you like an introduction or a glossary?

[gkasprow]

Do both.

[jbqubit]

@filipswit How's the documentation coming? Your docs are helpful for interfacing with M-Labs too. :) https://github.com/m-labs/artiq/issues/854#issuecomment-349905800

[jbqubit]

@filipswit ping

[filipswit]

This is what I've done so far.

p. 4 -- lots of the acronyms don't make sense without a short introduction to uTCA.4

There is a glossary, please say which one should I add yet. Introduction will come later.

p. 4 -- spell out what MMC is an abbreviation for, give brief synopsis of properties of LPC1776

abbreviation is in glossary. Properties will be later.

p. 7 -- back view of PCB connectors

will take a photo this week

p. 7 -- consider illustrating the location and description of the different IC's in the way that iFixIt does

p.17 - p18 Did you expect sth like this?

p. 13 -- LED call outs look great!

Thanks!

p. 18 -- excerpts from schematics are helpful at explaining; more verbose would help

Text in progress

SaymaAMC.pdf

Did you change the max size of attached file? I can barley fit with my doc in 10MB, had to cropp some ilustrations.

[jbqubit]

Thanks @filipswit -- not enough changes to warrant a re-read. Please respond to existing comments before sending for review. See eg https://github.com/m-labs/sinara/issues/133#issuecomment-346499637 Also please post both source and PDF next round.

[jbqubit]

@filipswit When you've implemented documentation changes in this Issue please close it. For next documentation review please create a new Issue for that revision. This is Issue is getting too long.

[jbqubit]

@filipswit How's the documentation coming? Please feel free to check items off (above) as you implement them.

[filipswit]

@jbqubit I've checked few boxes before in this topic: https://github.com/m-labs/sinara/issues/430 but from some reason I can't do it now. I can check it off but it's not saved. I it is the same above.

[jbqubit]

@filipswit It's getting critical that the documentation be finalized for Sayma v1. The PLL and Ethernet Issues will be soon resolved. Docs are one of the last parts of the WUT contract.

[jbqubit]

@filipswit @gkasprow I'm not confident that details such as the following are being added to the Sayma v1.0 documentation.

• [ ] JTAG always-on configuration cf https://github.com/m-labs/sinara/issues/463#issuecomment-358046337
• [ ] document which ICs have Ethernet access for Sayma 1.0. https://github.com/m-labs/artiq/issues/854

[filipswit]

I added most of the Issues, some of them may still nedds some corrections. But I think is good moment to get some new remarks;)

Actualy, I dont't know what you expect about this "document which ICs have Ethernet access for Sayma 1.0. m-labs/artiq#854" - for now just added placeholder.

The source code is already in git. SaymaAMC.pdf

[jbqubit]

Awesome @filipswit! Looking forward to jumping in and reading. :)

[jbqubit]

For reference, this version of the documentation provided by @filipswit has commit id https://github.com/m-labs/sinara/commit/e745783298d77aebc46f4e7b66e0e10ec732c286.

This is a big job @filipswit! Thanks for tackling it. Some comments are below.

Introduction to ARTIQ Sinara

Start with a section outlining the ARTIQ Sinara ecosystem. Language from wiki seems fine. https://github.com/m-labs/sinara/wiki/images/overview.png

---

Sinara is an open-source hardware ecosystem originally designed for use in quantum physics experiments running the ARTIQ control software. It is licensed under CERN OHL v1.2.

Control electronics used in many trapped-ion and other quantum physics experiments suffers from a number of problems. In general, an ad-hoc solution is hastily put together in-house without enough consideration about good design, reproducibility, testing and documentation. This makes those systems unreliable, fragile, and difficult to use and maintain. It also duplicates work in different laboratories. In addition, the performance and features of the existing systems (e.g. regarding pulse shaping abilities) is becoming insufficient for some experiments.

To alleviate those problems, Sinara aims to be:

• high-quality
• simple to use and "turn-key"
• reproducible and open
• flexible and modular
• well tested
• well supported by the ARTIQ control software

Sinara is currently developed by a collaboration including M-Labs, Warsaw University of Technology (WUT), US Army Research Laboratory (ARL), the University of Oxford, the University of Maryland and NIST. The majority of the hardware was designed by WUT. The work was funded by ARL, Duke University, the University of Oxford, and the University of Freiburg.

Following the ARTIQ model, an experiment consists of a Core Device (master) -- typically either a Metlino or Kasli -- controlling multiple slave devices in real time using ARTIQ's distributed real-time IO (DRTIO) protocol. DRTIO provides both gigabit communication links and time distribution over copper cable or optical fibres. It synchronises all device clocks, ensuring they have deterministic phase relationships, and enables nanosecond timing resolution for input and output events across all devices in the experiment. More detailed information about communication between devices and time distribution inside Sinara can be found here.

Sinara uses two main form factors for hardware requiring real-time control: microTCA (uTCA) and Eurocard Extension Modules (EEM). Non real-time hardware is typically connected to the host PC using Ethernet.

MicroTCA (uTCA) is Sinara's preferred form factor for high performance hardware with high-speed data converters requiring deterministic phase control, such as the Sayma Smart Arbitrary Waveform Generator (SAWG). Information about uTCA hardware, including a list of parts needed to build a Sinara uTCA crate can be found here.

EEMs provide a lower cost, simpler platform than uTCA for hardware that requires real-time control, but not bandwidth or complexity of uTCA hardware.

Extension modules connect to a carrier, such as Kasli or the VHDCI carrier, which provides power and DRTIO. They are designed to be mounted either in stand-alone enclosures, or in a rack with a carrier, and connect to the carrier via ribbon cable. More details about the extension module standard can be found here.

uTCA hardware interfaces with the extension modules either directly, using a VHDCI carrier, or indirectly, using a Kasli DRTIO slave.

---

1 Overview

Rename section to "uTCA.4 Overview".

"We make use of the most recent extension to the uTCA standard, uTCA.4" is not accurate. True we use uTCA.4. But latest is uTCA.4.1 is the latest.

Figure 1: There's an extra board in AMC 7 that's fine to have in the picture but confusing if the boards are not labeled. Add colored overlay labeling MCH, Sayma AMCs, power module.

New Figure: add photo of Sayma_AMC attached to Sayma_RTM sitting on bench so how they're connected is clear.

2 uTCA.4 RF Backplane

Just lump this together with pervious section. Yes, please add data sheet.

3 uTCA in Sinara

Delete this section since it's covered in the new Sinara overview.

4 uTCA parts and suppliers

Here's what I ordered in the past.

• NAT AC 600D, qty 1
• NAT MCH-Basic v3.5, mid-size front panel, qty 1
• NAT Native-R5, qty 1

5 Schematic / Layout Viewer

Add additional content.

• ARTIQ_EE folder is for designs made with the Mentor Graphics Xpedition Enterprise CAD tool.
• ARTIQ_ALTIUM folder is for designs made with Altium Designer CAD tool.

6 Sayma

Please adjust language. Sayma is distinct from SAWG. Sayma is hardware that supports M-Lab's Smart Arbitrary Waveform Generator (SAWG) gateware.

Combine Section 6 and 8. No need to list hardware components twice.

Link back to git repository for hardware versions.

• git commit hash for as-printed Sayma v1
• do we want a section that lists white wire modifications?

Analog Front End acronym is AFE.

6.1

Specification here. Link is broken.

~~ Flexible feedback to SAWG parameters planned. Specification here. ~~

6.2

~~ – motivation for this FPGA choice is here ~~


6.3

data rate is 1.2 GS/s at 16-bit

data rate is up to 1.2 GS/s at 16-bit

6.4

For transceiver usage make a table for each Transceiver (Rx and Tx) listing where its routed. Some transceivers are routed to multiple places (eg SATA and backplane fabric). I would leave out ARTIQ-dependent detail (eg DRTIO-upstream).

7 Glossary

Expand glossary... MCH, AMC, COTS, HEPP, RTM, RFBP, AFE, Sayma, Sinara, BaseMod, MixMod, EEM, uTCA, MMC,

Move glossary to very beginning/end. It's funny floating in middle of manual.

8

Merge with Section 6. Add subsections for AMC and RTM.

10

Lots of typos in preample.

Figure 7

This is an annoyingly partial-information schematic. I propose changing caption to "Sayma AMC external I/O" and just make it about that. Add other components only as needed to explain IO routing (eg FPGA, MMC, UART).

Here and in all similar following Figures...

• The thing in the teal box is not the RTM. Rather it's a connector. Give schematic label to that connector. Add arrow pointing to right labeled "to RTM".
• Similar comment for peach box labeled "AMC".

The clock information here is too spare to be useful. Remove Si5324 and tell reader to refer to Figure 10 for clock domains.

Add symbol for front panel Clock SMA. Draw arrow to floating text "see clock Figure 10".

I/O--> TTL Use name that matches schematics.

USB, SMA, SMA, SMA are physically on edge of board. Give them a different color box than yellow and put them on the front edge of the board in the drawing.

Give SMA same names as in Schematic.

Strike uFL and MSATA as these are internal-facing connectors.

USB-->UART Draw arrow to floating text "see USB interface in Fig XX".

Figure 8

Caption should indicate that you're illustrating transceiver routing. Seems like there's room on this figure to also include Ethernet routing (and it's corresponding SATA).

Use two distinct grey wires for each of two FPGA to AMC+SATA.

Fonts "1x" etc are too small.

Figure 9

Indicate in caption that hex code is I2C address. Put hex code in parenthesis in all cases.

The grey wires in Fig 8 are very confusing. This is exacerbated by lack of narrative in Figure caption. Explain why of two MUXs.

There are several I2C busses. Distinguish between them and what's attached to each. Give busses names so they can be referenced in other parts of text. You use some bus names in 19.1 eg I2C_SENS and PMI2C. Be consistent (I2C vs _I2C).

Discuss I2C bus sharing between MMC and FPGA. How does that work?

Circle indicating that grey lines are joined should be in the front-most layer of the drawing package.

Say what the different LM75 are attached to in Sensors block.

Figure 10

Put J27 on front side of the board schematic.

Discuss in caption role of different oscillators and what the point of the clock recovery chip is. See following writeup.

https://github.com/m-labs/sinara/wiki/SinaraClocking

No need to mention DRTIO as that's an ARTIQ feature.

What clock should I apply if I want to operate Sayma? What power level?

Figures 11, 12

Try to put these on the same page as the call out table.

Pair of RTM connectors and AMC card-edge connector labels are missing.

Figure 13

Try to put Figure 13 on the same page as the LED table. Move IC column to third column.

Doesn't the "nominal state" of the MMC involve some of the LEDs flashing?

Failure column doesn't seem useful.

Somehow the LED table ends up being difficult to read. I know I asked for "nominal sate column" but didn't help. For each LED I want to know what IC drives it an the relationship between LED state and IC state. Examples,

• Callout 5: LD13 is
  • off if MMC is OK
  • red if MMC is in an error state
• Callout 6: LD4 is
  • off if MMC is ????
  • blue if MMC is indicating ??? about hot swapping
• Callout 7: LD21 is
  • on if FPGA is configured
• Callout 11: LD14 is
  • off is MMC is ???
  • green if MMC is ???

Descriptors in call out table are not altogether helpful. I want to know what IC is connected to each and where appropriate what net it's connected to and its function. Examples,

• J15 and J26 are general purpose FPGA IO

• J27 is clock output driven by Si5324 (CDR_CLK_CLEAN3)

• XXX is FMC header -- why not call it J28??

• PB3 is push button routed to MMC that does ___

• USB to FT4232H IC that exposes 4 UARTs (See Section XX)

• JTAG headers are connected to which chips?

That callout 6 points to two different LEDs is confusing. Why not just give them different callout numbers?

12.3

Narrative is needed to discuss what SW1 configures. What is is connected to? Master/Slave with respect to what bus?

14.1

Typos.

Consider moving this down to 14.2.

The interaction between MMC and FPGA is not well described.

Please say exactly what the expected behavior is for "Selection between MMC and USB is preformes automaticly."

What does this mean?? "When micro-USB is connected S siglnal is high and Multiplexer connects USB to FPGA." Which UART?

14.2

Consider moving this section up one level to 14.1 Say how UARTS are connected.

- UART0 is connected to X, configuration is 1N8 xxx baud
- UART1 is " 
- UART2 is "
- UART3 is " 

I don't see the end user use case for this: Another FTDI USB driver, the D2XX driver, can also be used with application software to directly access the FT4232H on the Mini Module though a DLL.

15 JTAG

Tell user that there are two independent JTAG chains. Say what ICs are connected to each. Say what options a user has for driving each chain.

Move narrative about Scansta112 to before Figure 19. Figure 21 is super confusing. More explanation is needed in caption.

"JTAG can be connected either from USB (FT4232H) or from IDC header." Which IDC header? Which JTAG chain?

How does Sayma JTAG relate to uTCA JTAG?

16 FPGA bootstrapping

This section isn't helpful. I'd just delete it.

17 Power

17.1 Power supply

Introductory orientation paragraph is missing. Outline.... As card is inserted... power is applied to MMC first. Explain that there are XXX power rails derived from +12V. There are several power chips ... VCCINT PSU, P5V0 PSU (helper PSU) and Exar LDOs. They are all are actively configured by MMC using an I2C bus, right. There's a particular power on order. What is it? Exar firmware monitors current and responds if its too high. Can MMC readout Exar debug information? How about VCCINT and P5V0 information? Is MMC supply information exposed to user?

Pull content from 17.2.3 into this section.

17.2.2 Exar parameters

Figure 24 needs a caption. What's the point of this Figure? What are the different cells? What's the blue block? And OTP? UVLO? Please include narrative that describes the very nice power system you've built.

If there's a story to tell in Figure 25 additional narrative is needed. What's a group? Is there a power configuration file? Or is it embedded in the MMC firmware?

Figure 26

Showing a bunch of screen shots of Exar tool is lame documentation. What are you trying to say in Figure 26?

18 MMC

configures CPU, UART

• Where does CPU get it's configuration information? UART or Flash?
• Are there LED changes that indicate this has completed?

sets port directions

• What do you mean by "port"?

configures I2C switch base address initializes I2C controller and chain (switch)

Which I2C busses?

checks if RTM is inserted, if yes, then enables its power, waits 200ms and initializes RTM power supply via I2C. It also configures Si5324 on RTM

• Which I2C bus?

runs task.

• Do you mean that that the CPU loops over the tasks that follow?
• 'blinks front panel LEDs alternately' which LEDs exactly?
• 'If not, it keeps Ethernet PHY in reset state.' Missing earlier in documentation is how Ethernet is shared between MMC and FPGA. Say effect of holding Ethernet PHY in reset state.

18.2 Bootstrapping MMC

There's more detail needed here.

Compiling

What version of LPCXpresso? Are there any particular configuration options? Say which chip to taret.

header flashing

What version of NXP programmer software? Is NXP programmer a particular piece of hardware? If so which model number? Which header on PCB? Use designators from earlier.

USB flashing

How does this work? Why use USB vs header?

19 Housekeeping

Narrative discussing uTCA housekeeping concept where MMC collects info and passes on to MCH via IPMI. Discuss that OpenMMC will run on MMC chip and that an OpenXXX will run on MCH and collect AMC device housekeeping information.

Discuss how I2C busses are shared between MMC and FPGA. Do this here or refer to earlier section where it's discussed.

19.2 Safety interlocks

temperature and current safety interlocks. Is this enforced by MMC or by Exar chips? How does a user know if an interlock is active?

21 Factory acceptance tests

This is being written by Creotech, right? Please ask that they commit their test jigs and code to repository.

A Appendix

Give this Appendix a name.

Gateware configurations

https://github.com/m-labs/sinara/wiki/Sayma-gateware#data-and-sample-rates

Missing

• OpenMMC interaction.
• Ethernet sharing between MMC and FPGA.

[jbqubit]

@filipswit Please refactor in light of my comments. I'd like to do review of AMC and RTM docs at the same time.