Open jbqubit opened 4 years ago
@jbqubit is there a definitive list of the gateware/software support that was funded and the relative priorities? Eg was 1gsps operation funded in the end, and where does that lie in the priorities list?
@marmeladapk when do you think you can ship a board to m-labs?
@marmeladapk told me a few days ago that Sayma AMC passed initial tests and they are working on bringing up other boards. I'm out of the office so don't know the exact status.
@hartytp The contracts are detailed here.
Hardware solicitation W911QX-19-Q-0060 https://www.fbo.gov/index.php?s=opportunity&mode=form&id=9f270a64ff963bf91530947b5449845b&tab=core&_cview=0
Software/gateware solicitation W911QX-19-Q-0166 https://www.fbo.gov/index?s=opportunity&mode=form&id=6d0280e7265cd24ebbd85e1dc1b06fdc&tab=core&_cview=0
Details you asked about are in Extended Deliverable C.3.2.2.1 of the second solicitation.
@akaminska
Thanks @jbqubit (although, I think you got the hw/software contracts the wrong way around there).
Am I right in interpreting that as saying that the first priority (after getting the bring up working and RF output reliable) is phase synchronisation. After that, 1GSPS operation is next?
We started testing Sayma AMC. So far power, I2C and SDRAM were tested. We're waiting for Sayma RTM to test GTx links and other functionalities. After that we could send one Sayma AMC to @sbourdeauducq. Then after we verify that RTM is working correctly we could send RTM.
Great
For everyone curious out there @sbourdeauducq - the first two Sayma RTM boards are out of production and we started working on them already. More details soon.
@akaminska @marmeladapk Any update on the RTM testing? Also, assuming the tests look ok, when are you planning to stuff more boards?
@sbourdeauducq
This Wednesday I finished checking power supplies on Sayma RTM (tedious process, when there are many of them). Wojciech ran parts of test suite and so far DAC JESD PRBS is working on 4GBPS, today we'll know if 10GBPS is working.
On AMC we checked that it gets power from crate (openMMC).
Things that we are planning to do before sending one pair of boards to @sbourdeauducq:
We plan to green-light further production this or next Monday depending on progress. Next week we should also get Basemods.
However I have bad news concerning Metlino as PCB production is delayed. I cannot tell right now when it will arrive (>1month?).
That's great! Good work @marmeladapk
IMO Metlino delays are less important since we can use Kasli instead, but Sayma has features that aren't present in any other system. Do you know when more Sayma boards will be stuffed?
Both Hardware Developer and Software/Gateware developer contracts share specifications and terminology. I created a wiki page to keep this sorted.
I created several Issues for Hardware Developer tasks based on the contract. @marmeladapk please create additional tasks corresponding to the remaining deliverable items.
@sbourdeauducq asks
Do you know when more Sayma boards will be stuffed?
I don't expect that the will be stuffed until the first round of stuffed boards pass testing. But that's at the discretion of @marmeladapk. However the contract is explicit that the Hardware Developer is to complete baseline testing (HT3 #122) prior to prototype shipment. This workflow is designed to maximize liklihood that hardware bugs are found and patched prior to shipping.
IMO Metlino delays are less important since we can use Kasli instead
Best if Metlino not delay shipping of Sayma prototypes. Testing Sayma using Kasli core device is a fine way to boostrap the future all-uTCA Metlino+Sayma system.
Basically the only issue that is preventing us from sending Sayma to @sbourdeauducq is #126. We tested JESD @ 10 GBPS and PORT0 @ 4 GBPS. We postponed testing DAC output until we can figure out flash issue.
Other than that, AMC+RTM have problems with powering up in the crate, but we'll debug it after flash (AMC powers up fine, however I'm not sure that it's overcurrent since I checked that RTM consumes only around ~10 mA on 3V3MP).
The work plan is structured so that other labs/contractors receive hardware only after thorough testing is complete at Creotech. Recall that HT3 calls for both thermal tests (in TS7) simulation of heavy loading (in TS2) prior to shipping PCBs to other labs.
@jbqubit First photos, that we have of these boards (before any hardware changes).
🎆 cool!
Splendid.
What's the status of Metlino? On 9/6 word was it would be a month.
By email @marmeladapk said "I don't anticipate that we'll be ready to send Metlino until December." Is M-Labs is OK with proceeding with tasks including Sayma-Sayma synchronization using Kasli as Core Device until Metlino is available?
Yes, either works.
@marmeladapk What's status of BaseMod v2.0 and TestMod v2.0? How's the testing coming along?
@marmeladapk Ping. Please let us know the status.
Sorry for the delay, I wanted to catch up with one of the developers.
All BaseMods are on my desk and are waiting for me to test them.
Stress test is complete and we didn't find any issues.
Additionally I fixed one issue that disabled crate power when AMC+RTM were starting. Then I was able to start 7 Saymas+RTMs in a small crate.
Clip of all Saymas starting up in crate
What still remains for us to do:
NAT power supply supports staggered power-on for AMC slots. Have you tried using that feature?
Seems odd that the LED load indicator on NAT-PM-AC600D doesn't change as hot swap handles are pushed in. Guess that must be without thermal dissipation with TestMod_FMC resistors?
When can you ship TestMod and BaseMod to M-Labs?
ping @marmeladapk
Update:
We need to write support for:
All of these should be easy. After we finish this we'll do all tests from HT3 which require DAC output + test all boards. I hope we'll be able to distribute them before holidays.
@marmeladapk When we spoke on 11/27 you said:
@marmeladapk Ping.
* All AMC and RTM cards were tested.
Did you get any RF out with ARTIQ?
@sbourdeauducq No, DAC setup on ARTIQ now fails on checking SYSREF margin (I'm speaking from memory now, I cannot remember the exact message now). However I checked that SYSREF phase was changing during this test with oscilloscope. If you want we can try to debug it (and your card) and discuss it in https://github.com/sinara-hw/Sayma_AMC/issues/141.
We can test SYSREF/sync and RF output separately.
From call with Creotech this morning. Several Issues remain for HT3.
Front Panel (FP) status
@jbqubit All basemods have frontpanels moutned, however without any markings.
@marmeladapk When we spoke on Friday you indicated that the Sayma and Metlino front panels would arrive at Creotech on Monday. Did that happen? Can you now ship Sayma to M-Labs?
@marmeladapk asked that I post the contract docs to the wiki as a reminder of the division of labor for Sayma v2. They are here.
Creotech shipped the following to M-Labs:
Creotech shipped the following to Oxford:
@hartytp @sbourdeauducq Please indicate when you've received them.
Received all. I still want to swap at some point the Sayma boards without front panel and with 1 DAC not working for boards with front panels and both DACs working.
Please edit PCB modification wiki page to indicate for which board "1 DAC not working."
Received all
@marmeladapk When can you ship 2 full Sayma to M-Labs? I understand @sbourdeauducq will return to you his broken board upon receipt. Sayma-Sayma synchronization is not yet been properly tested so this is a high priority.
https://github.com/m-labs/artiq/issues/795#issuecomment-605371138
@jbqubit what's the point of worrying about Sayma->Sayma synchronization when FPGA->DAC synchronisation does not yet work reliably on a single Sayma? Surely the bottleneck here is someone looking into FPGA->DAC sync (@sbourdeauducq are there plans to get back to that?)...
@hartytp Agreed that there getting FPGA-DAC synchronization to work is the first step. M-Labs needs a pair of Sayma to validate FPGA-DAC synchronization via DRTIO in a multiple-Sayma context. A pair of Sayma is also needed for later analog synchronization tests. It takes a week or more to ship hardware to HK.
@marmeladapk When can you ship 2 full Sayma to M-Labs?
ping @marmeladapk
Hi Joe,
We are working on shipping the Saymas. Logistics is not easy when all R&D is on home office. The current plan is that a mechanical engineer will come in to the R&D office on Monday to fit the panels (they arrived, but need a few adjustments). On Tuesday we will have a possibility of transporting goods from our R&D office to our main office and warehouse for packing and preparing for shipment. On Wednesday we should be able to ship the package.
Best wishes, Anna
Wiadomość napisana przez Joe Britton notifications@github.com w dniu 02.04.2020, o godz. 17:12:
@hartytp https://github.com/hartytp Agreed that there getting FPGA-DAC synchronization to work is the first step https://github.com/sinara-hw/Sayma_AMC/wiki/Sayma_v2_contract_m-labs#c301-dac-fpga-synchronization. M-Labs needs a pair of Sayma to validate FPGA-DAC synchronization via DRTIO in a multiple-Sayma context. A pair of Sayma is also needed for later analog synchronization tests. It takes a week or more to ship hardware to HK.
@marmeladapk https://github.com/marmeladapk When can you ship 2 full Sayma to M-Labs?
ping @marmeladapk https://github.com/marmeladapk — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/sinara-hw/Sayma_AMC/issues/121#issuecomment-607906488, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK5FLFLXUOE26BMQZJBBJG3RKSTONANCNFSM4IQLBRZA.
@akaminska OK. Thank you for the update.
I'm annoyed that this wasn't done a month ago. You guys did great work on Sayma v2. I want to be sure that M-Labs has an opportunity to work with a fully functional board pair.
@akaminska said via email to @sbourdeauducq ...
We are preparing for you in the first half of next week a package which will contain two pairs of Sayma AMC + Sayma RTM (we will ask you later to return the malfunctioning Sayma AMC + Sayma RTM pair that you have at M-Labs now), 1 Metlino and 2 BaseMod, all with panels. We will include a set of components required to integrate Metlino and MCH and prepare a documentation showing how to perform this process.
On a separate note, we will also include a Kasli v2 in the package.
@sbourdeauducq asked
What about Metlino, is it ready?
I agree with @sbourdeauducq. Do please also send a Metlino to M-Labs.
As I mentioned in the e-mail you quote, we are sending 1 Metlino to M-Labs as well :)
Wiadomość napisana przez Joe Britton notifications@github.com w dniu 06.04.2020, o godz. 19:05:
@akaminska https://github.com/akaminska said via email to @sbourdeauducq https://github.com/sbourdeauducq ...
We are preparing for you in the first half of next week a package which will contain two pairs of Sayma AMC + Sayma RTM (we will ask you later to return the malfunctioning Sayma AMC + Sayma RTM pair that you have at M-Labs now), 1 Metlino and 2 BaseMod, all with panels. We will include a set of components required to integrate Metlino and MCH and prepare a documentation showing how to perform this process.
On a separate note, we will also include a Kasli v2 in the package.
@sbourdeauducq https://github.com/sbourdeauducq asked
What about Metlino, is it ready?
I agree with @sbourdeauducq https://github.com/sbourdeauducq. Do please also send a Metlino to M-Labs.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/sinara-hw/Sayma_AMC/issues/121#issuecomment-609918809, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK5FLFMIEKHRVESWUYIGCCTRLIDVHANCNFSM4IQLBRZA.
To make it easier for Sinara community to follow Sayma progress let's put high level discussion of hardware production and testing timeline in this Issue. As usual create independent Issues for specific test results and problems.
Here's the latest as I understand it.
@gkasprow @marmeladapk