sinara-hw / sinara

Sayma AMC/RTM issue tracker
Other
42 stars 7 forks source link

Ethernet carrier for EEM #473

Closed jbqubit closed 6 years ago

jbqubit commented 6 years ago

In #461 there was discussion of adding an Ethernet-to-serial interface to a temperature controller EEM (https://github.com/m-labs/sinara/wiki/Thermostat) proposed by @hartytp. Presently there are 2 EEM carriers (interface boards) in the Sinara ecosystem: Kasli Carrier and VHDCI Carrier. An Ethernet-to-EEM interface would be a third low-cost option for interfacing with EEMs. Let's discuss that possibility in this Issue as it doesn't relate to a specific EEM.

jbqubit commented 6 years ago

@dtcallcock noted

Ethernet connectivity would be nice for standalone operation. A transparent Ethernet->UART convertor like this would only add about $10 to the board cost and could easily be DNPed. The only reason I can think not to add this is if it's a buggy piece of junk - anyone used one?

@hartytp replied

I did think about that, the reason I decided against it was just the usual feature creep etc., extra bugs etc. My feeling is that it's better just to make an Arduino/Beaglebone/whatever EEM shield and put the ethernet on that.

@dtcallcock

I think the board will be much less likely to be used standalone if you need to faff around with a second EEM (that will also need firmware and debugging). Another issue with things like Beaglebones is that as they run an OS, which means we can't easily put them on the NIST network. I can also imagine a few other EEM boards in a similar vein to this one that would benefit from cheap, simple and robust ethernet connectivity, so if we can find a good solution it could be reused.

@gkasprow

Let's use Ethernet solution we use in uTCA chassis based on Texas Instruments CPU compatible with RUST. @sbourdeauducq will like it. It's a single chip solution.

@gkasprow

The module is similar cost to ADI and Maxim chips that do similar job, but require analog compensation which is painful. In case of troubles with availability we can always develop replacement - the HW part of this module is trivial - just CPU and DC/DC converter, but the FW is considerable effort to develop.

@dtcallcock

Given we don't need fast control and it's unlikely that any physics experiment will ever need to talk to this module, aren't we just using Kasli/Metlino as an expensive and inconvenient Ethernet->UART convertor.


@hartytp said

Re the ethernet thing:


@dtcallcock said

I think I'm missing something here. Right now, using the core device to intermittently talk to misc non-real time hardware, do logging, and other general housekeeping is very undesirable and unscalable as it directly interferes with running experiments. Having a direct connection from the hardware to a PC running an Artiq slave seems like the right way of doing this and the one that was envisioned from the outset. However it seems that @hartytp is envisioning extra Kaslis being dotted around the lab to handle this kind of thing instead of PCs. Is one picture correct or is it really just a matter of personal preference?

One can easily covert it to USB or ethernet using readily available adapters and one can use it directly with most micro controllers.

If my picture above is correct then I think any piece of hardware where direct connection to a PC is the preferred mode of operation should have the necessary hardware (eg. ethernet) built in. One of the big wins of Sinara for me will be not having to add my own adaptors or micro controllers to things.

I don't want anything on here that needs firmware if at all possible (that's one of the big advantages of the thorlabs IC IMHO). The cost of programming boards at production then debugging and maintaining firmware is just too great for a project this simple in my opinion.

The ethernet to UART convertor I suggested obviously has firmware but, just like the Thorlabs controller it is pre-loaded and hidden from the user (as @sbourdeauducq points out).

The ethernet connection is generally quite complex and could be a large time sink to debug and get working robustly

These convertors are supposedly transparent (like the Ethernet to GPIB convertors we use). I mean if they don't actually work, that's one thing, but if they do then it doesn't seem like any extra work. Does anyone have experience with these? If they are junky then having something equivalent would be a great addition to Sinara in my opinion. In that case I think it should be considered a separate project (albeit one that Thermostat can take advantage of) and we should move it to another thread.

jbqubit commented 6 years ago

Comment from @restelli by email. Topic: JQI's SPI to Ethernet solution. Reproduced here with permission of @restelli.


Hi Joe, We're using Wiznet+Teensy to talk with Ethernet. Wiznet is a chip with hard-coded TCP-IP stack and integrated PHY that communicates through SPI interface to a microcontroller that must be used to configure the Wiznet and transfer data. http://www.wiznet.io/product-item/wiz850io/

Teensy is a microcontroller that is Arduino compatibile and has already libraries to talk with the Wiznet. https://www.pjrc.com/store/teensy32.html

You can therefore program the Teensy to open a TCP-IP connection and a serial connection and you have a custom serial-to TCP-IP interface, or you can just program the Teeensy to directly talk to the temperature sensor or whatsoever... Teensy has already very well written C++ libraries for the Wiznet.

gkasprow commented 6 years ago

I use Wiznet as well, i.e. in Booster design but anyway it needs firmware to talk to it. As @sbourdeauducq noted some time ago, it is ASIC so would be impossible to port it to IPv6 or fix some issues. I'm not sure if it is real problem. TM4C is real single chip solution but so far with no Arduino support, which can be drawback for some communities :)

gkasprow commented 6 years ago

If we want to use low cost Ethernet to LVDS converter, maybe it's better to stick with 20$ Microsemi SoC that has Cortex M3 + tiny FPGA in it. One can also use any Cortex M0 CPU that has feature that SPI and UARTS can be mapped to any IOs. And It is supported by Arduino, at least SAMD21 chip.

sbourdeauducq commented 6 years ago

The ethernet connection is generally quite complex and could be a large time sink to debug and get working robustly

Only on Sayma. Again if my suggestion of copying the KC705 schematics had been followed, it would have worked a long time ago. And we have a very nice TCP/IP stack now...

sbourdeauducq commented 6 years ago

TM4C is real single chip solution but so far with no Arduino support, which can be drawback for some communities :)

But with Rust support, which is an advantage for others. You can for example look at how many external high-quality contributions smoltcp has... https://github.com/m-labs/smoltcp/pulls

gkasprow commented 6 years ago

@sbourdeauducq if we connect the Marvell PHY chip used on KC705 (NDA required) with TX clock routed to the general purpose IO, the problem would be the same. I didn't have much troubles with getting this PHY chip to work in gigabit mode. I can response in another way - if my suggestion to check the FPGA signal assignment was followed, we wouldn't have problems with RGMII :)

sbourdeauducq commented 6 years ago

We would have had some degree of problems with RGMII in any case: there was no existing MiSoC/ARTIQ gateware for it (we only had MII and GMII). You explicitly told me to stay away from MMC-related stuff like this Ethernet chip and charged ahead.

gkasprow commented 6 years ago

Yes, but RMII signal assignment is not related with MMC :)

hartytp commented 6 years ago

In general, I'm in favour of the idea of an ethernet EEM carrier; I have a lot of non-realtime use cases where I need to do something like hook up an ADC to do some data logging/setup a DDS to run CW/etc. The hardware ecosystem we have would be great for this with a suitable ethernet carrier.

However, my current plan is to hack up one of our existing BeagleBone projects to do this. Doing a "proper" job of it would be great in the long run, but is way down on my priorities list and not something I'm going to have time for any time soon.

As always, the hardware one would need for the carrier is pretty simple, but getting the firmware to work properly could be a lot of work/time/money. What kind of software interface would we have? Ideally, I'd like to have something which provided software-configurable (probably bit-banged) phys (SPI/TTL/etc) for each EEM, and then allowed SPI reads and writes/TTL toggling/etc via a simple text-based interface. But, that all seems like more work than we have time for at the moment.

tl;dr nice idea, but should probably be shelved for the foreseeable future...

gkasprow commented 6 years ago

So maybe instead of Beaglebone,we can use one of ZynQ SBCs. The cheapest ones are roughly 50$ and can be plugged to the main board. Then we have plenty of LVDS IOs for free and Linux on board. SPI interface can be generated without writing single line of code.

hartytp commented 6 years ago

@jbqubit Appreciate the enthusiasm for organisation, but not sure that copying email chains out of context is the best way forwards for this kind of thing.

dtcallcock commented 6 years ago

Obviously this is a fairly low priority for groups like us and Oxford that have already deployed Artiq. Instead of shelving it though, perhaps someone else could take the lead?

The reason is that this board would allow groups that are a way off using Artiq to get going with EEMs right away (assuming the interface is easy to make work with LabView/pyIoncontrol/M-ACTION etc.). This could significantly widen the base of Sinara participation/funding/purchasing.

That might be optimistic but I thought it worth mentioning.

hartytp commented 6 years ago

@dtcallcock If anyone wants to lead/fund this then I have no objections. I just don't have bandwidth to think about it right now.

Two brief comments though:

  1. If people really want to do this soon, they could just use Kasli running a microprocessor (misoc/microblaze). That doesn't require any new hardware design. And, as always, it's best to reuse designs where possible (easier to ensure high quality with fewer designs, better bulk-buying discounts etc).
  2. To make this project worthwhile, the new board would have to be significantly cheaper than Kasli (otherwise, one could just use Kasli + misoc or whatever). Before doing too much work on this, it would be interesting to know how much Kasli will cost when we produce it in quantity, and how much cheaper the microprocessor board would be.
hartytp commented 6 years ago

@dtcallcock If anyone wants to lead/fund this then I have no objections. I just don't have bandwidth to think about it right now.

Two brief comments though:

  1. If people really want to do this soon, they could just use Kasli running a microprocessor (misoc/microblaze). That doesn't require any new hardware design. And, as always, it's best to reuse designs where possible (easier to ensure high quality with fewer designs, better bulk-buying discounts etc).
  2. To make this project worthwhile, the new board would have to be significantly cheaper than Kasli (otherwise, one could just use Kasli + misoc or whatever). Before doing too much work on this, it would be interesting to know how much Kasli will cost when we produce it in quantity, and how much cheaper the microprocessor board would be.
gkasprow commented 6 years ago

We can get rid of 2 SFPs, clock distribution, USB, install smallest FPGA and use Kasli as it is. This would significantly cut the cost without changing the design.

hartytp commented 6 years ago

Could be a good idea, although I wonder how much money that will save? 50%? 25%?

Also, if we go for the cheapest FPGA in that package, we need to check that there are still transceivers available for SFP0 ethernet.

marmeladapk commented 6 years ago

@hartytp Even xca15t has 4 transcievers and has the same footprint as the one we use.

hartytp commented 6 years ago

Cool. It also has less GPIO, right? So, out of curiosity, what else do we loose? e.g how many complete EEM connectors do we have left? Do we loose the IO lines for the Si chip, etc.

But, anyway, if Kasli-Light or whatever can be cheap enough then it could be a good alternative to a microporcessor board IMHO.

marmeladapk commented 6 years ago

We loose one bank, which has eem11 and eem12 LVDS lines. Everything else stays. (I designed Kasli to be compatible with lower versions of Artix 7).

hartytp commented 6 years ago

Good for you!

Well, in that case, I don't see any issues. Maybe Kasli-lite will make a good microprocessor carrier then...

jbqubit commented 6 years ago

Closing this Issue. If this becomes a hot topic again reopen.