Opentrons Imaging Unit (XYZ)

WORK IN PROGRESS - more information will follow soon!

This device is supposed to help you automatically imaging a multiwell plate using a very simple GRBL-based Arduino Shield controlled by a Raspberry Pi, which also captures images. Curious how this looks like? Scroll to the very end of the page.

Of course this microscope can be used in a different setting as well.

Features:

Move camera in XY
Flexure-based focusing of a 10x objective lens
Fast
Costs ~300-400 €
All 3D printed + off-the-shelf components
Minimum of tools required
Use GRBL as the motion protocol
Use the openflexure microscope server as the control software

In-Action

You can directly control the microscope from the opentrons notebook. You can have a glimpse in the example notebook here

Table of Content

Quickstart
Electronics
Software
Hardware
Bill of materials
[First Use](#First Use)
Results

⛏ Quickstart

This guide assumes, you have assembled the microscope already.

Preparing the microscope

Plugin the powersupply in the Raspberry Pi (USB) and CNC Shield (12V)
Connect the USB cable with the CNC Shield and Raspberry Pi
Check all wires if there is a non-connected wire somewhere
Place the microscope objective lens inside the objective slot
Connect the raspberry Pi ribbon cable with the Raspberry Pi according to this explanation
Get the tools you need to level the bed (Hex keys for M5 and M3)
Place a multwell plate in the try and add some "gum" to avoid slipping of the well-plate

Starting the openflexure microscope server

connect the microscope to your computer using a LAN cable (or to your home network router)
Open up a terminal (MAC/Linux) or command line propmt (CMD, windows)
enter ssh UC2@UC2Pal015, where UC2 is the username and UC2Pal015 is the hostname
if everything works correctly, the commandline asks you for the password which is: youseetoo
after hitting enter, you are connected to the microscope using SSH
go to the folder, where the microscope software sits and start the server:
- cd /home/UC2/OFM/openflexure-microscope-server/openflexure_microscope/api
- python 3.7 app.py
If everything goes right, the server should start and outputs something like : server started on http://localhost:5000
You can access this using your browser by typing: http://UC2Pal015:5000 the website for the microscope should appear

Troubleshotting

The camera does not work: Check all wired connections and their correct orientation and reboot.

The Stage does not work: Check all connections (USB, 12V of CNC Shield) and the Settings which are accessible using: nano /home/UC2/OFM/openflexure-microscope-server/openflexure_microscope/openflexure/settings/microscope_configuration.json

and have a look at the port and type of the stage:

{
  "camera": {
    "type": "PiCamera"
  },
  "stage": {
    "port": "/dev/ttyACM0",
    "type": "grbl"
  }
}

If you have connected the stage properly and type dmesg | tail in the SSH terminal, you will receive something like:

UC2@UC2Pal015:~/OFM/openflexure-microscope-server/openflexure_microscope/openflexure/settings $ dmesg | tail
[87673.997032]  r5:60000013 r4:80109b18
[87673.997048] [<80109ae4>] (arch_cpu_idle) from [<80920660>] (default_idle_call+0x40/0x48)
[87673.997062] [<80920620>] (default_idle_call) from [<8015504c>] (do_idle+0x124/0x168)
[87673.997073] [<80154f28>] (do_idle) from [<80155368>] (cpu_startup_entry+0x28/0x30)
[87673.997084]  r10:00000000 r9:410fd034 r8:0000406a r7:80eaf350 r6:10c0387d r5:00000002
[87673.997090]  r4:0000008a r3:60000093
[87673.997102] [<80155340>] (cpu_startup_entry) from [<8010fd10>] (secondary_start_kernel+0x160/0x16c)
[87673.997116] [<8010fbb0>] (secondary_start_kernel) from [<001027cc>] (0x1027cc)
[87673.997124]  r5:00000055 r4:2f10806a
[87673.997136] ---[ end trace 715e6d19da8dbf53 ]---

where there will be something like ttyACM0 or ttyUSB1 representing the Arduino hosting the GRBL board. Use this for the settings above and add /dev/ to it.

First Use

Control
Page Up/down, scrol => Z stage
KEys x/y
homing

Extensions

Software

The software is based on already available open-source projects. We use GRBL for the hardware control using an Arduino and the OpenFlexure Microscope Server for the GUI and I/O control.

Hardware control software for the motors/light sources

We use the GLBR code to control the microscope - so you can use it from any computer equipped with a serial (e.g. USB) connection. In order to make it work you "simply" need to flash the GBRL code which is also available on Github on your Arduino Uno + CNC Shield V3. Therefore:

Connect it to the Computer using a USB cable
Open the arduino IDE
Copy the GRBL library in the Arduino Library folder
Open the grblUpload.ino Demo in the EXAMPLES folder and upload it

We slightly modified the standard GRBL firmware which to match the homing and PWM for laser/LEDs. You can find the GRBL Library here

GUI and general I/O using the Openflexure Microscope Server

The beautiful openflexure microscope server developed by Bowman et al. hosts a number of very helpful features:

Extending functionality using customized extensions
Web browser-based control of the microscope (from any device)
Native raspberry pi camera flatfielding
Python/Javascript/VUE.js based
User-friendly UI

Setting up the Raspberry Pi

The people from OpenFlexure provide a ready-to-use SD-card which we are going to use here as a start. This makes it easier to start with all dependencies already installed. Therefore go ahead and perform the following steps:

Download the OpenFlexure Lite image
Format a micro SD card (use rather large SD card, 64 GB) and "burn" the image on the SD card using e.g. Win32Disk on Windows or Etcher
Insert the SD card into the Raspberry PI and let it boot 3.1 For further details please have a look here
If you are connected to the Pi using a LAN connection, you should be able to access it on microscope.local (e.g. SSH) 4.1 The credentials are:
- username: pi
- password: openflexure

Connect Device to a WiFi Router

The easiest is to connect to a wifi hotspot created with your cellphone. Attention: It's not very fast for streaming.

sudo raspiconfig # -> setup wifi -> set country
# setup the SSID (e.g. Blynk)
# setup password e.g. 12345678)

Alternatively:

sudo nano /etc/wpa_supplicant/wpa_supplicant.conf

and add:

network={
    ssid="Blynk"
    psk="12345678"
    key_mgmt=WPA-PSK
}

Print the IP address:

ifconfig

Replace the OFM Server version with our modified version

All the original OFM components are sitting in /var/openflexure/, where we don't have access to. We want to change that to have it modifiable. Therefore we need to unregister the OFM service and add our own git repository.

Download and install the git:

cd ~
git clone -b opentrons-grbl https://gitlab.com/beniroquai/openflexure-microscope-server 
cd openflexure-microscope-server/openflexure_microscope/api
ATTENTION:
sudo rm -r static # we need to rebuild it this takes long, so we side-load it by building it on a computer or downloading it here:
wget static.zip # link will follow; gdrive: https://drive.google.com/file/d/1I15lDKNll91UxbFlYPiCGbfkYkHK_CM1/view?usp=sharing
unzip static.zip

Stop the OFM Server which is still running

ofm stop

Setup the settings

Make sure the stage is matching the GRBL one with the correct serial port by editing `nano ~/openflexure-microscope-server/openflexure_microscope/openflexure/settings

{
  "camera": {
    "type": "PiCamera"
  },
  "stage": {
    "port": "/dev/ttyACM0",
    "type": "grbl"
  }
}

If you have connected the stage properly and type dmesg | tail in the SSH terminal, you will receive something like:

UC2@UC2Pal015:~/OFM/openflexure-microscope-server/openflexure_microscope/openflexure/settings $ dmesg | tail
[87673.997032]  r5:60000013 r4:80109b18
[87673.997048] [<80109ae4>] (arch_cpu_idle) from [<80920660>] (default_idle_call+0x40/0x48)
[87673.997062] [<80920620>] (default_idle_call) from [<8015504c>] (do_idle+0x124/0x168)
[87673.997073] [<80154f28>] (do_idle) from [<80155368>] (cpu_startup_entry+0x28/0x30)
[87673.997084]  r10:00000000 r9:410fd034 r8:0000406a r7:80eaf350 r6:10c0387d r5:00000002
[87673.997090]  r4:0000008a r3:60000093
[87673.997102] [<80155340>] (cpu_startup_entry) from [<8010fd10>] (secondary_start_kernel+0x160/0x16c)
[87673.997116] [<8010fbb0>] (secondary_start_kernel) from [<001027cc>] (0x1027cc)
[87673.997124]  r5:00000055 r4:2f10806a
[87673.997136] ---[ end trace 715e6d19da8dbf53 ]---

where there will be something like ttyACM0 or ttyUSB1 representing the Arduino hosting the GRBL board. Use this for the settings above and add /dev/ to it.

Start the Server

We will need to use the preinstalled python environment sitting here: /var/openflexure/application/openflexure-microscope-server/.venv/bin/python, hence we spin up the server using this command:

cd ~/openflexure-microscope-server/openflexure_microscope/api/
/var/openflexure/application/openflexure-microscope-server/.venv/bin/python app.py

The server will output something like:

pi@microscope:~/openflexure-microscope-server/openflexure_microscope/api $ /var/openflexure/application/openflexure-microscope-server/.venv/bin/python app.py
WARNING:root:No config file found at /home/pi/openflexure/settings/microscope_settings.json. Creating...
INFO:root:Populating /home/pi/openflexure/settings/microscope_settings.json...
WARNING:root:No config file found at /home/pi/openflexure/settings/microscope_configuration.json. Creating...
INFO:root:Populating /home/pi/openflexure/settings/microscope_configuration.json...
INFO:root:Running with data path /home/pi/openflexure
INFO:root:Loading /home/pi/openflexure/settings/microscope_configuration.json...
INFO:root:Creating camera
INFO:root:setting zoom, centre [0.5 0.5], size 1.0
INFO:root:Setting stage
INFO:root:Trying SangaStage
INFO:root:Initialising ExtensibleSerialInstrument on port None
INFO:root:Updating ESI port settings
INFO:root:Opening ESI connection to port None
Auto-scanning ports
INFO:root:Trying port /dev/ttyAMA0
INFO:root:Creating serial.Serial instance...
INFO:root:Created Serial<id=0x73d7c330, open=True>(port='/dev/ttyAMA0', baudrate=115200, bytesize=8, parity='N', stopbits=1, timeout=None, xonxoff=False, rtscts=False, dsrdtr=False)
INFO:root:Testing communication to SangaBoard
INFO:root:Running firmware checks...
ERROR:root:We don't have a serial port to open, meaning you didn't specify a valid port.  Are you sure the instrument is connected?
WARNING:root:No compatible Sangaboard hardware found.
INFO:root:Handling fallbacks
INFO:root:Creating locks
INFO:root:Loading /home/pi/openflexure/settings/microscope_settings.json...
INFO:root:0 capture files found on disk
INFO:root:0 capture files successfully reloaded
INFO:root:Creating app
WARNING:root:No extension file found at /home/pi/openflexure/extensions/microscope_extensions. Creating...
INFO:root:Populating /home/pi/openflexure/extensions/microscope_extensions/defaults.py...
ERROR:root:Preferred capture path /home/pi/openflexure/data/micrographs is missing or cannot be written to. Restoring defaults.
INFO:root:0 capture files found on disk
INFO:root:0 capture files successfully reloaded
INFO:root:Starting OpenFlexure Microscope Server...
Registering zeroconf 4a2c735e300aa5cae894fc4625f157a7a9b34edc._labthing._tcp.local.
Starting LabThings WSGI Server
Debug mode: False
Running on http://localhost:5000 (Press CTRL+C to quit)

You will be able to see the server in the browser by navigating to http://microscope:5000

Closing the server

Hit ctrl + c and the shell will give you:

^CUnregistering zeroconf services...
Server stopped
INFO:root:Loading /home/pi/openflexure/settings/microscope_settings.json...
INFO:root:Saving /home/pi/openflexure/settings/microscope_settings.json...
INFO:root:Closing <openflexure_microscope.microscope.Microscope object at 0x73e1aa10>
INFO:root:Stopped MJPEG stream on port 1. Switching to (3280, 2464).
INFO:root:Closed <openflexure_microscope.camera.pi.PiCameraStreamer object at 0x73d6ad90>
INFO:root:Closing <openflexure_microscope.captures.capture_manager.CaptureManager object at 0x73f966f0>
INFO:root:Closed <openflexure_microscope.microscope.Microscope object at 0x73e1aa10>

Building the server

If you want to change anything to the server, please have a look at the original descritpion for the OFM server here

HARDWARE

Bill of material

Below you will find all components necessary to build this device

3D printing files

All these files need be printed. We used a Prusa i3 MK3 using PLA Prusmant (Galaxy Black) at layer height 0.3 mm and infill 80%.

Type	Details	Price	Link
Base for microscope	Holds everything together	4 €	Opentronscope_opentrons_gear2.stl
Optics Module	Holds all optics + camera	4 €	Opentronscope_01_opticsmodule.stl
Focusing mechanism		4 €	Opentronscope_01_Slide_camera_flexure_v1.stl
Y-slide for the camera module		4 €	Opentronscope_01_Slide_camera_v1.stl
Camera slide		4 €	Opentronscope_01_Slide_camera_y_v0.stl
Base for the microscope		4 €	Opentronscope_02_Opentrons_Microscope_Base_v0.stl
X-slide		4 €	Opentronscope_03_Support_Xslide.stl
Rail fastener		4 €	Opentronscope_04_Mount_Rail_Fastener.stl
Base for wellplate		4 €	Opentronscope_04_OPENTRONS_Microscope_wellplate_base.stl
Illumination arm		4 €	Opentronscope_06_Fluorescence_Arm.stl
BAseplate		4 €	Opentronscope_09_Opentrons_Microscope_Baseplate.stl
Module for fluorescence and reflection measurements		4 €	Opentronscope_10_OPENTRONS_LED_FLUO_MODULE.stl
Gear 1		4 €	Opentronscope_opentrons_gear1.stl
Gearbox for focusing		4 €	Opentronscope_opentrons_Gearbox.stl

Bill of material

This is used in the current version of the setup

Type	Details	Price	Link
3x Linear Bearing	MGN12H Linear Block	8 €	[Amazon]()
3x Linear Guide Rail	MGN12 (220cm, 130cm)	8 €	[Amazon]()
3x Stepper Motor	Nema 11	15 €	Ebay or Roboter Bausatz
2x Spindle drive	T8 Trapezgewindespindel Messingmutter 3D Drucker Fräse Spindel Mutter 300mm	8 €	Ebay
CNC Shield	CNC V3 Shield + UNO R3 + A4988 Driver Module Board für Arduino 3D Drucker	15 €	Roboter bausatz + Roboter Bausatz + Roboter Bausatz
M3, M4 Screws	Screw Set	12 €	Ebay
Ringmagnet	3x Sehr starke Neodym Ring Magnete Magnetring mit Bohrung Loch viele Größen Günstig, 25mm x 20mm x 3mm	12 €	Ebay
CCTV Linse	2X( fisheye cctv lens 5MP 1.8mm M12*0.5 mount 1/2.5 F2.0 180 degree for vi M1L	7 €	Ebay
Wires	Various	10 €	[ebay]()
Powersupply	5V, 3A, Various	10 €	Roboter Bausatz
Powersupply	12V, 3A, Various	10 €	Roboter Bausatz
Raspberry Pi + SD + Powersupply + Housing	Raspberry Pi 3 Set /Bundle: 16GB SD-Karte, HDMI, original Netzteil und Gehäuse	70 €	ebay
Camera	Raspberry Pi Camera v2.1	22 €	ebay	PLA filament	Prusament, space black, 1.75mm	22,50 €	ebay
Isolated Copper Wire	Kupferlackdraht W210 - Ø 0,20 bis 1,18mm - 100g / Spule Kupfer Trafo Motor CU, 0.1 mm	22,50 €	ebay

Assembly Tutorial

A dedicated assembly tutorial can be found here..

XYZ Stage in Action

Scanning microscope in XY inside the Opentrons OT2

Focusing to one position with the VCM

Focus-stacking and moving the microscope

Focus-stacking (full range)

Newest Version inside the Opentrons OT-2

Plate leveling

Since 3D printers are not really reliable in terms of accuracy and the stage is rather not really flat, I added springs to each corner of the base which holds the well plate. Similar to 3D printer plate leveling, you need to adjust the hight of the plate in the first run manually. This is necessary since the focus range so far is only ~1 mm I would say. I need to improve this ;-)

We are working on an extension which will replace this very soon!

ATTEMPT: Fluorescence Imaging

We try to integrate an LED ring (Adafruit Neopixel), which is controlled by the Raspberry Pi. We currently don't know yet if there is any fluorescent signal visible by the Picamera.

Steps to make the LED array work are the following.

Install the library:

sudo apt-get install build-essential python-dev git scons swig

git clone https://github.com/jgarff/rpi_ws281x.git
cd rpi_ws281x
scons

cd python
sudo python3 setup.py build
sudo python3 setup.py install
sudo pip3 install rpi_ws281x
sudo pip3 install adafruit-circuitpython-neopixel

Now we want to connect the LED ring to the Raspberry Pi. Therefore we need to supply it with 5V, GND and the data pin. It goes as follows:

LED Panel Pin	Raspberry Pi Pin
5V (in)	5V (#4)
GND (in)	GND (#6)
Data (in)	GPIO18 (#12)

Sample Code from this github repo:

You need to run the code as ROOT in order to access the GPIO pins!

#!/usr/bin/python

import time, sys
import datetime

from rpi_ws281x import *

LED_COUNT      = 16      # Number of LED pixels.
LED_CHANNEL    = 0       # PWM Channel (set to 1 when LED_PIN is 13 or 19, else 0)
LED_PIN        = 18      # GPIO pin connected to the pixels (must support PWM!).
LED_FREQ_HZ    = 800000  # LED signal frequency in hertz (usually 800khz)
LED_DMA        = 5       # DMA channel to use for generating signal (try 5)
LED_BRIGHTNESS = 100     # Set to 0 for darkest and 255 for brightest
LED_INVERT     = False   # True to invert the signal (when using NPN transistor level shift)

## main

strip = Adafruit_NeoPixel(LED_COUNT, LED_PIN, LED_FREQ_HZ, LED_DMA, LED_INVERT, LED_BRIGHTNESS, LED_CHANNEL)
strip.begin()

strip.setPixelColor(0,Color(0,0,255))
strip.show()

try:
    while True:
        time.sleep(0.1)

except KeyboardInterrupt:
    for i in range(strip.numPixels()):
        strip.setPixelColor(i, 0)
    strip.show()
    sys.exit()

ATTENTION: This is currently not working on the Raspi unfortunately. Need to investigate it a bit.

Which is nice, is that the RGB LEDs spectrum for the red, green and blue LED seem to match some Fluorescent spectra quiet well:

Red

Green

Blue

With the RING it looks like this here:

One can observe some fluorescent signal from the stained paper d

Get Involved

This project is open so that anyone can get involved. You don't even have to learn CAD designing or programming. Find ways you can contribute in CONTRIBUTING

License and Collaboration

This project is open-source and is released under the CERN open hardware license. Our aim is to make the kits commercially available. We encourage everyone who is using our Toolbox to share their results and ideas, so that the Toolbox keeps improving. It should serve as a easy-to-use and easy-to-access general purpose building block solution for the area of STEAM education. All the design files are generally for free, but we would like to hear from you how is it going.

You're free to fork the project and enhance it. If you have any suggestions to improve it or add any additional functions make a pull-request or file an issue.

Please find the type of licenses here

REMARK: All files have been designed using Autodesk Inventor 2019 (EDUCATION)

Showcase

First (poor) result

This is the first manually stitched crystalized sugar on a wellplate ;-)

Second (improved) result

HeLa cells in a 24-well plate:

Credits

If you find this project useful, please like this repository, follow us on Twitter and cite the webpage! :-)

beniroquai / Opentrons-Microscope-Platereader

readme

Opentrons Imaging Unit (XYZ)

Further reading

In-Action

Table of Content

⛏ Quickstart

Preparing the microscope

Starting the openflexure microscope server

Troubleshotting

First Use

Extensions

Software

Hardware control software for the motors/light sources

GUI and general I/O using the Openflexure Microscope Server

Setting up the Raspberry Pi

Connect Device to a WiFi Router

Replace the OFM Server version with our modified version

Start the Server

Closing the server

Building the server

HARDWARE

Bill of material

3D printing files

Bill of material

Assembly Tutorial

XYZ Stage in Action

Scanning microscope in XY inside the Opentrons OT2

Newest Version inside the Opentrons OT-2

Plate leveling

ATTEMPT: Fluorescence Imaging

Get Involved

License and Collaboration

Showcase

First (poor) result

Second (improved) result

Credits