search

HolyLab / Imagine

A graphical interface for recording with OCPI microscopes.
1 stars 1 forks source link

readme

Imagine

A graphical interface for recording with OCPI microscopes.

Companion paper

A paper explaining most of the implementation details and related hardware can be found here.

@article{Greer2019,
  title={Fast objective coupled planar illumination microscopy},
  author = {Greer, Cody and Holy, Timothy E.},
  journal={Nature Communications},
  volume={10},
  pages={4483},
  year={2019},
  URL = {https://doi.org/10.1038/s41467-019-12340-0},
  eprint = {https://www.nature.com/articles/s41467-019-12340-0.pdf},
  doi = {10.1038/s41467-019-12340-0}
}

Installation

We need to install below items

Installation steps

  1. Clone this Imagine repository

  2. Install Visual Studio 2015

    • make sure you installed "common tools" when you install

  3. Install QT5

    • make sure the version of QT is for 64bit

  4. Install QT add-in for vc2015

    • install add-in
    • go to QT VS Tools tab in visual studio and set the right directory ex) for vc2015 and Qt5.7 : set this as C:\Qt\5.7\msvc2015_64

  5. Install QWT v6.1.3

    • open the Visual Studio tools directory under the windows Start menu, select "Developer's command prompt"(run as administrator) 
      > cd VC
> vcvarsall amd64
> cd [qwt directory (extract .zip to c:\ then it will create Qwt-6.1.3 directory)]
> C:\Qt_5_7\5.7\msvc2015_64\bin\qmake qwt.pro
> nmake (if you recompile everything again, C:\Qwt-6.1.3\src\release directory should be clear)
> nmake install
    • add an environment variable QWTDIR as C:\qwt-6.1.3

  6. Install Boost v1.57.0

    • extract .zip file to c:\Program Files\boost
    • open the Visual Studio tools directory under the Start
      menu, select "Developer's command prompt"(run as administrator) 
      > cd c:\Program Files\boost\boost_1_57_0
> bootstrap.bat
> .\b2 address-model=64 toolset=msvc-12.0

  7. Install camera SDK

    • run SW_PCOSDKWIN_123
    • Choose interface type ex) Silicon Software Dll MeIV or Camera Link HS

    • Setup target project of visual studio : open project and bring up property page Configuration Properties C/C++>General>Additional>Include Directories

      C:\Users\user\AppData\Roaming\Digital Camera Toolbox\pco.sdk\include

      Linker>General>Additional>Library Directories

      C:\Users\user\AppData\Roaming\Digital Camera Toolbox\pco.sdk\lib64

C:\Users\user\AppData\Roaming\Digital Camera Toolbox\pco.sdk\bin64

      Linker>Input>Additional>Dependencies

      SC2_cam.lib

  8. Install NI(National Instrument) DAQ driver

    • Setup target project of visual studio : open project and bring up property page Configuration Properties C/C++>General>Additional>Include Directories

      C:\Program Files (x86)\National Instruments\NI-DAQ\DAQmx ANSI C Dev\include

      Linker>General>Additional>Library Directories

      C:\Program Files (x86)\National Instruments\Shared\ExternalCompilerSupport\C\lib64\msvc

      Linker>Input>Additional>Dependencies

      NIDAQmx.lib

  9. Modify some items in the preference of Imagine project, if the directory of camera sdk installation is changed from the OCPI's setting.

    • include dir : .....pco.edge\include
    • additional library dir : ...pco.edge\lib64 ...pco.edge\bin64(need this?)
    • additional dependency : ...\lib64\sc2_cam.lib

  10. Install FFTW for frequency analysis of piezo control waveform

    • Download FFTW 3.3.5 Windows : ftp://ftp.fftw.org/pub/fftw/fftw-3.3.5-dll64.zip
    • unzip and copy to c:\FFTW
    • Open the Visual Studio Developer Command prompt Start -> All Apps -> Visual Studio 2015 -> Developer Command prompt Or, run C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\Tools\VsDevCmd.bat
    • Produce .lib file : Navigate to the unzip location and type lib /machine:x64 /def:libfftw3-3.def lib /machine:x64 /def:libfftw3f-3.def lib /machine:x64 /def:libfftw3l-3.def

    • Setup visual studio : open project and bring up property page Configuration Properties C/C++>General>Additional>Include Directories

      C:\FFTW

      Linker>General>Additional>Library Directories

      C:\FFTW

      Linker>Input>Additional>Dependencies

      libfftw3-3.lib, libfftw3f-3.lib, libfftw3l-3.lib

  11. Now, we can build Imagine project. But, to execute the Imagine, we need to copy all files below under the 'Output Directory' and 'Working Directory' of Imagine project.

    • Camera related dlls (located in C:\Program Files (x86)\Digital Camera Toolbox\pco.sdk\bin) : SC2_Cam.dll and camera interface dll (sc2_clhs.dll or sc2_cl_me4.dll)
    • QT related dlls (located under the installed directory (ex) C:\Qt\Qt5.7.0\5.7\msvc2015_64\bin) : Qt5Core.dll, Qt5Gui.dll, Qt5OpenGL.dll, Qt5PrintSupport.dll, Qt5Script.dll, Qt5SerialPort.dll, Qt5Svg.dll, Qt5Widgets.dll
    • QWT related dll (located in C:\Qwt-6.1.3\lib): qwt.dll
    • FFTW related dll (located in C:\FFTW): libfftw3-3.dll, libfftw3f-3.dll, libfftw3l-3.dll
    • JavaScript files (located in (Imagine directory)\Imagine\scripts) : imagine.js, 'related rig name'.js ((ex) ocpi-2.js)

Usage

OCPI Operation Modes

  1. Parameter and waveform control modes

alt text

  1. Script control mode In the script control mode, we can execute javascript code in Imagine. In addition, Imagine provides several javascript functions to provide access to some essential operations which are used to record data. Thereby, we can execute several experiments according to the control flow specified with javascript code by one execution without intervention.

In the Imagine GUI, general settings, parameter control settings and .json file name for waveform control can be saved in a configuration file with a ‘save configuration’ command in file menu. In script control mode, we can load, verify and record with this configuration file.

Typical work flow is as follows.

a. Prepare specimen

b. Capture full live image with a "live" button

c. Set parameters of display : intensity scale, zoom region, etc

d. Set parameters for an experiment : camera, positioner, laser, stimuli, output filename, etc

e. Press "Apply all" button : This verifies if the setting parameters are valid. If not, Imagine rejects the settings

f. Start acquisition : If the parameters are valid, we can start acquisition by clicking a "record" button.

g. Inspect the acquired data : Once the acquisition is end, we can verify the acquisition if it is ended normally or interrupted by some errors, Using "display" tab and "AI and DI" tab, we can verify that and also briefly check if we got the desired data.

Graphic User Interface

alt text

Imagine GUI is composed with several sections as listed below with numbers corresponding the numbers in the above image.

  1. Parameters and display control window

This window is designed for setting overall parameters for the microscope operation and display. This window consists of three main sections which are "Config and control", "Histogram" and "Intensity". Currently, the "Intensity" tab is not used. "Histogram" section displays histogram of an image displayed in "Display window". "Config and control" tab is designed for setting overall parameters such as camera control, positioner control, laser control and stimuli control which are used to configure an experiment. Also, it provides some basic tools for later inspection of acquired data.

  1. Display window

This window displays live images from the cameras attached to the microscope or captured images during the acquisition or after the acquisition by loading the '.imagine' file at the "Display" tab. In this window, we can zoom the image. This zoomed region can be set as a soft ROI region at the "camera" tab.

  1. Log window

This window displays log messages.

  1. Stimuli window

This window makes a list of stimulus numbers according to the stack numbers when we load stimuli file. Details are explained in the explanation of "Stimuli" tab.

  1. Main menu

Details are explained in the explanation of main menu page.

Main Menu

  1. File menu

alt text

We can save a current configuration and load it later using this menu.

  1. Acquisition menu

alt text

We can start acquisition or live displaying the images from the cameras.

  1. Hardware menu

alt text

We can open and close a laser shutter of laser. "Heatsink Fan" and "Temperature" menus are not usable currently.

  1. View menu

alt text

Parameters and display control window

  1. General tab

alt text

This tab is designed for setting the location and filename of output file and adding some comments about the parameter settings of this experiment. These settings and comments can be saved in a configuration file which will be created by clicking the "Save Configuration" item in the "File" menu.

  1. Camera tab

alt text

Some parameters such as exposure time and idle time between stacks are tightly related with some parameters in positioner tab. And, some parameters related with ROI are restricted according to ROI mode such as soft ROI (details about ROI modes are explained in the PCO camera manual). Imagine internally blocks undesired number of these parameters.

Positioner control waveform and camera control pulse for bi-directional imaging
alt text
  1. Stimuli tab

alt text

this figure, 'HR_test.stim' file is loaded, which shows an example description of stimuli. Here, several pairs of numbers are listed. The left number is a stack number and the right number is a stimulus number. If "Apply stimuli" is enabled, the stimulus number will be encoded with a 4-bit binary number and then each bits will be used to generate TTL pulses to channel P0.0 ~ P0.3.

  1. Positioner tab

alt text

This tab is for setting the start position and stop position of positioner and also positioner moving back time.

Positioner control waveform and camera control pulse
alt text
  1. Display tab

alt text

  1. Laser tab

alt text

The laser module of OCPI-2 has five different wavelengths. We can turn on and off an individual laser by checking the checkbox beside which the wavelength is labeled. The power of the laser can be also adjusted with slide bars and spinboxes. During the laser setting, we need to open the main laser shutter by clicking the "Open Shutter" button in the main menu. If we use waveform control mode, we can control the each shutters of the five individual wavelengths differently.

  1. Waveform tab

alt text

  1. AI and DI tab

alt text

  1. Script tab

alt text

In this figure, 'scriptControlTest.js' file is loaded, which shows example script descriptions. This script control mode will be explained in more detail in the next section.

Script control mode

In the script control mode, we can execute javascript code in the Imagine. In addition, Imagine provides several javascript functions for user to access some essential operations which are used to record data. Thereby, we can execute several experiments according to the control flow specified with javascript code by one execution without intervention.

In the Imagine GUI, general settings, parameter control settings and .json file name for waveform control can be saved in an configuration file with a ‘save configuration’ command in file menu. In script control mode, we can load, verify and record with this configuration file.

  1. System functions provided by the Imagine

prints string to the log terminal.

ex) print("Hello world!");

applies loaded configuration to data acquisition task and checks the validity of config files. This works as if we press a ‘Apply all’ button in the Imagine GUI after loading configuration files if these files are specified.

ex) var isOK = validityCheck("OCPI_cfg1.txt", "OCPI_cfg2.txt"); var isOK = validityCheck();

This works same as validityCheck() function.

ex) applyConfiguration();

begins to record. This works as if we press a ‘Record’ button in the Imagine GUI after loading configuration files if these are specifies. If configuration files are not specified, this just records according to the current configuration. This function return ‘false’ if execution time passes ‘timeout’ time, otherwise return ‘true.

ex) var isSucceed = record("OCPI_cfg1.txt", "OCPI_cfg2.txt", timeout); var isSucceed = record(timeout);

loads configuration files.

ex) loadConfig("OCPI_cfg1.txt", "OCPI_cfg2.txt"); loadConfig("OCPI_cfg1.txt“);

loads a waveform file.

ex) loadWaveform("OCPI_waveform.json");

makes the script execution sleep specified times in millisecond.

ex) sleepms(1000); // sleep 1000msec

changes output file names with specified file names.

ex) setOutputFilename("t1.imagine","t2.imagine"); setOutputFilename("t1.imagine");

returns estimated recording time in second. This time is calculated from total sample number of waveform signal. Therefore, it would take more time in actual acquisition which requires additional preparing and finishing times. These overhead times are less than 30sec in total.

ex) var time = getEstimatedRunTime();

stop recording. This function is useful to get a control again when the Imagine lose a control after record() is fail.

ex) stopRecord();

return various elapsed times according to 'dt' value.

0: record start to DAQ pulses output start

1: DAQ pulses output start to DAQ pulses output end

2: DAQ pulses output end to recording end

3: current time

ex) total_elapsed_time = getTimeElapsed(0) + getTimeElapsed(1) + getTimeElapsed(2);

ex) function current_time() {return getTimeElapsed(3);}

  1. Javascript objects

We can also use existing javascript objects.

ex) print(Math.sqrt(64)+Math.abs(-1)+Math.log(10));

  1. Example
var ret1, ret2, ret3, ret11, ret22, ret33;

// output filename
var paramCam1ConfigFile = "d:/test/OCPI_cfg1.txt";
var paramCam2ConfigFile = "d:/test/OCPI_cfg1.txt"
var waveCam1ConfigFile = "d:/test/OCPI_cfg1_wav.txt"
var waveCam2ConfigFile = "d:/test/OCPI_cfg1_wav.txt"
var test1Cam1File = "d:/test/test1_cam1.imagine";
var test1Cam2File = "f:/test/test1_cam2.imagine";
var test2Cam1File = "d:/test/test2_cam1.imagine";
var test2Cam2File = "f:/test/test2_cam2.imagine";
var test3Cam1File = "d:/test/test3_cam1.imagine";
var test3Cam2File = "f:/test/test3_cam2.imagine";
var test3WaveformFile = "d:/test/t_script_wav2.json";

// setup recording timeout time
var timeout = 30; // 30sec
var timeoutMargin = 20;

/* Check all the configurations before running them */
print("Checking the validity of all three configurations");
// loard OCPI_cfg1.txt (this is parameter control config file)
loadConfig(paramCam1ConfigFile, paramCam2ConfigFile);
setOutputFilename(test1Cam1File, test1Cam2File);
// check the 1st configuration
ret1 = validityCheck();

// OCPI_cfg1_wav.txt (waveform control : t_script_wav.json is specified in)
loadConfig(waveCam1ConfigFile, waveCam2ConfigFile);
setOutputFilename(test2Cam1File, test2Cam2File);
// check the 2nd configuration
ret2 = validityCheck();

// replace t_script_wav.json with t_script_wav2.json
ret3 = loadWaveform(test3WaveformFile);
setOutputFilename(test3Cam1File, test3Cam2File);
// check the 3rd configuration
ret3 = validityCheck();

/* Execute all the configurations */
if(ret1 && ret2 && ret3) { // If all the configurations are valid

    var estimatedRunTime;
    // setup the 1st configuration
    print("Recoding the 1st configuration");
    loadConfig(paramCam1ConfigFile, paramCam2ConfigFile);
    setOutputFilename(test1Cam1File, test1Cam2File);
    applyConfiguration(); // this can be also used to apply this configuration to the system
    estimatedRunTime = getEstimatedRunTime();
    // Recording
    timeout = estimatedRunTime + timeoutMargin;
    ret11 =  record(timeout);
    if (!ret11) stopRecord();

    // sleep 3000 msec
    print("Waiting 3 seconds");
    sleepms(3000);

    // 2nd configuration
    print("Recoding the 2nd configuration");
    loadConfig(waveCam1ConfigFile, waveCam2ConfigFile);
    setOutputFilename(test2Cam1File, test2Cam2File);
    applyConfiguration();
    estimatedRunTime = getEstimatedRunTime();
    timeout = estimatedRunTime + timeoutMargin;
    ret22 = record(timeout);
    if (!ret22) stopRecord();

    // sleep 100000 msec
    print("Waiting 10 seconds");
    sleepms(10000);

    // 3rd configuration
    // If we are here, we already loaded OCPI_cfg1_wav.txt.
    // So, we just replace waveform file
    print("Recoding the 3rd configuration");
    ret3 = loadWaveform(test3WaveformFile) ;
    setOutputFilename(test3Cam1File, test3Cam2File);
    applyConfiguration();
    estimatedRunTime = getEstimatedRunTime();
    timeout = estimatedRunTime + timeoutMargin;
    ret33 = record(timeout);
    if (!ret33) stopRecord();
}

// Display execution results
print("All the recodings are finished");
var msg1 = "configuration 1 validity: "+ ret1 +", recording: " + ret11;
print(msg1);
var msg2 = "configuration 2 validity: "+ ret2 +", recording: " + ret22;
print(msg2);
var msg3 = "configuration 3 validity: "+ ret3 +", recording: " + ret33;
print(msg3);