Micromanager Script for Tile Scanning with 3 Point Z Correction

[Credits:  Manoj V Mathew
Experiments were conducted at the Central Imaging and Flow Cytometry Facility (CIFF), National Center for Biological Sciences (NCBS), Bangalore, India.]

A couple of months ago a start-up-company approached me seeking help in building a low-cost slide digitizer for pathology applications. One of the issues they faced is the amount of time it took them to scan a single slide, which runs into more than a 1000 tiles at 20X mag. What adds most to this time is the Z drift compensation (software based) which needs to be performed on every tile or every few tiles.

This got me interested in writing a micromanager script for tile scanning with a built in three point Z correction option. In most applications where low magnification is used, like in the case of pathology scanners, it is not the Z drift that is the main culprit. The slide is tilted by a small amount when it is mounted. When tile scanning over the whole slide or a very large ROI using an automated XY stage, this tilt results in the image going out of focus. This tilt can be measured and corrected by providing a variable offset to the Z drive for each tile. This gives very good results even without the need for a time-consuming Z drift compensation algorithm. A hardware based (IR light used to locate the coverslip) would be a better solution for this problem, but then it is expensive.

The script can be downloaded from my GitHub Page.

The script is written to be independent of the hardware you use.The script will work if the micromanager configuration file has the camera, Z drive, and the XY motorized stage loaded.  In our setup I used a Nikon Tie, with an Hamamatsu Flash 4 V2 Camera and Prior Proscan III XY sample stage. The screen shot of the micromanager Hardware Configuration used is shown Figure B1451/1.

Figure B1451/1: Micromanager Hardware Configuration

For the script to work, you should make sure that the XY stage and the Camera are oriented right. If you move the joystick left, the stage should move left, if you move the joy stick up, the stage should move up and so on. If this is happening then the stage is configured right. Now if you check the live image and you see the same thing with respect to the stage movement, then the camera is oriented right as well. If not you might have to rotate your camera, 90 or 180 degrees. Also, make sure that a single axis motion of the stage results in a single axis motion in the camera image. Else the camera is slightly tilted. Make sure to set the microscope port to the one that has the camera connected and that the illumination is set to the required intensity.

Before you run the script enter the following system parameters in the script :

  1. Objective magnification (multiplied by intermediate magnification if used)
  2. Number of columns of pixels in the camera chip
  3. Number of rows of pixels in the camera chip
  4. Pixel size of the camera in micrometers
  5. Percentage overlap required between tiles for seamless stitching (10% usually works)

When you run the script the following happens:

  1. A GUI pops up requesting you to select the folder where you want to store the images. Select a folder to store the tiles. 
  2. The live window turns ON 
  3. A dialog pops up requesting you to move the stage to the bottom right (of the ROI on the slide you want to image) and adjust the focus manually to the optimal focus.
  4. Once you move the stage to the appropriate location and click OK another dialog pops up requesting you to move the stage to the top right (of the ROI on the slide you want to image) and adjust the focus manually to the optimal focus.
  5. Again you move the stage to the appropriate location and click OK another dialog pops up requesting you to move the stage to the top left (of the ROI on the slide you want to image) and adjust the focus manually to the optimal focus.
  6. This marks the three corners of the rectangle (ROI) you intend to image and measures the optimal Z focus at each corner.
  7. From the system parameters and the data obtained from the three corners, the script calculates the following: a. the size of each tile, b. No: of columns of the tile matrix, c. No: of rows of the tile matrix, d. the fixed X and Y stage offset for each tile and e. the linearly variable Z offset for each tile.
  8. The stage and Z drive is offset for each tile and an image is acquired and saved in the specified folder. The images are consecutively named using integers in '.tif' format. The first tile will be named 1.tiff and the last one (No: columns of tile matrix X No: of rows of the tile matrix).tif. 

The images can be stitched using the stitching plugin in FIJI.

For stitching in FIJI select: Plugins->Stitching->Grid/Collection Stitching

In Grid/Collection Stitching use the parameters as shown in Figure B1451/2.

Select Type=Grid:row-by-row. Select Order=Right&Down.

Figure B1451/2: Grid Collection  Parameters

Click OK and configure the stitching parameters as shown in Figure B1451/3.

Figure B1451/4: Image stitiching parameters

The micromanager script computes the column and row sizes of the tile matrix. This is printed while the script executes.

We had some very old blood smears. I imaged few of them using the micromanager script and stitched the tiles using FIJI. A screen shot of one of the stitched images is shown in Figure B1451/3. The figure shows the screen shot of an image of a blood smear slide imaged at 30X (20x Objective and 1.5X intermediate magnification). The image is a stitch of 779 (41x19) tiles, spanning an area of approximately 2cmX1cm of the slide.

Figure B1451/4: Screen shot of a stitched image of 779 (41x19) tiles, spanning an area of approximately 2cmX1cm of the slide

Another such image is in Figure B1451/4. This figure shows the screen shot of an image of a blood smear slide imaged at 15X (10x Objective and 1.5X intermediate magnification). The image is a stitch of 780 (52x15) tiles, spanning an area of approximately 4.5cmX1.5cm of the slide.

Figure B1451/5: Screen shot of a stitched image of 780 (52x15) tiles, spanning an area of approximately 4.5cmX1.5cm of the slide

It can be seen from both the above images that the 3 point Z correction algorithm has maintained the Z focus. It was observed that the Z focus was maintained for the entire area of a standard slide. However, the ability to stitch very large tile matrices was limited using the FIJI plugin. For stitched image sizes greater than 4GB the stitching plugin gives an error. So tiling more than around 800 tiles was difficult. This can be improved by reducing the bit depth of the camera while acquisition or downsampling post acquisition. 

Generating Trigger Signals Using Arduino

[Credits: Manoj V Mathew. Experiments were conducted at the Central Imaging and Flow Cytometry Facility (CIFF), National Center for Biological Sciences (NCBS), Bangalore, India.]
For conducting the external trigger experiments on the sCMOS camera, we used the Tektronix AFG 3252 which costs about USD 12,000. I realized that the maximum frequency I required to trigger the sCMOS camera was not more than 25KHz. I could do this with a much cheaper trigger source. That is when I stumbled upon the Arduino.
So what is Arduino? 
Let me quote from the Arduino Web Page. "Arduino is an open-source electronics platform based on easy-to-use hardware and software. Arduino boards are able to read inputs - light on a sensor, a finger on a button, or a Twitter message - and turn it into an output - activating a motor, turning on an LED, publishing something online. You can tell your board what to do by sending a set of instructions to the microcontroller on the board. To do so you use the Arduino programming language (based on Wiring), and the Arduino Software (IDE), based on Processing."
The excellent thing about the Arduino is that for what it offers it is really inexpensive. We got an Arduino UNO R3 which cost us only about USD 20 !
Arduino hardware is open source. Complete documentation, electronic connection diagrams, PCB layouts etc. are available.
The board offers 14 Digital Input Output (DIO) pins. Of the 14, 6 can be used to generate Pulse Width Modulation (PWM) signals to mimic an analog output. It has 6 analog input pins with individual 10bit Analog to Digital Converters (ADC). It is a tiny powerful package. Arduino can be programmed by connecting it to a computer through a USB interface. Quoting Arduino Webpage . "The open-source Arduino Software (IDE) makes it easy to write code and upload it to the board. It runs on Windows, Mac OS X, and Linux. The environment is written in Java and based on Processing and other open-source software. A sketch is a name that Arduino uses for a program. It's the unit of code that is uploaded to and run on an Arduino board."
Once a program is loaded (burned) into the Arduino ROM, you do not need not keep it connected to the computer. It can run on a 5v power supply. When you turn the power ON the code executes. You can restart the code by pushing the reset button on the Arduino Board.
I used Arduino UNO R3 to create a clock to trigger the Hamamatsu Flash 4 sCMOS camera. The Arduino setup is shown in Figure B10.

Figure B10: Arduino Uno setup for generating trigger signals

Two single strand wires with pin connectors on both ends were used to connect one of their ends to the Arduino board and the other end to a coaxial cable.  The coaxial cable had crocodile clips on one end and SMA on the other. The SMA connector was connected to the Ex. Trig input of the sCMOS camera.
The program used to generate the trigger signals is shown in Figure B11.

Figure B11: Arduino sketch used to generate PWM signals. (Original Sketch)

The code works by configuring a Digital Input Output (DIO) pin to operate in the output mode. The output DIO pin is then switched between logic high and logic low for set time periods defined by Off-Time and On-Time of a PWM wave. The On time and Off time are calculated in the code from the required frequency and duty cycle. The example program in Figure B11 generates a PWM signal with frequency 1KHz and duty cycle 20%.
A screen shot of a trigger signal generated is shown in Figure B12 below. The code was programmed to generate 500Hz. The oscilloscope was set to 50 Ohms impedance and infinite persistane to observe jitter.

Figure B12: Oscilloscope screen shot of trigger signal at 500Hz

There are few things we can observe here:
  1. The peak to peak amplitude produced by the digital pin is about 3.3 volts
  2. At small frequencies like 500Hz used here the time jitter is almost negligible.
  3. There is an amplitude jitter of about 350V
  4. The generated frequency is slightly lower than 500 Hz (488.9 Hz). This because this code computer the ON and OFF delays solely based on output frequency and does not consider the internal delays. 
A slightly modified version of the sketch that includes a correction factor for the internal delay can be downloaded from my GitHub Page.
This code works well for frequencies from 0 Hz to about 8Khz. For frequencies greater than 8 kHz the time jitter is too large as can be seen in Figure B13. It shows an Oscilloscope screen shot of the Arduino output when the code is made to generate 10 kHz. This sketch worked really well for trigerring the Hamamatsu Flash 4 V2 camera during our several tests with this camera.  

Figure B13: Oscilloscope screenshot shows the large time jitter for frequencies greater than 8KHz

A better way to generate high frequency is to divide the onboard clock using the built in timer of Arduino Uno. The ATmega processor on Arduino Uno uses a 16MHz clock. 
I have modified a sketch available on the Arduino Forum to generate high frequencies. 
The sketch can be downloaded from my GitHub page. The minimum frequency we can generate by this method is 31.5 kHz and a maximum frequency of 8 MHz. You can run the sketch by editing the Frequency variable in the sketch. The signal is output on pin 11. Figure B14 shows an oscilloscope screenshot of 1 MHz signal generated using the sketch. The oscilloscope is operated at infinite persistence. 

Figure B14: Oscilloscope screen shot of a 1MHz signal generated using Arduino Uno by dividing the onboard clock.

As you can see the timing jitter is negligible at this high frequency. 

Figure B15: Oscilloscope screenshot of 8MHz signal

Figure B15 shows an oscilloscope screenshot of 8 MHz signal generated using the sketch. The oscilloscope is operated at infinite persistence. 8 MHz is the maximum frequency that can be generated using this sketch. At 8MHz the output is no longer a square wave due to the on negligible capacitance effects in the circuits at high frequency. However, as can be seen from the figure the time jitter even in this case is negligible and hence this signal can be used for timing and triggering applications. 
There are few points you need to note when you use this sketch to generate high frequencies:
  1. The sketch can generate a minimum frequency of 31.5 kHz and maximum frequency of 8MHz.
  2. The frequency can only be varied in discrete steps by dividing the onboard clock by integer steps ranging from 2 to 510
  3. The signal is output on pin 11 of the DIO pins since we used Timer 2.
   

Getting the best out of sCMOS cameras-II

[Credits: Amit Cherian and Manoj V Mathew
Experiments were conducted at the Central Imaging and Flow Cytometry Facility (CIFF), National Center for Biological Sciences (NCBS), Bangalore, India.]

We conducted a series of experiments by triggering the Hamamatsu Orca Flash 4 C11440 camera using an external trigger source. 

Exp:1 Speed Test using Camera Link Interface and Internal Trigger

We tested the maximum speed of operations in FPS for different frame sizes while using the external trigger. The trigger signal was generated using a Tektronix AFG 3252 Arbitrary Function Generator. The following parameters were used:

  1. Waveform Used: Square Wave
  2. Duty Cycle: 20%
  3. Data Interface: Camera Link
  4. Camera Speed: Standard Scan
  5. Capture Mode: External Edge Trigger
  6. External Trigger Input Option: Pos (Trigger on Positive Edge of the Pulse)

The connection diagram is shown in Figure B8.

Figure B8: Connection diagram for external triggering of Hamamatsu Flash 4 using Tektronix AFG.

The signal output of the function generator was connected to the Ext.Trig SMA connector at the back of the camera using a BNC to SMA cable.

Like in the case of experiments with the internal trigger, we used the Hamatusu HCI software as well as the Micromanager software by operating the camera in the live mode. Frame rates were read out from the indications on the live window. Both HCI and micromanager software data were consistent.

The results of the experiment are summarized in Table T5.

Table T5: Maximum speed of operation in FPS for various frame sizes using the camera link interface and external trigger (PDF Format).

The log-log plot of speed of operation in FPS as a function of Frame Size for external triggering is shown in Figure B9.

Figure B9: Log-Log plot of frame size vs speed of acquisition in fps for Camera Link Interface and external trigger

When comparing the internal and external trigger modes we observe the following:

  1. For frame sizes 2048x256 or larger the speed of operation in external trigger mode is slightly lesser than the speed of operation in internal trigger mode.
  2. For frame Sizes smaller than 2048x256 while internal trigger mode enables the use of exposure times lower than 1.0037ms, for external triggering the minimum exposure time value is fixed at 1.0037ms for all frame sizes. So for frame sizes smaller than 2048x256 the speed is significantly lower than those obtained for internal triggering.

 

 

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Contributions:

  1. Manoj V Mathew: Conceptualized the ideas, designed the experimental setups, and wrote the article.  
  2. Amit Cherian (NCBS, Bangalore): Setup the equipment, conducted the experiments, tabulated and analyzed data and generated the figures.

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------