search

leginon-org / leginon-redmine-archive

1 stars 0 forks source link

Get Leginon working on JEOL 1400 and 3200 #1575

Closed leginonbot closed 8 months ago

leginonbot commented 8 months ago

Author Name: Neil Voss (@vosslab) Original Redmine Issue: 1575, https://emg.nysbc.org/redmine/issues/1575 Original Date: 2012-03-05 Original Assignee: Neil Voss

I am working on getting Leginon working on JEOL 1400 (like a spirit) and JEOL 3200 with energy filter at Northwestern University for Vinzenz Unger. This is a page to keep track of my progress.

leginonbot commented 8 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2012-03-05T17:58:59Z

JEOL 1400

I've installed pyscope and set it up to use the JEOL 1230 script from the Stokes group.

We have two computers, (1) hooked directly to microscope and (2) hooked to gatan CCD camera running DigitalMicrograph. DigitalMicrograph uses some serial interface over ethernet (this is key) to interface microscope.

When running JEOL 1230 script it almost expects to interface over serial interface -- will check with JEOL engineers on communication interface.

leginonbot commented 8 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2012-03-05T18:00:17Z

Hi AMI,

Can I get a contact email for the JEOL 1230 script? Based on their webpage it seems that they may have a newer version than is available in subversion.

http://cryoem.nysbc.org/CemRobotics.html

leginonbot commented 8 months ago

Original Redmine Comment Author Name: Anchi Cheng (@anchi2c) Original Date: 2012-03-05T18:12:20Z

Try Nicolas Coudray in David Stokes's lab. ncoudray@nysbc.org

leginonbot commented 8 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2012-03-28T12:11:44Z

So, unlike the JEOL 1230 that uses the serial port for communication, the JEOL 1400 does its communication over ethernet.

I emailed the JEOL engineers:

"JEOL is not aware of any action that needs to be taken by JEOL.  The person
who maintains the software told me that it works out-of-the-box with the
JEOL scopes. This is Bridget Carragher at Scripps. (I think Neil may need to
know this, because the 1230 he's referring to is probably the one at NYSBC).
As far as communications goes, the 1230 doesn't run at all like the newer
models. On the 1230 DM one can either use serial I/O or TCP/IP. The command
set for the 1230 is totally different from what the 1400 uses. The 1400 uses
TEM3 interface."

So it appears it will be some significant work, so my next response was "Do you have documentation on the TEM3 interface?" and they responded:

"It is our understanding that everything required is already included in
the Leginon program"

So, I am stuck.

leginonbot commented 8 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2012-03-28T12:32:07Z

Has anyone at AMI/NRAMM heard of the "TEM3 interface"? I am assuming it some JEOL propietary interface.

leginonbot commented 8 months ago

Original Redmine Comment Author Name: Jim Pulokas (Jim Pulokas) Original Date: 2012-03-28T15:16:29Z

Neil, I don't know anything about the TEM3 interface, but I can provide some advice on reverse engineering without documentation.

I am curious how the overall setup is compared to FEI scopes. Is it like Tecnai, where the Windows PC connected to the scope is an essential component of the system, required for it to function at all? Or is the PC an optional add on like the old CM scopes?

Have you searched the connected PC for any scripting documentation in the form of pdf, doc, chm, etc?

It could be as easy as the Tecnai scripting interface which is implemented as a COM object. COM objects can be inspected to learn what functions they implement. If you have installed the pywin32 package, you can browse through all the registered COM objects on the system using the script: C:\Python25\Lib\site-packages\win32com\client\makepy.py The hard part is finding one that may be an interface to the TEM control functions. Look for anything with a name like "JEOL..." or "TEM3...." or even just "TEM...". If you find a possible candidate, select it and click OK. This will generate a python wrapper around the COM object which you can then inspect to see what functions are available. Just look for the newly generated py file in c:\Python25\Lib\site-packages\win32com\gen_py\ You can also browse through the COM objects and their functions without generating a py file using c:\Python25\Lib\site-packages\win32com\client\combrowse.py (see "Registered Type Libraries") I prefer to generate the py file with makepy.py and inspect it using vim.

reference: Quick Start to Client side COM and Python: http://docs.activestate.com/activepython/2.4/pywin32/html/com/win32com/HTML/QuickStartClientCom.html

If they don't use COM, maybe they have a library and some *.h files that we can use?

good luck!

Jim

leginonbot commented 8 months ago

Original Redmine Comment Author Name: Bridget Carragher (@bcarr-czi) Original Date: 2012-03-28T15:23:34Z

Hi Neil, I could not quite make sense of the email form Jeol. Are they saying that they heard from me that Leginon already works with the 1230? I certainly have not said this. If you like, send me the email thread and I will follow up with them. I woudl tell them that if they want Leginon on the 1230 (which I believe hey do) then they ought to get you some help with the TEM3 lickety split. I would also cc Jaap Brink on this emial - I saw him at the hybrid methods meeting and he appeared to be keen to ensure that Leginon would run on Jeols. Bridget

leginonbot commented 8 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2012-03-29T13:45:59Z

Hi Bridget, That is pretty much the whole email chain from JEOL. The problem is that I am not emailing an individual, but rather their general support. The engineers do not have their email on their business cards, so I have been working with Craig Koht (JEOL Central Region Service Manager)

Craig is unfamiliar with Leginon and he has been forwarding my information to Dr. Jaap Brink at JEOL, whose comments you see above. I just looked into it and it appears that Jaap works is still at Baylor CoM and used to publish with Wah Chiu, so maybe that explains something.

http://www.linkedin.com/pub/jaap-brink/13/51b/13a http://ncmi.bcm.tmc.edu/~brink

leginonbot commented 8 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2012-03-29T13:50:30Z

Hi Jim, thanks for the info. I've included a setup diagram for the systems we have. The microscope computer is separate from the camera computer connected via Ethernet.

Digital Micrograph is running on a separate machine from the EM, but it is a different driver than the one shown on the stokes webpage (they use JEM 12n0), I forget what we have, but that must be the TEM3 interface.

http://cryoem.nysbc.org/CemRobotics.html

Anyway, I was hoping to use the same system calls the Digital Micrograph uses to get the information.

leginonbot commented 8 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2012-05-21T10:53:14Z

We had a fairly successful trip to Northwestern on Friday (the 18th).

leginonbot commented 8 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2012-05-21T10:59:40Z

Computer dump

>>> from pyscope import jeol_tem3 as j
>>> s = j.Jeol()
Jeol 3100 is ready to serve and the current hitension is 100000.00.
Set function mode as 2
Objective Mini Lens is set as 63828.
Set function mode as 0
Objective Lens Fine is set as 33624.
Objective Lens Coarse is set as 35432.
Jeol is MAG1 mode and focus is defined for the future defocus.
Screen Position is up
Current MDS Mode is Exposure.
>>> s.theScreenPos
'up'
>>> s.getImageShift()
Image Shift 1 (x: 32768, y: 32768)
{'y': 0.0, 'x': 0.0}
>>> s.setImageShift({'x':0, 'y':0})
Image Shift 1 (x: 32768, y: 32768)
Image Shift 1 is set as (x: 32768, y: 32768).
set image shift (leginon):  0 0
set image shift (jeol):  0 0
>>> s.getDefocus()
Objective Lens Fine: 33624
Objective Lens Coarse: 35432
0.0
>>> s.setDefocus(-2e-6)
0
focus coarse: 36471 fine: 31
Objective Lens Coarse is set as 36471.
Objective Lens Fine is set as 31.
0
>>> s.getMagnification()
Magnification is 4000, X, X
4000
5580
>>> s.setMagnification(6570)
Magnification is set by 5000.
0
>>> s.getMagnification()
Magnification is 1000, X, X
1000
1770
>>> s.setMagnification(24400)
Magnification is set by 20000.
0
>>> s.getMagnification()
Magnification is 4000, X, X
4000
5580
>>> s.setMagnification(400)
Magnification is set by 250.
0
>>> s.getMagnification()
Magnification is 120, X, X
120
182

>>> from pyscope import gatan as g
>>> dir(g)
['Gatan', '__builtins__', '__doc__', '__file__', '__name__', 'ccdcamera', 'comar
ray', 'numpy', 'pythoncom', 'pywintypes', 'sys', 'time', 'win32com']
>>> c = g.Gatan()
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "C:\Python25\lib\site-packages\pyscope\gatan.py", line 34, in __init__
   raise RuntimeError('unable to initialize Gatan interface')
RuntimeError: unable to initialize Gatan interface

C:\Documents and Settings\Gatan user>cd C:\Python25\DLLs

C:\Python25\DLLs>regsvr32 tecnaiccd.dll

C:\Python25\DLLs>C:\Python25\python.exe
Python 2.5.4 (r254:67916, Dec 23 2008, 15:10:54) [MSC v.1310 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
from pyscope import gatan
c = gatan.Gatan()
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "C:\Python25\lib\site-packages\pyscope\gatan.py", line 34, in __init__
   raise RuntimeError('unable to initialize Gatan interface')
RuntimeError: unable to initialize Gatan interface

C:\Python25\Lib\site-packages\pyscope>updatecom.py
Generating .py files from type libraries...
initializing TEM Scripting Error, cannot find typelib for "TEM Scripting"
initializing Tecnai Scripting Error, cannot find typelib for "Tecnai Scripting"
initializing TOM Moniker Error, cannot find typelib for "TOMMoniker 1.0 Type Lib
rary"
initializing Tecnai Low Dose Kit Error, cannot find typelib for "Low Dose Server
Library"
initializing Tecnai Exposure Adaptor Error, cannot find typelib for "adaExp Libr
ary"
initializing Gatan CCD Camera done.
initializing Tietz CCD Camera Error, cannot find typelib for "CAMC4 1.0 Type Lib
rary"
initializing TIA CCD Camera Error, cannot find typelib for "ES Vision 3.0 Type L
ibrary"
enter to quit.

Note: Gatan works fine in updatecom.py

C:\Python25\Lib\site-packages\pyscope>C:\Python25\python.exe
Python 2.5.4 (r254:67916, Dec 23 2008, 15:10:54) [MSC v.1310 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
from pyscope import gatan
c = gatan.Gatan()
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "C:\Python25\lib\site-packages\pyscope\gatan.py", line 34, in __init__
   raise RuntimeError('unable to initialize Gatan interface')
RuntimeError: unable to initialize Gatan interface
leginonbot commented 8 months ago

Original Redmine Comment Author Name: Jim Pulokas (Jim Pulokas) Original Date: 2012-05-21T14:54:14Z

Neil, I sent you email regarding other possible versions of tecnaiccd. Also, just want to make sure you have Digital Micrograph running when you initialize the pyscope Gatan object. And there is also some weirdness in pyscope that sometime this:from pyscope import gatan c = gatan.Gatan() does not work properly and instead you have to do```from pyscope import registry cls = registry.getClass('Gatan') c = cls()

leginonbot commented 8 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2012-05-21T17:29:05Z

Actually, I do not know if Digital Micrograph was running. It probably was not. So, that might be it right there. I will try when I get back there.

leginonbot commented 8 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2012-06-01T13:02:10Z

From Jim:

Neil, I'm now collecting different versions of TecnaiCCD here:

http://www.scripps.edu/~pulokas/leginon/tecnaiccd/

The one you have already tried is now called: tecnaiccd.dll-2008-07-07 (unknown origin)

The new ones are: tecnaiccd.dll-2007-05-10-ami-t1 (currently used with our Gatan) tecnaiccd.dll-2008-10-30-nysbc (another one I found)

So the one we use is most likely to work, but let us know what works for you.

leginonbot commented 8 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2012-06-01T13:30:13Z

Still get the error with the gatan import, Digital Micrograph (version 1.84.1282) is open and I collected an image with it. I tried all three tecnaiccd.dll files you provided, but it still crashes on line 34 of gatan.py. I replicated the line below:

import win32com
camera = win32com.client.dynamic.Dispatch('TecnaiCCD.GatanCamera.2')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python25\lib\site-packages\win32com\client\dynamic.py", line 116, in Dispatch
    IDispatch, userName = _GetGoodDispatchAndUserName(IDispatch,userName,clsctx)
  File "C:\Python25\lib\site-packages\win32com\client\dynamic.py", line 108, in _GetGoodDispatchAndUserName
    return (_GetGoodDispatch(IDispatch, clsctx), userName)
  File "C:\Python25\lib\site-packages\win32com\client\dynamic.py", line 85, in _GetGoodDispatch
    IDispatch = pythoncom.CoCreateInstance(IDispatch, None, clsctx, pythoncom.IID_IDispatch)
pywintypes.com_error: (-2147221164, 'Class not registered', None, None)
leginonbot commented 8 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2012-06-01T13:49:02Z

tried:

REGSVR32 c:\Python25\DLLs\tecnaiccd.dll
leginonbot commented 8 months ago

Original Redmine Comment Author Name: Clint Potter (Clint Potter) Original Date: 2012-06-11T16:05:01Z

Neil,

Regarding the Gatan camera interface. What version of DM re you trying to use. Is this the 64 bit version?

Thanks.

Clint

leginonbot commented 8 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2012-06-11T21:46:07Z

Hi Clint, It is 32bit Windows XP install (I double checked that), so I can only assume it is 32bit DM. But yeah, when I googled the error -- the only suggestions I found was the 64bit/32bit problem. I asked Christian Suloway, but he also uses a Mac now.

I think I am going to try it again more systematically, with the 3 tecnaiccd.dll files that Jim gave me and cross my fingers.

leginonbot commented 8 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2012-07-20T16:49:59Z

DM version info 1.84.1282 (copyright 2010)

leginonbot commented 8 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2013-02-01T17:38:29Z

Just some personal notes...

Tried the JEOL 1400 again same error, tried Jim's new Gatan K2 interface on JEOL 1400, we have GSM 1.x NOT 2.x like the K2. Required installing Python 2.7, which was not compatible with JEOL.dll file (sadly needs to be recompiled). Nonetheless, the camera was giving a connection error. I assume it has something to do with SERIALEMCCD_PORT thing. This page was really helpful [[Gatan_K2_Installation_Notes]]

leginonbot commented 8 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2013-02-01T18:06:58Z

More notes:

Starting working the JEOL 3200. 

DM version info 1.85.1535 (copyright 1996-2010)

exact same error:

import win32com.client
camera = win32com.client.dynamic.Dispatch('TecnaiCCD.GatanCamera.2')

pywintypes.com_error: (-2147221164, 'Class not registered', None, None)
leginonbot commented 8 months ago

Original Redmine Comment Author Name: Jim Pulokas (Jim Pulokas) Original Date: 2013-07-11T18:14:41Z

Neil and Christian, What is the status of getting the Gatan camera to work? Was there any success with either the SerialEM plugin or tecnaiccd.dll? Jim

leginonbot commented 8 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2013-07-12T00:44:34Z

Hi Jim,

Christian has it working great. It is a spin off from your gatan-k2 file.

There are some quirks with zeroing the defocus and other JEOL-isms in the jeol_tem3, which is a result of leginon being designed around a tecnai.

leginonbot commented 8 months ago

Original Redmine Comment Author Name: Jim Pulokas (Jim Pulokas) Original Date: 2013-07-12T15:06:46Z

Sounds like you did your own custom hacks, so do you think you can contribute any of it back into pyscope? Not sure if you are up to date with the current trunk, but gatank2.py has become dmsem.py, to be more generic for any camera (not just K2) that is controlled through the DM SerialEM plug-in. Will probably remove gatank2.py soon. Since you got it to work with your CCD, do you have any suggestions for making dmsem.py more flexible for different cameras? Currently it defines a class for each camera. For the multiple camera case, each subclass has a different camera ID for DM. This is not too flexible since the camera id is hard coded into the class definition. It would work better to have the DM camera id configured in instruments.cfg.

leginonbot commented 8 months ago

Original Redmine Comment Author Name: Christian Suloway (Christian Suloway) Original Date: 2013-07-15T14:18:54Z

I had to modify gatank2.py for the older (1.X?) version of the SerialEM plugin. This are not checked into the svn tree. I think the only difference was the readout protocol. I will look at dmsem.py to see if any changes are necessary and what the best way to implement them might be.

leginonbot commented 8 months ago

Original Redmine Comment Author Name: Anchi Cheng (@anchi2c) Original Date: 2014-02-04T17:08:10Z

model name correction from 3400 to3200

leginonbot commented 8 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2015-08-06T19:46:36Z

UltraScan was fixed by using dmsem.py instead of gatan.py:

[camera]
class: dmsem.GatanUltraScan
zplane: 50
width: 4096
height: 4096
leginonbot commented 8 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2015-08-07T20:35:57Z

Note I also had to change the camera id number in dmsem.py, otherwise it used the Orius camera:

class GatanOrius(DMSEM):
    name = 'GatanOrius'
    cameraid = 1
    binning_limits = [1,2,4]
    binmethod = 'exact'

class GatanUltraScan(DMSEM):
    name = 'GatanUltraScan'
    cameraid = 0
    binning_limits = [1,2,4,8]
    binmethod = 'exact'