[cameras.uc480] Cannot import uc480

[spacecrusador]

I've been working so far with a ThorLabs camera on my laptop and didn't have any trouble. I have been trying to do the same on a desktop and I am not sure what is wrong but I cannot get it working. Any help would be very appreciated :)

On my laptop, with my camera plugged or not as well as with or without the .dll files added to the path, I could run from instrumental.drivers.cameras import uc480 without any error raised And in either of the cases, the camera was working properly.

My desktop is running on windows 10. I use python 3.7.1, NiceLib 0.5.2, pywin32 223 and Instrumental-lib 0.6.dev0

On my desktop, I have tried three ways of adding the three .dll files to the path:

• By adding the directory location to the user variable 'Path'
• By copying the three .dll files to the current directory (might be too easy!)
• By adding the directory to the path using the sys library:

import sys sys.path.append(r"C:\Program Files (x86)\Thorlabs\DCx Cameras\Driver\i386") sys.path.append(r"C:\Program Files (x86)\Thorlabs\DCx Cameras\Driver\amd64") sys.path.append(r"C:\Program Files\IDS\uEye\USB driver package")

I have had the following error (I run python on a jupyter notebook):

ModuleNotFoundError                       Traceback (most recent call last)
~\Anaconda3\lib\site-packages\nicelib\__init__.py in load_lib(name, pkg, dir, builder, kwargs)
     61     try:
---> 62         lib_module = import_module(lib_name, pkg)
     63     except ImportError:

~\Anaconda3\lib\importlib\__init__.py in import_module(name, package)
    126             level += 1
--> 127     return _bootstrap._gcd_import(name[level:], package, level)
    128 

~\Anaconda3\lib\importlib\_bootstrap.py in _gcd_import(name, package, level)

~\Anaconda3\lib\importlib\_bootstrap.py in _find_and_load(name, import_)

~\Anaconda3\lib\importlib\_bootstrap.py in _find_and_load_unlocked(name, import_)

ModuleNotFoundError: No module named 'instrumental.drivers.cameras._uc480lib'

During handling of the above exception, another exception occurred:

StopIteration                             Traceback (most recent call last)
~\Anaconda3\lib\site-packages\nicelib\process.py in modify_pattern(tokens, pattern)
   2007             if keep != 'a':
-> 2008                 t = next(it)
   2009         except StopIteration:

StopIteration: 

During handling of the above exception, another exception occurred:

StopIteration                             Traceback (most recent call last)
~\Anaconda3\lib\site-packages\nicelib\process.py in modify_pattern(tokens, pattern)
   2013                     yield buf_tok
-> 2014             raise StopIteration
   2015 

StopIteration: 

The above exception was the direct cause of the following exception:

**RuntimeError                              Traceback (most recent call last)
<ipython-input-1-64a9d6f4b44c> in <module>
----> 1 from instrumental.drivers.cameras import uc480
      2 #from instrumental import instrument, list_instruments
      3 #list_instruments()
      4 #import matplotlib.pyplot as plt
      5 #import numpy as np

~\Anaconda3\lib\site-packages\instrumental\drivers\cameras\uc480.py in <module>
     30 _INST_CLASSES = ['UC480_Camera']
     31 
---> 32 info = load_lib('uc480', __package__)
     33 ffi = info._ffi
     34 

~\Anaconda3\lib\site-packages\nicelib\__init__.py in load_lib(name, pkg, dir, builder, kwargs)
     65             builder = prefix + '_build_{}'.format(name)
     66         build_module = import_module(builder, pkg)
---> 67         build_module.build(**kwargs)
     68         lib_module = import_module(lib_name, pkg)
     69 

~\Anaconda3\lib\site-packages\instrumental\drivers\cameras\_build_uc480.py in build()
     33 def build():
     34     build_lib(header_info, lib_names, '_uc480lib', __file__, token_hooks=(declspec_hook,),
---> 35               ignore_system_headers=True)
     36 
     37 

~\Anaconda3\lib\site-packages\nicelib\build.py in build_lib(header_info, lib_name, module_name, filedir, ignored_headers, ignore_system_headers, preamble, token_hooks, ast_hooks, hook_groups, debug_file, logbuf, load_dump_file, save_dump_file)
    129                                  debug_file=debug_file,
    130                                  load_dump_file=load_dump_file,
--> 131                                  save_dump_file=save_dump_file)
    132     else:
    133         if not preamble:

~\Anaconda3\lib\site-packages\nicelib\process.py in process_headers(header_paths, predef_path, update_cb, ignored_headers, ignore_system_headers, debug_file, preamble, token_hooks, ast_hooks, hook_groups, return_ast, load_dump_file, save_dump_file)
   1729                           return_ast=return_ast,
   1730                           load_dump_file=load_dump_file,
-> 1731                           save_dump_file=save_dump_file)
   1732 
   1733 

~\Anaconda3\lib\site-packages\nicelib\process.py in process_source(source, predef_path, update_cb, ignored_headers, ignore_system_headers, debug_file, preamble, token_hooks, ast_hooks, hook_groups, return_ast, load_dump_file, save_dump_file)
   1775                     ast_hooks=ast_hooks,
   1776                     debug_file=debug_file)
-> 1777     header_src, macro_src, tree, argnames = gen.generate()
   1778 
   1779     if return_ast:

~\Anaconda3\lib\site-packages\nicelib\process.py in generate(self)
   1287         for hook in self.token_hooks:
   1288             log.debug("Applying hook '{}'".format(hook.__name__))
-> 1289             tokens = hook(tokens)
   1290 
   1291         # Generate parseable chunks

~\Anaconda3\lib\site-packages\nicelib\process.py in add_line_directive_hook(tokens)
   2032     chunk_start_line = -1
   2033     high_mark, low_mark = 0, -1
-> 2034     for t in tokens:
   2035         if (t.fpath, t.line) != (cur_fpath, expected_line):
   2036             if high_mark > low_mark >= 0:

~\Anaconda3\lib\site-packages\nicelib\process.py in cdecl_hook(tokens)
   2078     Enabled by default.
   2079     """
-> 2080     for token in tokens:
   2081         if token not in ('cdecl', '_cdecl', '__cdecl'):
   2082             yield token

~\Anaconda3\lib\site-packages\nicelib\process.py in stdcall_hook(tokens)
   2271 
   2272     while True:
-> 2273         token = ph.pop()
   2274         if token == '(':
   2275             if ph.peek_true_token() in ('__stdcall', 'WINAPI'):

~\Anaconda3\lib\site-packages\nicelib\process.py in pop(self)
   2125         try:
   2126             if not self.peek_deque:
-> 2127                 self.peek_deque.append(next(self.tok_it))
   2128         except StopIteration:
   2129             pass

RuntimeError: generator raised StopIteration

[natezb]

You might try updating to the git version of NiceLib. This seems like the bug fixed in mabuchilab/NiceLib@5dc3f7594a7fa51312e8183e75e70491c68348de.

[spacecrusador]

Unfortunately, this didn't work. I have installed windows 10 today on another dektop and tried similar things as the ones mentioned above and have ended up with the same error.

[natezb]

Can you provide the exact traceback like in your previous comment? If NiceLib has been successfully updated to the current dev version, you can't be getting the exact same traceback (since the code in that traceback has since changed).

[spacecrusador]

Sure, here it is:

ModuleNotFoundError                       Traceback (most recent call last)
~\Anaconda3\lib\site-packages\nicelib\__init__.py in load_lib(name, pkg, dir, builder, kwargs)
     66         log.info('Loading %s from %s...', lib_name, pkg)
---> 67         lib_module = import_module(lib_name, pkg)
     68     except ImportError:

~\Anaconda3\lib\importlib\__init__.py in import_module(name, package)
    126             level += 1
--> 127     return _bootstrap._gcd_import(name[level:], package, level)
    128 

~\Anaconda3\lib\importlib\_bootstrap.py in _gcd_import(name, package, level)

~\Anaconda3\lib\importlib\_bootstrap.py in _find_and_load(name, import_)

~\Anaconda3\lib\importlib\_bootstrap.py in _find_and_load_unlocked(name, import_)

ModuleNotFoundError: No module named 'instrumental.drivers.cameras._uc480lib'

During handling of the above exception, another exception occurred:

CDefError                                 Traceback (most recent call last)
<ipython-input-2-64a9d6f4b44c> in <module>
----> 1 from instrumental.drivers.cameras import uc480
      2 #from instrumental import instrument, list_instruments
      3 #list_instruments()
      4 #import matplotlib.pyplot as plt
      5 #import numpy as np

~\Anaconda3\lib\site-packages\instrumental\drivers\cameras\uc480.py in <module>
     30 _INST_CLASSES = ['UC480_Camera']
     31 
---> 32 info = load_lib('uc480', __package__)
     33 ffi = info._ffi
     34 

~\Anaconda3\lib\site-packages\nicelib\__init__.py in load_lib(name, pkg, dir, builder, kwargs)
     71         log.info('Loading build module %s from %s...', builder, pkg)
     72         build_module = import_module(builder, pkg)
---> 73         build_module.build(**kwargs)
     74         lib_module = import_module(lib_name, pkg)
     75 

~\Anaconda3\lib\site-packages\instrumental\drivers\cameras\_build_uc480.py in build()
     33 def build():
     34     build_lib(header_info, lib_names, '_uc480lib', __file__, token_hooks=(declspec_hook,),
---> 35               ignore_system_headers=True)
     36 
     37 

~\Anaconda3\lib\site-packages\nicelib\build.py in build_lib(header_info, lib_name, module_name, filedir, ignored_headers, ignore_system_headers, preamble, token_hooks, ast_hooks, hook_groups, debug_file, logbuf, load_dump_file, save_dump_file)
    151     logbuf.write("Compiling cffi module...\n")
    152     ffi = cffi.FFI()
--> 153     ffi.cdef(clean_header_str)
    154     ffi.set_source('.' + module_name, None)
    155     ffi.compile(tmpdir=filedir)

~\Anaconda3\lib\site-packages\cffi\api.py in cdef(self, csource, override, packed)
    105         cdef are packed, i.e. laid out without any field alignment at all.
    106         """
--> 107         self._cdef(csource, override=override, packed=packed)
    108 
    109     def embedding_api(self, csource, packed=False):

~\Anaconda3\lib\site-packages\cffi\api.py in _cdef(self, csource, override, **options)
    119         with self._lock:
    120             self._cdef_version = object()
--> 121             self._parser.parse(csource, override=override, **options)
    122             self._cdefsources.append(csource)
    123             if override:

~\Anaconda3\lib\site-packages\cffi\cparser.py in parse(self, csource, override, packed, dllexport)
    313                              'packed': packed,
    314                              'dllexport': dllexport}
--> 315             self._internal_parse(csource)
    316         finally:
    317             self._options = prev_options

~\Anaconda3\lib\site-packages\cffi\cparser.py in _internal_parse(self, csource)
    353                     else:
    354                         realtype, quals = self._get_type_and_quals(
--> 355                             decl.type, name=decl.name, partial_length_ok=True)
    356                     self._declare('typedef ' + decl.name, realtype, quals=quals)
    357                 elif decl.__class__.__name__ == 'Pragma':

~\Anaconda3\lib\site-packages\cffi\cparser.py in _get_type_and_quals(self, typenode, name, partial_length_ok)
    542         if isinstance(typenode, pycparser.c_ast.PtrDecl):
    543             # pointer type
--> 544             itemtype, itemquals = self._get_type_and_quals(typenode.type)
    545             tp = self._get_type_pointer(itemtype, itemquals, declname=name)
    546             quals = self._extract_quals(typenode)

~\Anaconda3\lib\site-packages\cffi\cparser.py in _get_type_and_quals(self, typenode, name, partial_length_ok)
    585             if isinstance(type, pycparser.c_ast.Struct):
    586                 # 'struct foobar'
--> 587                 tp = self._get_struct_union_enum_type('struct', type, name)
    588                 return tp, quals
    589             #

~\Anaconda3\lib\site-packages\cffi\cparser.py in _get_struct_union_enum_type(self, kind, type, name, nested)
    732         #
    733         if tp.fldnames is not None:
--> 734             raise CDefError("duplicate declaration of struct %s" % name)
    735         fldnames = []
    736         fldtypes = []

CDefError: <cdef source string>:16: duplicate declaration of struct __autotag_BOARDINFO

[natezb]

Yes, this is a different error. I haven't been able to reproduce it on my system, so it might have to do with a change in the header file. Could you upload the DCx header file(s) from your system? Mine are in C:\Program Files\Thorlabs\Scientific Imaging\DCx Camera Support\Develop\Include.

[lucashofer]

I'm getting a similar error as well when running a ThorLabs camera on Windows 10 with NiceLib 0.6 and Instrumental 0.5. I've included the relevant header files

uc480.zip

along with the error output.

  File "C:\Users\LocalUser\Anaconda3\lib\site-packages\nicelib\__init__.py", line 67, in load_lib
    lib_module = import_module(lib_name, pkg)
  File "C:\Users\LocalUser\Anaconda3\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'instrumental.drivers.cameras._uc480lib'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\LocalUser\Downloads\Instrumental2\instrumental\drivers\cameras\uc480.py", line 32, in <module>
    info = load_lib('uc480', __package__)
  File "C:\Users\LocalUser\Anaconda3\lib\site-packages\nicelib\__init__.py", line 73, in load_lib
    build_module.build(**kwargs)
  File "C:\Users\LocalUser\Downloads\Instrumental2\instrumental\drivers\cameras\_build_uc480.py", line 35, in build
    ignore_system_headers=True)
  File "C:\Users\LocalUser\Anaconda3\lib\site-packages\nicelib\build.py", line 153, in build_lib
    ffi.cdef(clean_header_str)
  File "C:\Users\LocalUser\Anaconda3\lib\site-packages\cffi\api.py", line 107, in cdef
    self._cdef(csource, override=override, packed=packed)
  File "C:\Users\LocalUser\Anaconda3\lib\site-packages\cffi\api.py", line 121, in _cdef
    self._parser.parse(csource, override=override, **options)
  File "C:\Users\LocalUser\Anaconda3\lib\site-packages\cffi\cparser.py", line 315, in parse
    self._internal_parse(csource)
  File "C:\Users\LocalUser\Anaconda3\lib\site-packages\cffi\cparser.py", line 355, in _internal_parse
    decl.type, name=decl.name, partial_length_ok=True)
  File "C:\Users\LocalUser\Anaconda3\lib\site-packages\cffi\cparser.py", line 544, in _get_type_and_quals
    itemtype, itemquals = self._get_type_and_quals(typenode.type)
  File "C:\Users\LocalUser\Anaconda3\lib\site-packages\cffi\cparser.py", line 587, in _get_type_and_quals
    tp = self._get_struct_union_enum_type('struct', type, name)
  File "C:\Users\LocalUser\Anaconda3\lib\site-packages\cffi\cparser.py", line 734, in _get_struct_union_enum_type
    raise CDefError("duplicate declaration of struct %s" % name)
cffi.error.CDefError: <cdef source string>:29: duplicate declaration of struct __autotag_BOARDINFO
>>>

[natezb]

Unfortunately, those headers seem to be the same as mine, so that doesn't explain it. This error is reminiscent of an issue that was fixed a few years back. Basically, if you have multiple typedefs for a single struct definition, e.g.

typedef struct
{
  char          SerNo[12];          // e.g. "1234512345"  (11 char)
  char          ID[20];             // e.g. "Company Name"
  char          Version[10];        // e.g. "V2.10"  (9 char)
  char          Date[12];           // e.g. "24.01.2006" (11 char)
  unsigned char Select;             // contains board select number for multi board support
  unsigned char Type;               // e.g. IS_BOARD_TYPE_UC480_USB
  char          Reserved[8];        // (7 char)
} BOARDINFO, *PBOARDINFO;

pycparser will parse this into an AST that contains two separate typedefs (one for BOARDINFO and one for PBOARDINFO), each with a copy of the struct definition. If we regenerate a header from this AST, we'll have duplicated the struct definition, which cffi does not like.

What versions of cffi and pycparser are you using? I wonder if either of those has changed this behavior recently.

I'll probably need to ask for some extra debug info in a little while.

[lucashofer]

I'm using cffi 1.11.5 and pycparser 2.19 currently.

[spacecrusador]

Hi thank you very much for helping!

I've managed to make it work by downgrading pycparser to 2.18!

[natezb]

Ha, you just beat me to comment. I've just pushed mabuchilab/NiceLib@2de6f82e39f45f73f1c1da264158011f43ece326, which fixes the issue for me (I was able to reproduce by upgrading pycparser to 2.19). The new version of pycparser exposed a latent bug in NiceLib that should now be fixed. I'm considering making a change to how these kinds of structs are generated from AST, but this should fix things in the meantime.

[lucashofer]

I got another error with the new NiceLib version. However, downgrading to pycparser 2.18 ending up working for me as well.

[natezb]

Could you post that error traceback if you get a chance? I'd like to make sure this works with the latest version.