Build with EPICS 7 (unbundled pcas)

[kasemir]

pcaspy expects to find cas and gdd in the EPICSBASE directory. With EPICS 7, pcas has been unbundled as https://github.com/epics-modules/pcas. So when building, you end with for example this directory layout:

# EPICS Base
base-7.0.1.1

# Unbundled PCAS with casdef.h, libcas.a, gdd
pcas-4.13.2

# PCASPY
pcaspy-0.7.1path

The original setup.py will fail to find the cas and gdd include and library files because it only checks for EPICSBASE:

$ python setup.py build
...
pcaspy/casdef_wrap.cpp:3565:20: fatal error: casdef.h: No such file or directory

One possible fix would be to not only require an EPICSBASE environment variable but also a new PCAS environment variable.

As a hack, below is a setup.py where I simply added the relative ../pcas-4.13.2/.. to the include and library paths to get it compiled on Linux.

---

#!/usr/bin/env python

"""
setup.py file for pcaspy
"""
import os
import platform
import imp
import shutil
import subprocess

# Use setuptools to include build_sphinx, upload/sphinx commands
try:
    from setuptools import setup, Extension
except:
    from distutils.core import setup, Extension

from distutils.command.build_py import build_py as _build_py

# build_py runs before build_ext so that swig generated module is not copied
# See http://bugs.python.org/issue7562
# This is a workaround to run build_ext ahead of build_py
class build_py(_build_py):
    def run(self):
        self.run_command('build_ext')
        _build_py.run(self)

# define EPICS base path and host arch
EPICSBASE = os.environ.get("EPICS_BASE")
if not EPICSBASE:
    EPICSROOT = os.environ.get("EPICS")
    if EPICSROOT:
        EPICSBASE = os.path.join(EPICSROOT, 'base')
if not EPICSBASE:
    raise IOError("Please define EPICS_BASE environment variable")
if not os.path.exists(EPICSBASE):
    raise IOError("Please correct EPICS_BASE environment variable, "
                  "the path {0} does not exist".format(EPICSBASE))

HOSTARCH  = os.environ.get("EPICS_HOST_ARCH")
if not HOSTARCH:
    raise IOError("Please define EPICS_HOST_ARCH environment variable")

# check EPICS version
PRE315 = True
if os.path.exists(os.path.join(EPICSBASE, 'include', 'compiler')):
    PRE315 = False

# common libraries to link
libraries = ['cas', 'ca', 'gdd', 'Com']
if PRE315:
    libraries.insert(0, 'asIoc')
else:
    libraries.insert(0, 'dbCore')
umacros = []
macros   = []
cflags = []
lflags = []
dlls = []
extra_objects = []
# platform dependent libraries and macros
UNAME = platform.system()
if  UNAME.find('CYGWIN') == 0:
    UNAME = "cygwin32"
    CMPL = 'gcc'
elif UNAME == 'Windows':
    UNAME = 'WIN32'
    # MSVC compiler
    static = False
    if HOSTARCH in ['win32-x86', 'windows-x64', 'win32-x86-debug', 'windows-x64-debug']:
        dlls = ['cas.dll', 'ca.dll', 'gdd.dll', 'Com.dll']
        if PRE315:
            dlls += ['dbIoc.dll', 'dbStaticIoc.dll', 'asIoc.dll']
        else:
            dlls += ['dbCore.dll']
        for dll in dlls:
            dllpath = os.path.join(EPICSBASE, 'bin', HOSTARCH, dll)
            if not os.path.exists(dllpath):
                static = True
                break
            shutil.copy(dllpath,
                        os.path.join(os.path.dirname(os.path.abspath(__file__)), 'pcaspy'))
        macros += [('_CRT_SECURE_NO_WARNINGS', 'None'), ('_CRT_NONSTDC_NO_DEPRECATE', 'None'), ('EPICS_CALL_DLL', '')]
        cflags += ['/Z7']
        CMPL = 'msvc'
    if HOSTARCH in ['win32-x86-static', 'windows-x64-static'] or static:
        libraries += ['ws2_32', 'user32', 'advapi32']
        macros += [('_CRT_SECURE_NO_WARNINGS', 'None'), ('_CRT_NONSTDC_NO_DEPRECATE', 'None'), ('EPICS_DLL_NO', '')]
        umacros+= ['_DLL']
        cflags += ['/EHsc', '/Z7']
        lflags += ['/LTCG']
        if HOSTARCH[-5:] == 'debug':
            libraries += ['msvcrtd']
            lflags += ['/NODEFAULTLIB:libcmtd.lib']
        else:
            libraries += ['msvcrt']
            lflags += ['/NODEFAULTLIB:libcmt.lib']
        CMPL = 'msvc'
    # GCC compiler
    if HOSTARCH in ['win32-x86-mingw', 'windows-x64-mingw']:
        macros += [('_MINGW', ''), ('EPICS_DLL_NO', '')]
        lflags += ['-static',]
        CMPL = 'gcc'
    if HOSTARCH == 'windows-x64-mingw':
        macros += [('MS_WIN64', '')]
        CMPL = 'gcc'
elif UNAME == 'Darwin':
    CMPL = 'clang'
    extra_objects = [os.path.join(EPICSBASE, 'lib', HOSTARCH, 'lib%s.a'%lib) for lib in libraries]
    libraries = []
elif UNAME == 'Linux':
    # necessary when EPICS is statically linked
    extra_objects = list()
    for lib in libraries:
        if 'cas' in lib  or  'gdd' in lib:
            extra_objects.append(os.path.join('../pcas-4.13.2/lib', HOSTARCH, 'lib%s.a'%lib))
        else:
            extra_objects.append(os.path.join(EPICSBASE, 'lib', HOSTARCH, 'lib%s.a'%lib))

    libraries = ['rt']
    if subprocess.call('nm -u %s | grep -q rl_' % os.path.join(EPICSBASE, 'lib', HOSTARCH, 'libCom.a'), shell=True) == 0:
        libraries += ['readline']
    CMPL = 'gcc'
elif UNAME == 'SunOS':
    # OS_CLASS used by EPICS
    UNAME = 'solaris'
    CMPL = 'solStudio'
else:
    raise IOError("Unsupported OS {0}".format(UNAME))

cas_module = Extension('pcaspy._cas',
                       sources  =[os.path.join('pcaspy','casdef.i'),
                                  os.path.join('pcaspy','pv.cpp'),
                                  os.path.join('pcaspy','channel.cpp'),],
                       swig_opts=['-c++','-threads','-nodefaultdtor','-I%s'% os.path.join(EPICSBASE, 'include')],
                       extra_compile_args=cflags,
                       include_dirs = [ '../pcas-4.13.2/include',
                                        os.path.join(EPICSBASE, 'include'),
                                        os.path.join(EPICSBASE, 'include', 'os', UNAME),
                                        os.path.join(EPICSBASE, 'include', 'compiler', CMPL)],
                       library_dirs = [ os.path.join('../pcas-4.13.2/lib', HOSTARCH),
                                        os.path.join(EPICSBASE, 'lib', HOSTARCH),],
                       libraries = libraries,
                       extra_link_args = lflags,
                       extra_objects = extra_objects,
                       define_macros = macros,
                       undef_macros  = umacros,)
# other *NIX linker has runtime library path option
if UNAME not in ['WIN32', 'Darwin', 'Linux']:
    cas_module.runtime_library_dirs += os.path.join(EPICSBASE, 'lib', HOSTARCH),

long_description = open('README.rst').read()
_version = imp.load_source('_version','pcaspy/_version.py')

dist = setup (name = 'pcaspy',
              version = _version.__version__,
              description = """Portable Channel Access Server in Python""",
              long_description = long_description,
              author      = "Xiaoqiang Wang",
              author_email= "xiaoqiangwang@gmail.com",
              url         = "https://pypi.python.org/pypi/pcaspy",
              ext_modules = [cas_module],
              packages    = ["pcaspy"],
              package_data={"pcaspy" : dlls},
              cmdclass    = {'build_py':build_py},
              license     = "BSD",
              platforms   = ["Windows","Linux", "Mac OS X"],
              classifiers = ['Development Status :: 4 - Beta',
                             'Environment :: Console',
                             'Intended Audience :: Developers',
                             'License :: OSI Approved :: BSD License',
                             'Programming Language :: C++',
                             'Programming Language :: Python :: 2',
                             'Programming Language :: Python :: 3',
                             ],
              )

# Re-run the build_py to ensure that swig generated py files are also copied
build_py = build_py(dist)
build_py.ensure_finalized()
build_py.run()

[xiaoqiangwang]

Thanks for the heads up. The separate of pcas library is indeed a confusion. Currently my simple workaround is to still build the pcas together with epics base, and that is supported by the build system.

$ cd /home/scratch/base-7.0.1.1/modules/
$ git clone git@github.com:epics-modules/pcas.git
$ echo EPICS_BASE = /home/scratch/base-7.0.1.1 > RELEASE.local
$ echo -e "SUBMODULES += pcas\npcas_DEPEND_DIRS = libcom" > Makefile.local
$ cd /home/scratch/base-7.0.1.1
$ make

Maybe a separate PCAS environment variable could be used to locate the library?

[kasemir]

As for running with EPICS 7 and the unbundled pcas-4.13.2, all seems to work except the 'asyn' mode results in a warning.

For example, running python pysh.py and then

caput -S -c -w 10 MTEST:COMMAND whoami

works OK, the MTEST:OUTPUT is updated, the completion callback is performed, but there's this additional message in the python terminal:

('DEBUG: Run ', 'whoami')
Application returned 34340869 from casChannel::write() - expected S_casApp_asyncCompletion
('DEBUG: Finish ', 'whoami')

[kasemir]

Building pcas inside the base/modules is reasonable, I hadn't thought about that. Maybe mention that on the installation page. Certainly a lot easier than updating setup.py to handle a potentially separate location for gdd & pcas.

[xiaoqiangwang]

After a brief check the error related to asynchronous write is related to the change of status code definition. (PCASpy)https://github.com/paulscherrerinstitute/pcaspy/blob/master/pcaspy/errMdef.i (EPICS7) https://github.com/epics-base/epics-base/blob/7.0/modules/libcom/src/error/errMdef.h (EPICS3) https://github.com/epics-base/epics-base/blob/3.16/src/libCom/error/errMdef.h

Because pcaspy copies these constants to python file, in EPICS 7, they would have totally different meanings. That confuses the pcas library. I will check whether feasible not to copy the constants definition.

[xiaoqiangwang]

Addressed in 0.7.2 release.

[kasemir]

Should the build setup be reconsidered? PCAS remains unbundled, to be built outside of the EPICS 7 source tree. Might be time to have a designated environment variable tell pcaspy where to find the PCAS library

[xiaoqiangwang]

I added the check of PCAS environment variable in case no bundled pcas is found. https://github.com/paulscherrerinstitute/pcaspy/commit/ae786ccb68bc08acdf505700755ceb2493de768c

[kasemir]

Excellent, thank you!