Bug Report: Release checkout procedure for release v3.0.0 doesn't work

bdorney commented 6 years ago

Brief summary of issue

Tried checkout release v3.0.0 from xhal following procedure described at:

https://github.com/cms-gem-daq-project/xhal/wiki/Quick-start-guide

And was unsuccessful.

Types of issue

[X] Bug report (report an issue with the code)
[ ] Feature request (request for change which adds functionality)

Looks like the problem is just one of missing libraries that were mistakenly not included in the release.

Expected Behavior

Documentation should work out of the box. Required libraries should be included in release.

Current Behavior

Required libraries are missing in the release.

Steps to Reproduce (for bugs)

From: https://github.com/cms-gem-daq-project/xhal/wiki/Quick-start-guide

git checkout tags/v3.0.0
python .github/get_binaries -t v3.0.0 -l .github/uploads.cfg This generates an error since get_binaries is a typo.
When correcting the typo:

(default)[gemuser@gem904qc8daq xhal]$ python .github/get_binaries.py -t v3.0.0 -l .github/uploads.cfg 
downloading https://github.com/cms-gem-daq-project/xhal/releases/download/v3.0.0/ipbus
HTTP Error: 404 https://github.com/cms-gem-daq-project/xhal/releases/download/v3.0.0/libxhal.so
HTTP Error: 404 https://github.com/cms-gem-daq-project/xhal/releases/download/v3.0.0/librwreg.so
HTTP Error: 404 https://github.com/cms-gem-daq-project/xhal/releases/download/v3.0.0/libwiscrpcsvc.so
HTTP Error: 404 https://github.com/cms-gem-daq-project/xhal/releases/download/v3.0.0/libxhal_ctp7.so
downloading https://github.com/cms-gem-daq-project/xhal/releases/download/v3.0.0/libxerces-c.so
downloading https://github.com/cms-gem-daq-project/xhal/releases/download/v3.0.0/liblmdb.so
downloading https://github.com/cms-gem-daq-project/xhal/releases/download/v3.0.0/liblog4cplus.so

The 404 errors are generated because release v3.0.0 does not have those libraries listed:

https://github.com/cms-gem-daq-project/xhal/releases/tag/v3.0.0

Indeed they are missing in the release.

When trying to run reg_interface a crash occurs since necessary libraries are not found:

(default)[gemuser@gem904qc8daq reg_interface]$ python reg_interface.py 
Traceback (most recent call last):
  File "reg_interface.py", line 4, in <module>
    from rw_reg import *
  File "/home/gemuser/gemdaq/xhal/python/reg_interface/rw_reg.py", line 17, in <module>
    lib = CDLL(os.getenv("XHAL_ROOT")+"/lib/x86_64/librpcman.so")
  File "/usr/lib64/python2.7/ctypes/__init__.py", line 360, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libwiscrpcsvc.so: cannot open shared object file: No such file or directory

Possible Solution (for bugs)

Step two above should be: python .github/get_binaries.py -t <tag_name> -l .github/uploads.cfg

Right now I'll just build the required libraries myself but I think this probably defeats the purpose of the release.

Context (for feature requests)

User will complain it doesn't work.

Your Environment

Version used: https://github.com/cms-gem-daq-project/xhal/releases/tag/v3.0.0
Shell used: /bin/bash

bdorney commented 6 years ago

I've corrected the get_binaries typo in the project wiki.

mexanick commented 6 years ago

For some reason certain uploads were missing in the release. I'll investigate the reason (I actually doubt this is a real bug, it might be rather user mistake (mine) when configuring the release upload scripts)

bdorney commented 6 years ago

@mexanick New details on this issue, trying to checkout release v3.0.0 on the p5 machine:

$ git clone git@github.com:cms-gem-daq-project/xhal.git
$ git checkout tags/v3.0.0
$ python .github/get_binaries.py -t v3.0.0 -l .github/uploads.cfg
Traceback (most recent call last):
  File ".github/get_binaries.py", line 42, in <module>
    main()
  File ".github/get_binaries.py", line 38, in main
    local_path = val.replace('$XHAL_ROOT', os.getenv('XHAL_ROOT'))
TypeError: expected a character buffer object

I think the instructions under Getting the code need to add sourcing the setup scripts before the call of the python script.

Okay sourcing the setup script causes the error message to no longer appear. However the script seems to hang indefinitely (probably because it does not play well with the network configuration).

Giving a keyboard interrupt causes the following output (maybe useful):

$ python .github/get_binaries.py -t v3.0.0 -l .github/uploads.cfg
URL Error: [Errno 110] Connection timed out https://github.com/cms-gem-daq-project/xhal/releases/download/v3.0.0/ipbus
^CTraceback (most recent call last):
  File ".github/get_binaries.py", line 42, in <module>
    main()
  File ".github/get_binaries.py", line 39, in main
    dlfile(url, local_path)
  File ".github/get_binaries.py", line 15, in dlfile
    f = urlopen(url)
  File "/usr/lib64/python2.6/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib64/python2.6/urllib2.py", line 391, in open
    response = self._open(req, data)
  File "/usr/lib64/python2.6/urllib2.py", line 409, in _open
    '_open', req)
  File "/usr/lib64/python2.6/urllib2.py", line 369, in _call_chain
    result = func(*args)
  File "/usr/lib64/python2.6/urllib2.py", line 1198, in https_open
    return self.do_open(httplib.HTTPSConnection, req)
  File "/usr/lib64/python2.6/urllib2.py", line 1162, in do_open
    h.request(req.get_method(), req.get_selector(), req.data, headers)
  File "/usr/lib64/python2.6/httplib.py", line 914, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib64/python2.6/httplib.py", line 951, in _send_request
    self.endheaders()
  File "/usr/lib64/python2.6/httplib.py", line 908, in endheaders
    self._send_output()
  File "/usr/lib64/python2.6/httplib.py", line 780, in _send_output
    self.send(msg)
  File "/usr/lib64/python2.6/httplib.py", line 739, in send
    self.connect()
  File "/usr/lib64/python2.6/httplib.py", line 1112, in connect
    sock = socket.create_connection((self.host, self.port), self.timeout)
  File "/usr/lib64/python2.6/socket.py", line 560, in create_connection
    sock.connect(sa)
  File "<string>", line 1, in connect
KeyboardInterrupt

It seems the python script get_binaries.py relies on https communication; but the technical network seems to only work with ssh forwarding via the ssh agent.

@jsturdy any suggestions?

mexanick commented 6 years ago

release outdated, closing

cms-gem-daq-project / xhal