search

ubuntu / ubuntu-make

Easy setup of common tools for developers on Ubuntu.
GNU General Public License v3.0
1.2k stars 189 forks source link

latest versions of maven use sha512 not sha1 #584

Closed pellcorp closed 5 years ago

pellcorp commented 5 years ago

The latest versions of maven use sha512 not sha1, so umake fails. This MR updates to use sha512.

Choose installation path: /home/jason/.local/share/umake/maven/maven-lang ERROR: https://www.apache.org/dist/maven/maven-3/3.6.0/binaries/apache-maven-3.6.0-bin.tar.gz.sha1 couldn't finish download: 404 Client Error: Not Found for url: https://www.apache.org/dist/maven/maven-3/3.6.0/binaries/apache-maven-3.6.0-bin.tar.gz.sha1 ERROR: Unhandled exception Traceback (most recent call last): File "/usr/lib/python3/dist-packages/umake/tools.py", line 158, in wrapper function(*args, **kwargs) File "/usr/lib/python3/dist-packages/umake/frameworks/maven.py", line 102, in get_sha_and_start_download checksum = res.buffer.getvalue().decode('utf-8').split()[0] AttributeError: 'NoneType' object has no attribute 'getvalue'

LyzardKing commented 5 years ago

Thanks for the contribution. As you can see the style tests fail. Also, you need to update the test framework as well. in the tests folder there's a copy of the download page of every framework that we use in tests. The change has to be done there as well.

pellcorp commented 5 years ago

It would help if your docs actually explained clearly what is required to setup a dev environment. Nothing I tried worked, very frustrating.

pellcorp commented 5 years ago

Pretty sure this is not supposed to happen:

(env)  ~/Development/Github/ubuntu-make (fix_maven_due_to_sha512) $ ./runtests tests/medium/test_maven.py
[nose.plugins.manager] DEBUG: DefaultPluginManager load plugin nose_json = nose_json.plugin:JsonReportPlugin
[nose.plugins.manager] DEBUG: DefaultPluginManager load plugin cov = nose_cov:Cov
[nose_cov] DEBUG: nose-cov options
[nose_cov] DEBUG: nose-cov configure
E
======================================================================
ERROR: Install Maven from scratch test case
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jason/Development/Github/ubuntu-make/tests/medium/test_maven.py", line 34, in setUp
    super().setUp()
  File "/home/jason/Development/Github/ubuntu-make/tests/medium/__init__.py", line 102, in setUp
    self.container_id = subprocess.check_output(command).decode("utf-8").strip()
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 403, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.6/subprocess.py", line 709, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.6/subprocess.py", line 1258, in _execute_child
    executable = os.fsencode(executable)
  File "/home/jason/Development/Github/ubuntu-make/env/lib/python3.6/os.py", line 800, in fsencode
    filename = fspath(filename)  # Does type-checking of `filename`.
TypeError: expected str, bytes or os.PathLike object, not NoneType

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (errors=1)
pellcorp commented 5 years ago

I tried this on master too, so its not my changes. I am on ubuntu 18.04, 

(env)  ~/Development/Github/ubuntu-make (master) $ ./runtests tests/medium/test_maven.py
[nose.plugins.manager] DEBUG: DefaultPluginManager load plugin nose_json = nose_json.plugin:JsonReportPlugin
[nose.plugins.manager] DEBUG: DefaultPluginManager load plugin cov = nose_cov:Cov
[nose_cov] DEBUG: nose-cov options
[nose_cov] DEBUG: nose-cov configure
E
======================================================================
ERROR: Install Maven from scratch test case
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jason/Development/Github/ubuntu-make/tests/medium/test_maven.py", line 34, in setUp
    super().setUp()
  File "/home/jason/Development/Github/ubuntu-make/tests/medium/__init__.py", line 102, in setUp
    self.container_id = subprocess.check_output(command).decode("utf-8").strip()
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 403, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.6/subprocess.py", line 709, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.6/subprocess.py", line 1258, in _execute_child
    executable = os.fsencode(executable)
  File "/home/jason/Development/Github/ubuntu-make/env/lib/python3.6/os.py", line 800, in fsencode
    filename = fspath(filename)  # Does type-checking of `filename`.
TypeError: expected str, bytes or os.PathLike object, not NoneType

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (errors=1)
pellcorp commented 5 years ago

I would like to understand why my dev env failed, here is the log of setting it up according to your wiki page:

 ~/Development/Github/ubuntu-make (fix_maven_due_to_sha512) $ env/bin/pip install -r requirements.txt
Obtaining apt from git+git://git.launchpad.net/python-apt@1.1.0_beta1#egg=apt (from -r requirements.txt (line 2))
  Cloning git://git.launchpad.net/python-apt (to revision 1.1.0_beta1) to ./env/src/apt
  Running setup.py (path:/home/jason/Development/Github/ubuntu-make/env/src/apt/setup.py) egg_info for package apt produced metadata for project name python-apt. Fix your #egg=apt fragments.
Requirement already satisfied: argcomplete in /usr/lib/python3/dist-packages (from -r requirements.txt (line 3)) (1.8.1)
Collecting progressbar33 (from -r requirements.txt (line 4))
Requirement already satisfied: pyyaml in /usr/lib/python3/dist-packages (from -r requirements.txt (line 5)) (3.12)
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (from -r requirements.txt (line 6)) (2.18.4)
Requirement already satisfied: pyxdg in /usr/lib/python3/dist-packages (from -r requirements.txt (line 7)) (0.25)
Requirement already satisfied: python-gnupg in /usr/lib/python3/dist-packages (from -r requirements.txt (line 8)) (0.4.1)
Requirement already satisfied: nose in /home/jason/.local/lib/python3.6/site-packages (from -r requirements.txt (line 11)) (1.3.7)
Collecting nose-cov (from -r requirements.txt (line 12))
Collecting nose-json (from -r requirements.txt (line 13))
Requirement already satisfied: pep8 in /home/jason/.local/lib/python3.6/site-packages (from -r requirements.txt (line 14)) (1.7.1)
Requirement already satisfied: pexpect>=4.0 in /usr/lib/python3/dist-packages (from -r requirements.txt (line 15)) (4.2.1)
Collecting cov-core>=1.6 (from nose-cov->-r requirements.txt (line 12))
Requirement already satisfied: simplejson in /usr/lib/python3/dist-packages (from nose-json->-r requirements.txt (line 13)) (3.13.2)
Collecting coverage>=3.6 (from cov-core>=1.6->nose-cov->-r requirements.txt (line 12))
  Using cached https://files.pythonhosted.org/packages/a4/5c/b3eac6aa7eb4af83c8580d9c1578343910bdcc59e64e3565402f9a527f9c/coverage-4.5.2-cp36-cp36m-manylinux1_x86_64.whl
Installing collected packages: python-apt, progressbar33, coverage, cov-core, nose-cov, nose-json
  Found existing installation: python-apt 1.6.3
    Not uninstalling python-apt at /usr/lib/python3/dist-packages, outside environment /home/jason/Development/Github/ubuntu-make/env
    Can't uninstall 'python-apt'. No files were found to uninstall.
  Running setup.py develop for python-apt
Successfully installed cov-core-1.15.0 coverage-4.5.2 nose-cov-1.6 nose-json-0.2.4 progressbar33-2.4 python-apt
 ~/Development/Github/ubuntu-make (fix_maven_due_to_sha512) $ source env/bin/activate
(env)  ~/Development/Github/ubuntu-make (fix_maven_due_to_sha512) $ ./runtests medium
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
======================================================================
ERROR: Install android ndk from scratch test case
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jason/Development/Github/ubuntu-make/tests/medium/test_android.py", line 69, in setUp
    super().setUp()
  File "/home/jason/Development/Github/ubuntu-make/tests/medium/__init__.py", line 102, in setUp
    self.container_id = subprocess.check_output(command).decode("utf-8").strip()
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 403, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.6/subprocess.py", line 709, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.6/subprocess.py", line 1258, in _execute_child
    executable = os.fsencode(executable)
  File "/home/jason/Development/Github/ubuntu-make/env/lib/python3.6/os.py", line 800, in fsencode
    filename = fspath(filename)  # Does type-checking of `filename`.
TypeError: expected str, bytes or os.PathLike object, not NoneType
pellcorp commented 5 years ago

sorry about the bold text, I promise I am not shouting on purpose, weird.

LyzardKing commented 5 years ago

The only thing I can think now is if you have docker installed.. We use that for the medium tests..

LyzardKing commented 5 years ago

Don't worry about the bold text...I believe it's some markdown formatting. If you need to post things like this you're better off linking to a pastebin, or using the insert code (or quote) in the comment. Unfortunately I can't do much testing now, but tomorrow I might be able to work on this. Anyway, as far as I can tell you changed the right thing...

LyzardKing commented 5 years ago

Ok, in docker the tests work. Also the large tests (a real full install) works properly. Merging. Thanks!