rwnx / pynonymizer

A universal tool for translating sensitive production database dumps into anonymized copies.
https://pypi.org/project/pynonymizer/
MIT License
101 stars 38 forks source link

Broken Pipeline on project #123

Closed rwnx closed 1 year ago

rwnx commented 1 year ago

Describe the bug scheduled pipelines are failing due to a mssql install error during the mssql integration test

https://github.com/rwnx/pynonymizer/actions/runs/3667058815

We need to fix this. I've done some investigation and I think this is because the version of ubuntu we're using "ubuntu-latest" from github actions is not supported by the install/ apt sources we're using. We should lock it to a previous version (presumably the previous LTS? for now)

rwnx commented 1 year ago
2023-03-26T02:35:21.9657157Z ##[group]Run actions/setup-python@v2
2023-03-26T02:35:21.9657417Z with:
2023-03-26T02:35:21.9657609Z   python-version: 3.8
2023-03-26T02:35:21.9658028Z   token: ***
2023-03-26T02:35:21.9658219Z env:
2023-03-26T02:35:21.9658412Z   MSSQL_SA_PASSWORD: pASSw0rd
2023-03-26T02:35:21.9658653Z   PYNONYMIZER_DB_USER: sa
2023-03-26T02:35:21.9658913Z   PYNONYMIZER_DB_PASSWORD: pASSw0rd
2023-03-26T02:35:21.9659128Z ##[endgroup]
2023-03-26T02:35:22.1208892Z Successfully setup CPython (3.8.16)
2023-03-26T02:35:22.1348852Z ##[group]Run bash tests_integration/mssql/install_mssql.sh
2023-03-26T02:35:22.1349250Z bash tests_integration/mssql/install_mssql.sh
2023-03-26T02:35:22.1406500Z shell: /usr/bin/bash -e {0}
2023-03-26T02:35:22.1406740Z env:
2023-03-26T02:35:22.1406965Z   MSSQL_SA_PASSWORD: pASSw0rd
2023-03-26T02:35:22.1407207Z   PYNONYMIZER_DB_USER: sa
2023-03-26T02:35:22.1407454Z   PYNONYMIZER_DB_PASSWORD: pASSw0rd
2023-03-26T02:35:22.1407746Z   pythonLocation: /opt/hostedtoolcache/Python/3.8.16/x64
2023-03-26T02:35:22.1408074Z   LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.8.16/x64/lib
2023-03-26T02:35:22.1408341Z ##[endgroup]
2023-03-26T02:35:22.1589760Z Adding Microsoft repositories...
2023-03-26T02:35:22.1738824Z   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
2023-03-26T02:35:22.1743883Z                                  Dload  Upload   Total   Spent    Left  Speed
2023-03-26T02:35:22.1744304Z 
2023-03-26T02:35:22.2417245Z   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
2023-03-26T02:35:22.2731126Z 
2023-03-26T02:35:22.2734355Z 100   983  100   983    0     0   9963      0 --:--:-- --:--:-- --:--:-- 10030
2023-03-26T02:35:22.4536243Z OK
2023-03-26T02:35:22.4694274Z   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
2023-03-26T02:35:22.4694958Z                                  Dload  Upload   Total   Spent    Left  Speed
2023-03-26T02:35:22.4695756Z 
2023-03-26T02:35:22.6996461Z   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
2023-03-26T02:35:22.7008872Z 100    14  100    14    0     0     60      0 --:--:-- --:--:-- --:--:--    60
2023-03-26T02:35:23.8720556Z Unable to handle repository shortcut '404: Not Found'
2023-03-26T02:35:23.8970903Z ##[error]Process completed with exit code 1.
rwnx commented 1 year ago

Looks like microsoft changed the format after 18.04:

https://packages.microsoft.com/config/ubuntu/

18.04: Index of /config/ubuntu/18.04/

../ FILE_MANIFEST 03-Nov-2022 01:15 insiders-fast.list 24-Oct-2022 22:50 insiders-slow.list 24-Oct-2022 22:47 mssql-server-2017.list 24-Oct-2022 22:47 mssql-server-2019-gdr.list 24-Oct-2022 22:46 mssql-server-2019.list 24-Oct-2022 22:44 mssql-server-preview.list 24-Oct-2022 22:50 multiarch/ 02-Nov-2022 23:21 packages-microsoft-prod.deb 25-Oct-2022 17:39 prod.list 24-Oct-2022 22:49

18.10(and beyond) ../ FILE_MANIFEST 02-Nov-2022 23:46 insiders-fast.list 24-Oct-2022 22:53 insiders-slow.list 24-Oct-2022 22:52 multiarch/ 02-Nov-2022 23:21 packages-microsoft-prod.deb 25-Oct-2022 17:40 prod.list

rwnx commented 1 year ago

I've locked this to ubuntu 20.04 and server 2019 (same as before)

I looked into upgrading it to 22.04 but the only mssql version available for that was 2022, and it looks like that's not supported. https://stackoverflow.com/questions/72428019/how-to-install-mssql-server-on-ubuntu-22-04