Add Continuous Integration (CI) & testing tools (Travis/ pytest/ actions/...)

[pz-max]

Could be set up for .py pushes to 'main', which should be operatable with the 'pypsa-africa' environment

[koen-vg]

I will look into cleaning up the workflow to the point where we can set up continuous integration.

[davide-f]

It seems that with the last commit f5d34f4199daff305f9bebd118a11592ddfd37bf, now CI works 🚀 🤩 at least on ubuntu... Last check on windows gave a super weird error related to corrupted files in miniconda... yet not dependent on our code.

[pz-max]

Hi @davide-f, It still seems not to have never worked for ubuntu? What is weird is that if I copy the CI config.yaml (mv test/config.test1.yaml config.yaml) and let the whole thing run with the pypsa-africa environment, everything works. There is just one error at the end caused by glpk.

However, when checking the CI feedback. The whole thing crashes earlier... Is it because the CI uses mamba. Any thoughts @euronion ?

[davide-f]

@pz-max it worked once yesterday night on ubuntu only, then I rerun the tests because in windows failed for a very weird reason and then they failed everywhere. The weird parts are that when I test locally, everything works, yet online they don't. I am not sure if they download the same stuff also, because in CI a lot of empty dataframe appears by the log, yet locally they don't.

[pz-max]

Ah, now I understand. Probably good to check the test/config.test1.yaml. I am also checking now how everything works with mamba instead of conda (mamba is used in the CI)

Get Outlook for iOShttps://aka.ms/o0ukef

---

From: Davide Fioriti @.> Sent: Friday, November 26, 2021 11:30:22 AM To: pypsa-meets-africa/pypsa-africa @.> Cc: PARZEN Maximilian @.>; Mention @.> Subject: Re: [pypsa-meets-africa/pypsa-africa] Add Continuous Integration (CI) & testing tools (Travis/ pytest/ actions/...) (#3)

This email was sent to you by someone outside the University. You should only click on links or attachments if you are certain that the email is genuine and the content is safe.

@pz-maxhttps://github.com/pz-max it worked once yesterday night on ubuntu only, then I rerun the tests because in windows failed for a very weird reason and then they failed everywhere. The weird parts are that when I test locally, everything works, yet online they don't. I am not sure if they download the same stuff also, because in CI a lot of empty dataframe appears by the log, yet locally they don't.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/pypsa-meets-africa/pypsa-africa/issues/3#issuecomment-979863504, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AOYZENMFF7MB3V2V3TDJBFDUN5OT5ANCNFSM4ZU46TKQ.

The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. Is e buidheann carthannais a th’ ann an Oilthigh Dhùn Èideann, clàraichte an Alba, àireamh clàraidh SC005336.

[davide-f]

The build_cutouts are not well generated because it cannot reach the server; it may be a simple setting issue probably https://github.com/pypsa-meets-africa/pypsa-africa/runs/4332790816?check_suite_focus=true

[davide-f]

The tutorial I abilited yesterday was to solve that problem I forgot to mention; when I investigated better the ci.yaml I saw the settings to download the data and that's why I mentioned a review in the commit.

[euronion]

However, when checking the CI feedback. The whole thing crashes earlier... Is it because the CI uses mamba. Any thoughts @euronion ?

Nah, seems like you have a few bugs in the workflow that need ironing out ;-)

[pz-max]

Let’s talk in discord

Get Outlook for iOShttps://aka.ms/o0ukef

---

From: Davide Fioriti @.> Sent: Friday, November 26, 2021 11:45:00 AM To: pypsa-meets-africa/pypsa-africa @.> Cc: PARZEN Maximilian @.>; Mention @.> Subject: Re: [pypsa-meets-africa/pypsa-africa] Add Continuous Integration (CI) & testing tools (Travis/ pytest/ actions/...) (#3)

This email was sent to you by someone outside the University. You should only click on links or attachments if you are certain that the email is genuine and the content is safe.

The tutorial I abilited yesterday was to solve that problem I forgot to mention; when I investigated better the ci.yaml I saw the settings to download the data and that's why I mentioned a review in the commit. It was needed...

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/pypsa-meets-africa/pypsa-africa/issues/3#issuecomment-979872826, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AOYZENMCO5CHGJLIY5ANE3DUN5QKZANCNFSM4ZU46TKQ.

The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. Is e buidheann carthannais a th’ ann an Oilthigh Dhùn Èideann, clàraichte an Alba, àireamh clàraidh SC005336.

[davide-f]

In our last run, ubuntu and mac were successful; on windows there is a super weird error due to an apparently corrupted library of miniconda: https://github.com/pypsa-meets-africa/pypsa-africa/runs/4333471322?check_suite_focus=true

Error occurred during initialization of VM
Corrupted ZIP library: C:\Miniconda\envs\test\Library\bin\zip.dll

It is most likely related to an incompatibility between some packages on windows on the VM machine. Many reported similar issues when using different packages: see https://issueexplorer.com/issue/kivy/pyjnius/610 The big question is which ones are in conflict...

@euronion @pz-max On my windows computer, the above does not generate any problem. It may be a problem of the VM machine as well. We may remove windows from the tested platforms for the time being, yet, we shall ask the community how we can solve that problem. Any ideas?

We may post a bug report on github or alike: https://support.github.com/contact/bug-report

[davide-f]

163 + subsequent commits till da25b08288dd1a657e480bc74ed82de7b5818796 solve this issue.

The weird issue occurring only in the CI on windows is still present, but I believe it is not 100% workflow-related. Obviously, we will work until it works also on windows, but I think we can close this issue; let's open it again if needed.