Running on Github Actions

[myselfhimself]

Hello, I am planning to try to use your work for https://github.com/myselfhimself/gmic-blender though I will not use Travis CI but the Github Actions instead. I was wondering how to enable an add-on from outside Blender programmatically. It seems your SetupPlugin() class does the work... I will see in <1 month. I do plan to use pytest as well. Thanks for contributing your work online. Let us keep in touch.

[douglaskastle]

First off thanks for the nice comments about the addon, it is great to see people use something that I thought was missing.

I also have been looking at G'MIC myself recently, mostly over on shotcut, but hoping to see support on blender improve, so I will be watching your work with some interest.

I personally prefer Travis, it has been around for a while and seems to work well. However on another project I did try and see if,in addition to Travis if I could also setup github actions and CircleCI, which I did. I am still not impressed enough to move off Travis for my main work. However this project is more designed as an enabling project, so in that respect it makes sense to add other CIs if that is what is required for other people to use it.

So, I spent a little time and have added some initial github actions support. I just merged into master this file.github/workflows/pythonpackage.yml, it sets up a matrix of blender versions and runs the test set to run via pytest.

Note at the time of this writing there is some download issue with the nightly builds, which is following on from a DDOS attack on blender.org yesterday. So while they are failing here, they are also failing on Travis. Give it a few days.

what is not done, which I would like to do is cache the versions of blender to save the download hit on the server side.

As to the blender/pytest integration, it took a while to figure out, but please refer to this post to understand how we got to where we got to:

https://blenderartists.org/t/add-on-pytest-unittest-and-travisci-integration-example/1140838 https://devtalk.blender.org/t/add-on-pytest-unittest-and-travisci-integration-example/4769

I'll leave CircleCI support for another day! ;)

[douglaskastle]

OK that nightly failure seems to have been a change in the naming conventions on the server. Either because of the DDOS attack or a coincidence.

Also needed to bulk up the apt install of blender (required so all the dependent libraries that blender needs is installed), as it was failing from time to time.

Please find a passing github action build from 2.78 to 2.83, checked into master.

https://github.com/douglaskastle/blender-fake-addon/actions/runs/33764008

I also added the passing badge to the README.

yay!

[myselfhimself]

Hello Douglas, thank you, you have been very fast sparing me hours of work digging into your system! Thanks for your interest for G'Mic inside Blender!

gmic-py - the gmic CPython binding compiles c/c++11 for 32/64bit x linux/macosx/(soon windows) It will be solid with numpy arrays i/o support in march/april 2020

gmic-blender - the future gmic-py-based 500+ G'Mic filters to 500+ Blender nodes addon, will have the same architecture(bits) x OS variability for testing, in addition to the Blender 2.7-2.8 versions interval which you have setup. The ETA for a few dozen nodes is May 2020. We will skip Python 2.x support, hence Blender 2.7x if I am not wrong - because of the CPython handmade binding's tightness to the CPython 3 API.

I will be back from vacation from next friday afternoon sorry. You are awesome Douglas, thanks again!!! Le 1 févr. 2020 22:03, "Douglas Kastle" notifications@github.com a écrit :

OK that nightly failure seems to have been a change in the naming conventions on the server. Either because of the DDOS attack or a coincidence.

Also needed to bulk up the apt install of blender (required so all the dependent libraries that blender needs is installed), as it was failing from time to time.

Please find a passing github action build from 2.78 to 2.83, checked into master.

https://github.com/douglaskastle/blender-fake-addon/actions/runs/33764008

I also added the passing badge to the README.

yay!

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/douglaskastle/blender-fake-addon/issues/4?email_source=notifications&email_token=AAJU5QRT436JJ3EVXNAEAMDRAXPSZA5CNFSM4KOH3TD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKRGR6A#issuecomment-581069048, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJU5QQ2SI3VJAEZL4Y4LCTRAXPSZANCNFSM4KOH3TDQ .

[myselfhimself]

I will test next week !

[douglaskastle]

some notes

Blender 2.79 is actually python3.5, blender has been on python3 for a while Blender 2.80 is python3.7

setting up nightly and weekly schedules under github actions doesn't work the way I would like. In the case of blender addon , in particular the nightly builds, you want to make sure that the changing blender releases don't end up breaking code that you had previously running and passing. This is my main aim with testing, your mileage may vary.

https://github.community/t5/GitHub-Actions/on-schedule-per-branch/m-p/45108#M6063

also just this week i added code coverage support. Which can be run locally to check you testing locally. But I also integrated it against TravisCI and codecov.io, i haven't looked at integrating it into github actions, i think codecov will only allow one.

github actions operation under windows works, which i can't get working on travisCI

[myselfhimself]

Hello @douglaskastle, I am busy finishing a numpy support for a Python binding. I will test next week. But it would highly useful and could the backbone of my Blender plugin's stability.

[myselfhimself]

Thank you very very much for your time spent opening this Github Actions possibility for many people!!

[myselfhimself]

Hello, would you mind showing how to use your project as python library for example ?

The github actions script as is needs to have the scripts/ dir there... It assumes that we are in the blender-fake-addon project's files.

An idea would be to use pip install -e <the github url>

[myselfhimself]

Here is a failing scenario illustrating my suggestion:

$ pip install -e git://github.com/douglaskastle/blender-fake-addon.git#egg=blender-fake-addon
Obtaining blender-fake-addon from git+git://github.com/douglaskastle/blender-fake-addon.git#egg=blender-fake-addon
  Cloning git://github.com/douglaskastle/blender-fake-addon.git to /export/home/AAA/.virtualenvs/blender-fake-addon-fiddling/src/blender-fake-addon
  Running command git clone -q git://github.com/douglaskastle/blender-fake-addon.git /export/home/AAA/.virtualenvs/blender-fake-addon-fiddling/src/blender-fake-addon
    ERROR: Command errored out with exit status 1:
     command: /export/home/AAA/.virtualenvs/blender-fake-addon-fiddling/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/export/home/AAA/.virtualenvs/blender-fake-addon-fiddling/src/blender-fake-addon/setup.py'"'"'; __file__='"'"'/export/home/AAA/.virtualenvs/blender-fake-addon-fiddling/src/blender-fake-addon/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info
         cwd: /export/home/AAA/.virtualenvs/blender-fake-addon-fiddling/src/blender-fake-addon/
    Complete output (5 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/export/home/AAA/.virtualenvs/blender-fake-addon-fiddling/lib/python3.6/tokenize.py", line 452, in open
        buffer = _builtin_open(filename, 'rb')
    FileNotFoundError: [Errno 2] No such file or directory: '/export/home/AAA/.virtualenvs/blender-fake-addon-fiddling/src/blender-fake-addon/setup.py'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

[myselfhimself]

It seems the run_blender / load_pytest scripts break python's modules discovery paths to a point where the installed pytest package cannot be found anymore: https://github.com/myselfhimself/gmic-blender/runs/474626916?check_suite_focus=true https://github.com/myselfhimself/gmic-blender/blob/f32c886ef8e696e18f01ff353c423a25729b8022/.github/workflows/blender-fake-addon-test.yml

[myselfhimself]

Eventually, I made the fake_addon addon test pass for blender 2.8 x linux64 with this Github Actions recipe: https://github.com/myselfhimself/gmic-blender/runs/474671492?check_suite_focus=true https://github.com/myselfhimself/gmic-blender/blob/6c9f7ea20d6c23d164d9a375122fdd9ca4b820ae/.github/workflows/blender-fake-addon-test.yml I am continuing work, now using my own little test suite (to be written), on gmic-blender's Issue Tracker.

[myselfhimself]

I would rather have some pip-installable library or executable from your project, runnable as: blender-addon-tester <blender_version> <addon_dir_or_zip_path> [test_suite_path_if_not_tests/] (the OS and architecture would be assumed from the current run matrix docker environment)

or as

import blender_addon_tester as bat
bat.test_addon(blender_version="2.80", addon="myaddon", test_path="tests/")

For the beginning, I would not require you module to be available on pypi.org but pip installable by Git-URL or another pip-installable CLI command form

Also the Blender install path would preferably be an OS temporary folder as provided by python's mktemp, not "../" if I am not mistaken.

For the rest, thanks for publishing already stuff on the internet! Let us see how I manage to write a little pytest test suite..

[douglaskastle]

As you have discovered, I think, you can not pip install an addon into blender it needs to go in via the blender install path.

As to the test harness, the current flow works quiet well for me, use it for a bit, it is not the part of the process that is the problem any more.

I take your point though that it could be contained in a module that could be managed under pip, I sadly ran out of bandwidth last week (newborn) I won't be able to look at this for a while. So you can proceed with what's there and/or investigate how to get it done and send me back a pull request.

[myselfhimself]

Ok perfect, I will do a PR if I have enough time in the coming weeks. Congratulations for the new little one! Enjoy and please dedicate more time to your family than this project :) they deserve you :)

[myselfhimself]

some notes

Blender 2.79 is actually python3.5, blender has been on python3 for a while Blender 2.80 is python3.7

setting up nightly and weekly schedules under github actions doesn't work the way I would like. In the case of blender addon , in particular the nightly builds, you want to make sure that the changing blender releases don't end up breaking code that you had previously running and passing. This is my main aim with testing, your mileage may vary.

https://github.community/t5/GitHub-Actions/on-schedule-per-branch/m-p/45108#M6063

also just this week i added code coverage support. Which can be run locally to check you testing locally. But I also integrated it against TravisCI and codecov.io, i haven't looked at integrating it into github actions, i think codecov will only allow one.

github actions operation under windows works, which i can't get working on travisCI

OK just read!! I am not yet worried about Blender nightly builds and code coverage. Github Actions has MacOSX+Windows in addition to Linux. This is super helpful indeed.

[myselfhimself]

Opening a PR related to this: https://github.com/douglaskastle/blender-fake-addon/pull/5 I will ping back when this seems mergeable.. After then, you might want to rename repository to something like blender-addon-tester.

[douglaskastle]

@myselfhimself watch out for download the version of blender locally. I dunt want to have to do that every time I do a test. If it is run the first time, fetch it of course, but my way has it persistent from run to run and also between addons.

In case you were wondering this is where I have been using my work in anger

https://github.com/douglaskastle/blender-import-lwo

[douglaskastle]

Also running against nightly builds is recommended when the code is mature, this is where you catch blender breaking your code early.

Coverage is handy to see if your testing is comprehensive especially if you are writing tests for code you didn't write. Coverage can also be run locally. I would at least keep the hooks in.

Not here, but I have also successfully gotten the debugger and profiler working, handy for noddy or top heavy code.

[myselfhimself]

Hello, I want all your features to fit in the fork and wish that it will land after some time back in your project. Thanks for details on the nightly builds usefulness. Nice project that you have with Lightwave, for sure continuous testing on various blender versions and OSes can make your add-on crazy solid! Let us hope the same for blender-gmic (still in its infancy). I am bit lost, but understood and agreed for caching the downloaded blender versions on the Github Actions side.. I just will prioritize making any test suite runnable in the fork, and then will refocus on this (or on a numpy support for gmic-py first). Have a nice week-end

[douglaskastle]

Just a quick note, my flow is designed to run locally as well as remote. On my host system, my own computer, I don't want to be downloading blender everytime I want to run a test. Also if I get a fail on the remote system I want to be able to replicate as much as I can locally debug via CI, someone said, is a time vampire.

On Travis I am trying to be a good citizen and keep downloaded copies around, 2.78c for example hasn't changed in 2 years.

I don't know how to do caching in github actions so it falls back to download everything every time. Not too nice for the blender.org servers

[myselfhimself]

Hello, I have spotted the cache environment variables that you use. I miss time tonight to have it working both locally and with github actions (possibly with github actions' official caching step). FYI, I now have managed a POC where your fake_addon is tested after a pip install and simple function call. See related Actions runs here: https://github.com/myselfhimself/blender-fake-addon/actions See how your fake_addon is now used in the PR: a tests/ dir, a requirements.txt importing the blender-addon-tester package, the fake_addon directory itself: https://github.com/myselfhimself/blender-fake-addon/tree/master/examples/testing-fake-addon

Maybe let us stabilize the much-needed caching, test the project also on blender-gmic, and we could merge the PR back with your required picky changes :)

[douglaskastle]

@myselfhimself please see the changes have have made to the branch, it now has travis support working-ish.

https://travis-ci.org/douglaskastle/blender-fake-addon/builds/656481878

I am not happy with the requirments.txt being this:

-e git://github.com/myselfhimself/blender-fake-addon.git#egg=blender_addon_tester As it is explicit, in this case to the master of your repo, which means when I am on the branch of my repo it still is picking it up, and my edits can't be picked up to be checked. I get that it is a bootstrap exercise, which is cool, but is there a better way of doing this so that it can track with what ever branch it is in. Maybe it should not be used here, I don't know, I am struggling to find a better fix. This will be a problem if any one is working between branches on their own repo.

[douglaskastle]

A note I would like to make here is that I find I have to hack load_pytest.py a lot. When tests are written it is fine as is, but during development, you might only want to run one test, or a sub suite.

Having it locked into the module makes that a lot harder.

I don't have a solution, but I wanted to mention it in case you have any ideas. Again look at the lwo import example to see it being used in an enhanced and non standardisable way.

[myselfhimself]

Hello to run a subtest, as in this pytest documentation chapter, you can use pytest -k <some regex pattern or glob to filter out the test(s) which you want to run exclusively>. This pattern could be injectable from the new module-level call: BEFORE def test_blender_addon(blender_version="2.80", addon_name="fake_addon", pytests_path="tests/"): AFTER (design idea) def test_blender_addon(blender_version="2.80", addon_name="fake_addon", test_expression_filter="", test_ pytests_path="tests/"):

@douglaskastle Would you like me to implement this tiny refactoring ?

also, we are very much pytest oriented, while some people may want to use unitttest or some other lib instead.. A refactoring to be agnostic in terms of testing library could be done in the more distant future.

[myselfhimself]

@douglaskastle regarding PRs: OK to reopen a different PR targetting this other non-master branch of yours. Regarding the requirements "-e" call consuming a github URL, the pip documentation allows for consuming a Git branch, so I will target that very branch you mentioned.

As far as this github URL consumption is concerned, in the long term, I would rather have a pip install blender-addon-tester, when the project is stabilized enough to be published as python module on pypi.org. Such an upload happens with the twine CLI command, I can set it up with github actions so that it is on any release push of your repository. Creating write tokens on pypi.org would also be needed. Let us shift this idea to a bit later, when stability is met.

[myselfhimself]

Also running against nightly builds is recommended when the code is mature, this is where you catch blender breaking your code early.

Coverage is handy to see if your testing is comprehensive especially if you are writing tests for code you didn't write. Coverage can also be run locally. I would at least keep the hooks in.

Not here, but I have also successfully gotten the debugger and profiler working, handy for noddy or top heavy code.

Nice !! Github Actions allows for attaching artifacts (ie. result files) to job runs. So I would like to put coverage or other files as artifacts in the coming day. Any job should be openable by click and its artifacts shall be downloadable.

[myselfhimself]

@douglaskastle regarding PRs: OK to reopen a different PR targetting this other non-master branch of yours. Regarding the requirements "-e" call consuming a github URL, the pip documentation allows for consuming a Git branch, so I will target that very branch you mentioned.

As far as this github URL consumption is concerned, in the long term, I would rather have a pip install blender-addon-tester, when the project is stabilized enough to be published as python module on pypi.org. Such an upload happens with the twine CLI command, I can set it up with github actions so that it is on any release push of your repository. Creating write tokens on pypi.org would also be needed. Let us shift this idea to a bit later, when stability is met.

Branch granularity added in new PR

[myselfhimself]

Some progress in the PR: The signature of a call is now as shown here: https://github.com/myselfhimself/blender-addon-tester/blob/blender-addon-tester/examples/testing-fake-addon/test_fake_addon_blender_advanced.py

BAT.test_blender_addon(addon_path=addon, blender_revision=blender_rev, config=config)

The config variable is optional and is an defaults-overriding dictionary :

• the builtin load_pytests.py script that gets run within Blender in order to load and run tests is now overridable by providing its path in config["blender_load_tests_script"]. The builtin script reads more environment variables that can be set before it is run, in order to configure itself.
• coverage reporting is disabled by default, but enablable through a config["coverage"] = True
• the tests path can be also overriden using config["tests"] otherwise CWD's "tests/" is used.

This is in use also in gmic-blenderhttps://github.com/myselfhimself/gmic-blender/blob/master/test_gmic_blender_addon.py coupled with this Github Actions job and an overriding load_pytests.py files that prevents zip compression and sets a correct bpy_module. Indeed right now, the blender-addon-tester does not like "addon" path as ".". Zipping explodes the Github Action, possibly because CWD is in a different location than the addon's root dir, and the bpy_module is then not properly set. It is also my bad not making a separate directory named after the Blender add-on's name... I will rearrange my repository for this and pipe something like "gmic-blender" instead of "." for the addon path to fix this.

Have a nice week-end and thanks for your work. I want to add caching and artifacts (coverage report retrieval) in Github Actions next.

[myselfhimself]

(open to critics, requests and patches)

[myselfhimself]

providing a .zip filename as the addon_path parameter has been added:

• This is showcased at this very line: https://github.com/myselfhimself/blender-addon-tester/runs/495843880?check_suite_focus=true#step:4:223

  Info: Modules Installed () from '/home/runner/work/blender-addon-tester/blender-addon-tester/examples/testing-fake-addon/fake_addon.zip' into '/home/runner/.config/blender/2.80/scripts/addons'
  Addon dir is - /home/runner/work/blender-addon-tester/blender-addon-tester/examples/testing-fake-addon/local_addon
  Detected zip path is - /home/runner/work/blender-addon-tester/blender-addon-tester/examples/testing-fake-addon/fake_addon.zip. No need to zip the addon beforehand.
  Change addon dir - local_addon
  reloading addon: fake_addon 1583776060.861803 1583776062.4897943 '/home/runner/.config/blender/2.80/scripts/addons/fake_addon/__init__.py'
  Modules Installed () from '/home/runner/work/blender-addon-tester/blender-addon-tester/examples/testing-fake-addon/fake_addon.zip' into '/home/runner/.config/blender/2.80/scripts/addons'
  Info: Modules Installed () from '/home/runner/work/blender-addon-tester/blender-addon-tester/examples/testing-fake-addon/fake_addon.zip' into '/home/runner/.config/blender/2.80/scripts/addons'
  Hello World
  ============================= test session starts ==============================
  platform linux -- Python 3.7.0, pytest-5.3.5, py-1.8.1, pluggy-0.13.1 -- /home/runner/work/blender-addon-tester/blender-addon-tester/examples/blender-2.80/blender

And also used by gmic-blender in its Github Actions workflow: https://github.com/myselfhimself/gmic-blender/runs/495833482?check_suite_focus=true#step:4:170

Coming up next: artifacts (coverage) and blender releases download caching.

[myselfhimself]

Added coverage.xml as artifact (for the fake_addon testadvanced file) See related Github Action runs:

• from local wheel
• from pip install from a github url And typical Github Action file (look for artifact at the very end)

[myselfhimself]

Tuned the Python code and Github Actions to add cache support within ../ or any path set in the BLENDER_CACHE environment variable. A cache key for the cache path is set and retrieved. It looks like OS-blender-revision (eg. Linux-blender-2.80 or later ubuntu-latest-blender-2.80). Further work could let the architecture appear (ie. 64 or 32bits in the key), but I do not want to develop this for now.

Log excerpts:```

2020-03-12T17:30:33.8249988Z env: 2020-03-12T17:30:33.8250108Z BLENDER_CACHE: /home/runner/work/blender-addon-tester/blender-addon-tester/.blender_releases_cache 2020-03-12T17:30:33.8250237Z pythonLocation: /opt/hostedtoolcache/Python/3.7.6/x64 2020-03-12T17:30:33.8250353Z ##[endgroup] 2020-03-12T17:30:38.9860532Z Cache Size: ~277 MB (290062348 B) 2020-03-12T17:30:38.9942484Z [command]/bin/tar -xz -f /home/runner/work/_temp/fab96288-a871-4b75-8c05-ec5ef7a5f1c6/cache.tgz -C /home/runner/work/blender-addon-tester/blender-addon-tester/.blender_releases_cache 2020-03-12T17:30:46.9283848Z Cache restored from key: Linux-blender-2.80

2020-03-12T17:31:00.5564510Z testing addon_path: fake_addon under blender_revision: 2.80 with config dict: {'blender_load_tests_script': '/home/runner/work/blender-addon-tester/blender-addon-tester/examples/testing-fake-addon/blender_advanced_load_pytest.py', 'coverage': True, 'tests': 'advanced_tests/'} 2020-03-12T17:31:00.5565130Z linux 2020-03-12T17:31:00.5565560Z BLENDER_CACHE environment variable found /home/runner/work/blender-addon-tester/blender-addon-tester/.blender_releases_cache 2020-03-12T17:31:00.5566308Z Cache directory already exists: /home/runner/work/blender-addon-tester/blender-addon-tester/.blender_releases_cache 2020-03-12T17:31:00.5566832Z Blender 2.80 (non-nightly) release found at: /home/runner/work/blender-addon-tester/blender-addon-tester/.blender_releases_cache/blender-2.80 2020-03-12T17:31:00.5567373Z Downloaded Blender is expected in this directory: /home/runner/work/blender-addon-tester/blender-addon-tester/.blender_releases_cache/blender-2.80 2020-03-12T17:31:01.6771065Z Blender 2.80 (sub 75) (hash f6cb5f54494e built 2019-07-29 17:17:04) 2020-03-12T17:31:01.6979378Z ALSA lib confmisc.c:767:(parse_card) cannot find card '0'

Related job link:
https://github.com/myselfhimself/blender-addon-tester/runs/503841012?check_suite_focus=true

[myselfhimself]

Just listed our work on a 2015 Blender Community wishlist issue regarding automated TravisCI addon testing: https://blender.community/c/today/0jbbbc/

I can now either pause (would love to.. in order to work seriously on that G'MIC Blender plugin's node generation) or proceed to making the blender-addon-tester a pypi.org module, as I had done for the G'MIC image processing library, this is not complicated to do using twine and upload tokens.

[douglaskastle]

OK merged your PR for review.

First up, while it looks like you got travis working for all revs, the rev is not being passed down, it is only blender 2.80 being run and passed many times.

I have updated the flow a bit so it is now running against each version of blender.

There is a failure happening with 2,78b/python on windows which I saw creep in earlier, but can't explain.

FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\douglas\\AppData\\Local\\Temp\\pip-4spt9vm2-uninstall\\cygwin64\\home\\douglas\\projects\\blender-fake-addon\\examples\\blender-2.78c-windows64\\2.78\\python\\lib\\site-packages\\pip\\_vendor\\requests\\packages\\urllib3\\packages\\ssl_match_hostname\\__pycache__\\__init__.cpython-35.pyc'
M

I noticed in your github actions you have only enabled 2.80 on ubuntu. While we are in development phase I appreciate not wanting to throw the kitchen sink at every checkin. However the point of this flow is to run against multiple builds and check we are running against multiple builds, so this has been brushed under the carpet. I have added 2.81 to github actions to review if more than one rev is being used.

[douglaskastle]

github actions not working for windows and under 2.78c

https://github.com/douglaskastle/blender-fake-addon/runs/512343461?check_suite_focus=true

WARNING: Requirement 'dist/*.whl' looks like a filename, but the file does not exist
ERROR: *.whl is not a valid wheel filename.

[douglaskastle]

@myselfhimself Oh thank you for all the work so far. I apologise for the silence, I don't have much free time to look at this, but I think it is also important that you get some what prompt feedback on your great work

[myselfhimself]

github actions not working for windows and under 2.78c

https://github.com/douglaskastle/blender-fake-addon/runs/512343461?check_suite_focus=true

WARNING: Requirement 'dist/*.whl' looks like a filename, but the file does not exist
ERROR: *.whl is not a valid wheel filename.

OK will investigate this week

Thank you for the merge and further commits by your team

[myselfhimself]

Windows support was tentatively added and half-working but then removed:

Non working actions run: https://github.com/douglaskastle/blender-fake-addon/runs/512343461?check_suite_focus=true This was a good idea and I see fixing ideas (use python instead of python3...) => I will try to stabilize this

Working actions run but with a strange pip error in the middle: https://github.com/douglaskastle/blender-fake-addon/runs/512343457?check_suite_focus=true

[douglaskastle]

Issue I am seeing with windows 2.78 I am only seeing locally on my home machine. It may work fine on CI windows machines. But for my work, having running locally is as important as running remotely, maybe even more.

[douglaskastle]

Only happens with the python 3.5 version so 2.78 an 2.79. version 2.80 works fine and above, which has python3.7

Exception:
Traceback (most recent call last):
  File "C:\cygwin64\home\douglas\projects\blender-fake-addon\examples\blender-2.79b-windows64\2.79\python\lib\shutil.py", line 544, in move
    os.rename(src, real_dst)
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'c:\\cygwin64\\home\\douglas\\projects\\blender-fake-addon\\examples\\blender-2.79b-windows64\\2.79\\python\\lib\\site-packages\\pip\\_vendor\\requests\\packages\\urllib3\\packages\\ssl_match_hostname\\__pycache__\\__init__.cpython-35.pyc' -> 'C:\\Users\\douglas\\AppData\\Local\\Temp\\pip-e9ejge9c-uninstall\\cygwin64\\home\\douglas\\projects\\blender-fake-addon\\examples\\blender-2.79b-windows64\\2.79\\python\\lib\\site-packages\\pip\\_vendor\\requests\\packages\\urllib3\\packages\\ssl_match_hostname\\__pycache__\\__init__.cpython-35.pyc'

[myselfhimself]

The Windows builds now all succeed on Github Actions on the new Mac OS X / Windows stabilization PR: https://github.com/douglaskastle/blender-fake-addon/pull/8 See I), II).

Details below.

I) pytest / pip resilient modules error

Only happens with the python 3.5 version so 2.78 an 2.79. version 2.80 works fine and above, which has python3.7

Exception:
Traceback (most recent call last):
  File "C:\cygwin64\home\douglas\projects\blender-fake-addon\examples\blender-2.79b-windows64\2.79\python\lib\shutil.py", line 544, in move
    os.rename(src, real_dst)
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'c:\\cygwin64\\home\\douglas\\projects\\blender-fake-addon\\examples\\blender-2.79b-windows64\\2.79\\python\\lib\\site-packages\\pip\\_vendor\\requests\\packages\\urllib3\\packages\\ssl_match_hostname\\__pycache__\\__init__.cpython-35.pyc' -> 'C:\\Users\\douglas\\AppData\\Local\\Temp\\pip-e9ejge9c-uninstall\\cygwin64\\home\\douglas\\projects\\blender-fake-addon\\examples\\blender-2.79b-windows64\\2.79\\python\\lib\\site-packages\\pip\\_vendor\\requests\\packages\\urllib3\\packages\\ssl_match_hostname\\__pycache__\\__init__.cpython-35.pyc'

Details of those (actually 2) errors, from this raw Github Actions log today.

2020-03-19T16:25:37.2354580Z   Found existing installation: pip 9.0.1
2020-03-19T16:25:37.2994143Z     Uninstalling pip-9.0.1:
2020-03-19T16:25:38.1255387Z Exception:
2020-03-19T16:25:38.1255904Z Traceback (most recent call last):
2020-03-19T16:25:38.1257603Z   File "d:\a\blender-addon-tester\blender-addon-tester\.blender_releases_cache\blender-2.79b-windows64\2.79\python\lib\shutil.py", line 544, in move
2020-03-19T16:25:38.1258356Z     os.rename(src, real_dst)
2020-03-19T16:25:38.1258812Z FileNotFoundError: [WinError 3] The system cannot find the path specified: 'd:\\a\\blender-addon-tester\\blender-addon-tester\\.blender_releases_cache\\blender-2.79b-windows64\\2.79\\python\\lib\\site-packages\\pip\\_vendor\\html5lib\\filters\\__pycache__\\alphabeticalattributes.cpython-35.pyc' -> 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\pip-yrkewq58-uninstall\\a\\blender-addon-tester\\blender-addon-tester\\.blender_releases_cache\\blender-2.79b-windows64\\2.79\\python\\lib\\site-packages\\pip\\_vendor\\html5lib\\filters\\__pycache__\\alphabeticalattributes.cpython-35.pyc'
2020-03-19T16:25:38.1259098Z 
2020-03-19T16:25:38.1259328Z During handling of the above exception, another exception occurred:
2020-03-19T16:25:38.1259515Z 
2020-03-19T16:25:38.1259728Z Traceback (most recent call last):
2020-03-19T16:25:38.1260083Z   File "d:\a\blender-addon-tester\blender-addon-tester\.blender_releases_cache\blender-2.79b-windows64\2.79\python\lib\site-packages\pip\basecommand.py", line 215, in main
2020-03-19T16:25:38.1260368Z     status = self.run(options, args)
2020-03-19T16:25:38.1260682Z   File "d:\a\blender-addon-tester\blender-addon-tester\.blender_releases_cache\blender-2.79b-windows64\2.79\python\lib\site-packages\pip\commands\install.py", line 342, in run
2020-03-19T16:25:38.1260953Z     prefix=options.prefix_path,
2020-03-19T16:25:38.1261267Z   File "d:\a\blender-addon-tester\blender-addon-tester\.blender_releases_cache\blender-2.79b-windows64\2.79\python\lib\site-packages\pip\req\req_set.py", line 778, in install
2020-03-19T16:25:38.1261562Z     requirement.uninstall(auto_confirm=True)
2020-03-19T16:25:38.1261884Z   File "d:\a\blender-addon-tester\blender-addon-tester\.blender_releases_cache\blender-2.79b-windows64\2.79\python\lib\site-packages\pip\req\req_install.py", line 754, in uninstall
2020-03-19T16:25:38.1262160Z     paths_to_remove.remove(auto_confirm)
2020-03-19T16:25:38.1262577Z   File "d:\a\blender-addon-tester\blender-addon-tester\.blender_releases_cache\blender-2.79b-windows64\2.79\python\lib\site-packages\pip\req\req_uninstall.py", line 115, in remove
2020-03-19T16:25:38.1262862Z     renames(path, new_path)
2020-03-19T16:25:38.1263169Z   File "d:\a\blender-addon-tester\blender-addon-tester\.blender_releases_cache\blender-2.79b-windows64\2.79\python\lib\site-packages\pip\utils\__init__.py", line 267, in renames
2020-03-19T16:25:38.1263431Z     shutil.move(old, new)
2020-03-19T16:25:38.1263861Z   File "d:\a\blender-addon-tester\blender-addon-tester\.blender_releases_cache\blender-2.79b-windows64\2.79\python\lib\shutil.py", line 558, in move
2020-03-19T16:25:38.1264102Z     copy_function(src, real_dst)
2020-03-19T16:25:38.1264402Z   File "d:\a\blender-addon-tester\blender-addon-tester\.blender_releases_cache\blender-2.79b-windows64\2.79\python\lib\shutil.py", line 257, in copy2
2020-03-19T16:25:38.1264650Z     copyfile(src, dst, follow_symlinks=follow_symlinks)
2020-03-19T16:25:38.1264947Z   File "d:\a\blender-addon-tester\blender-addon-tester\.blender_releases_cache\blender-2.79b-windows64\2.79\python\lib\shutil.py", line 121, in copyfile
2020-03-19T16:25:38.1265208Z     with open(dst, 'wb') as fdst:
2020-03-19T16:25:38.1265572Z FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\pip-yrkewq58-uninstall\\a\\blender-addon-tester\\blender-addon-tester\\.blender_releases_cache\\blender-2.79b-windows64\\2.79\\python\\lib\\site-packages\\pip\\_vendor\\html5lib\\filters\\__pycache__\\alphabeticalattributes.cpython-35.pyc'
2020-03-19T16:25:39.4884241Z No module named 'pytest'

Those Windows x Blender 2.78c / Blender 2.79 precisely mention absence first a failure to uninstall pip, which is a bit nonsense to me, and then no pytest module found. Though right after: further pip package installations do work and the pytest runs do run and pass successfully. So this error message could be omitted. The source of a tentative uninstall of pip is maybe the "--upgrade" flag found in: blender_addon_tester/get_blender.py: cmd = f"{python} -m pip install --upgrade -r {CURRENT_MODULE_DIRECTORY}/blender_requirements.txt -r {CURRENT_MODULE_DIRECTORY}/requirements.txt" and .travis.yml: - pip install -r ${TRAVIS_BUILD_DIR}/requirements.txt --upgrade

We could have supposed that the packages are fresh enough ( I rarely use the --upgrade pip flag).

=> We could let it live silently for now maybe.

II) python3 was not found on windows => renamed as python

github actions not working for windows and under 2.78c https://github.com/douglaskastle/blender-fake-addon/runs/512343461?check_suite_focus=true

WARNING: Requirement 'dist/*.whl' looks like a filename, but the file does not exist
ERROR: *.whl is not a valid wheel filename.

OK will investigate this week

Thank you for the merge and further commits by your team

Fixed: https://github.com/myselfhimself/blender-addon-tester/runs/519764306?check_suite_focus=true

[myselfhimself]

Github Actions caching of Blender downloaded releases between jobs does work well for Windows as well, without needed change: https://github.com/myselfhimself/blender-addon-tester/runs/519825095?check_suite_focus=true

[myselfhimself]

Focusing on Mac OS X issue douglaskastle/blender-fake-addon#7 now

[douglaskastle]

i tried to look at the pip uninstall issue, i removed the upgrade, and it still failed. it is not in the --upgrade, i believe it is also part of the requirements (somewhere). But uninstalling pip is interesting catch ... I wonder

[douglaskastle]

blender_requirments.txt has pip as a module. Due to the circular nature, i think i am pulling yours in, i am unsure how to easily disentangle the two worlds. What can i change locally, what is still being pulled from the remote server. It is a bit confusing.

I think the world has to be broken into two. Validation still has to be possible on the local work only, and then on the remote. Is that possible? Is that being done? if so how?

[douglaskastle]

OK I have remove pip from the requirements file some where. It now errors as before, but completes successfully. I suspect it might be something to do with the permissions the user is running at. A non admin trying to remove pip is probably verboten, but it is closer to the solution. Should probably be using venv any way to avoid this crap in the first place. Designed exactly for this problem.

[douglaskastle]

Scratch venv, it is the python on blender we are trying to change.

I think the problem is that the python with blender doesn't have the information to uninstall pip if it needs to be.

I added this line after ensure-pip, seemed to fix everything up:

    cmd = f"{python} c:\\cygwin64\\home\\douglas\\projects\\blender-fake-addon\\examples\\testing-fake-addon\\get-pip.py"
    os.system(cmd)

https://pip.pypa.io/en/stable/installing/

it's a bit excessive, but seems to do the job, might add it for now, or just tie it to the rev for now. Needs a tidy too, but that's all i can do for tonight.

[douglaskastle]

OK I decided to create a new repo, blender-addon-tester:

https://github.com/douglaskastle/blender-addon-tester

and have forked the whole repo over there. Please merge any future work over there.

blender-fake-addon has a different life ahead of it, it will be an example of a repo using the blender-addon-tester repo.

As of today I am just setting up the repo and making sure all the travis CI and code coverage dependencies are all in working order. The masterbranch of the blender-addon-tester repo is the blender-addon-tester branch of the blender-fake-addon repo.

I don't know how to migrate issues, this issue belongs over in the other repo now.

[myselfhimself]

Issues migrated by you thank you. I am glad of the new project created. Will open an Issue on making a pypi.org package so that at some time we do not have to target a github URL/branch anymore. Thank you for the Windows debugging and sorry for the badly assumed --upgrade flag rootcause. I must say that I am not really fan of debugging Windows more than you have done so far, as long as the whole jobs end up positively.. Resilient Windows error noise are OK for me. As to Windows work, I will have to debug Windows C++ Python compilation on a G'MIC Python binding project soon after I have my head free with blender-addon-tester. I am very lazy in local testing and run only Linux tests locally, counting on Github Actions to run for other OSes and Blender version combinations. Feel free to close this issue, maybe when the Mac OS X support is landed in https://github.com/douglaskastle/blender-addon-tester/issues/2 (hopefully for wednesday latest) and when you feel satisfied with Windows support either in Github Actions and/or your travis CI setup.

[douglaskastle]

I have opened a separate issue to capture the windows local solution.

https://github.com/douglaskastle/blender-addon-tester/issues/9

The Mac OS X has it's own issue.

PyPi has it's own issue.

Happy to and am closing this issue.

Please raise any issues as you find them.