search

nangtani / blender-addon-tester

The blender addon tester is a test harness to enable pytest hook to allow addons to be tested inside a defined version of blender.
MIT License
65 stars 14 forks source link

Add support for MacOS X #2

Closed myselfhimself closed 4 years ago

myselfhimself commented 4 years ago

We need this for gmic-py x blender :)

Failing build at: https://github.com/myselfhimself/gmic-blender/runs/513968946?check_suite_focus=true

raise Exception(f"dmg Unsupported")

douglaskastle commented 4 years ago

@myselfhimself I don't have access to an Mac anything here. I have written it so that it should download the dmg for a mac if the operating system is detected. I have no experience with mounting and then using a dmg image. I am hoping it is as simple as the other two

myselfhimself commented 4 years ago

I have not mac too but will run blender in the blind using Github Actions and fix it progressively

myselfhimself commented 4 years ago

Notes on mounting: https://apple.stackexchange.com/questions/73926/is-there-a-command-to-install-a-dmg

Blender .app file https://docs.blender.org/manual/en/latest/getting_started/installing/macos.html

Blender full path with .app directory to executable for command line execution: https://docs.blender.org/manual/en/latest/advanced/command_line/launch/macos.html

douglaskastle commented 4 years ago

@myselfhimself those are some good notes. If you don't get to it I'll have a look at getting that to work. Once the location of blender can be found somewhere it should all just all into place after that.

myselfhimself commented 4 years ago

Continuing work on this today

myselfhimself commented 4 years ago

dmglib seems to be a nice little python library to abstract dmg mounting and demounting, trying it out

douglaskastle commented 4 years ago

@myselfhimself how do I contact you outside of issues?

my email address is douglas.kastle gmail com

myselfhimself commented 4 years ago

My PR was merged too early, but I now have comitter access to the repository. Look I will just work in a branch and delete my fork. NP :)

myselfhimself commented 4 years ago

WIP branch is: https://github.com/douglaskastle/blender-addon-tester/tree/macosx-support

myselfhimself commented 4 years ago

Using dmglib was fruitful, preventing my needing to work with subprocesses.

I am struggling with the case of Blender 2.78c x mac os x. Back in the 2.78c days, packaging was obviously not done as a dmg but a zipped archive of a blender.app image folder (not Blender.app), with a lowercase blender executable in it (instead of uppercase). Hoping to fix this today. The code of getBlender is now starting to get messy with special cases.. OOP could help at some point after this..

myselfhimself commented 4 years ago

Github Actions shows flakiness for environment variables management or maybe is it our project. When running a 2.80 test alongside a 2.78c test, the 2.78c test switches from 2.78c to 2.80 in the middle and downloads blender 2.80. Still troubleshooting that.

douglaskastle commented 4 years ago

Check the files in the example/* directory, I edited one to pass version and addon I think there are 2 others that need the same edit.

This is where having a local machine is invaluable, and the reason I didn't develop it. Time vampire is the phrase I have heard.

myselfhimself commented 4 years ago

The 2.80 sticky variable issue was due to the rebuild_pipreinstall_and_testexamples.sh entry point not having the BLENDER_VERSION environment variable injected (I had forgotten it). Also each test was run one time too many. This is now solved.

Also, Blender Mac OS X releases in the 2.78c-like older zipped format (not dmg) needed a chmod +x on top of the bundled blender and python executables. This is now applied.

Now I am facing an issue of blender 2.78c's bundled Python not seeing pytest. This can be seen in this Github Actions run. Highlighting the suspecting log lines:

2020-03-25T18:18:18.8704570Z found bundled python: /Users/runner/runners/2.165.2/work/blender-addon-tester/blender-addon-tester/.blender_releases_cache/blender-2.78c/MacOS/../Resources/2.78/python
2020-03-25T18:18:18.8768200Z Running /Users/runner/runners/2.165.2/work/blender-addon-tester/blender-addon-tester/examples/testing-fake-addon/blender_advanced_load_pytest.py from Blender
2020-03-25T18:18:18.8768900Z No module named 'pytest'
2020-03-25T18:18:18.8979570Z Error: Not freed memory blocks: 1, total unfreed memory 0.000122 MB

Most probably, blender's python version is not the same as the OS's python or is too different from it. So possibly package directories are not shared or acknowledged the same way. Hence, sharing the location of a non-same-python-executable version of pytest inside the python path of Blender's bundled python might not help. I do not know what to do: enforce a lower python version for when macosx X blender 2.78c is run? pip install from inside blender, which would need adding pip there? @douglaskastle This is related to the trickery you have used and makes this project neat. Would you have some resolution idea?

douglaskastle commented 4 years ago

First are you sure you are picking up the blender python and not the system python? I can't tell from your log which you are using. In linux I would use readlink -f python to give me the full path. If you are using the right python ...

2.78c was only put in to have an option. I am now beginning to find it causing problems elsewhere, and since then 2.80, 2,81, 2,82 have come along and they provide options there weren't available when I started.

One of the problems I had, which may or may not be the problem, is that the current version of zipp, which is used by pytest, has deprecated support for 3.5. Look over in lwo import project, i had to have a special requirements file for 2.78c. I think he might have unrolled that change, because your work under ubuntu and windows on github works and there is no special requirements file for them.

Ultimately I think it is ok to remove 2.78c from the github actions flow. I would like to know whats wrong, but unless you have a hard requirement for support, it not worth chasing too hard.

I would love to get off 2.79, as this is the last using python3.5. There are certain language constructs on 3.7 that i would like to upgrade to. But 2,79 will be a magic build for a few years to come.

Operationally there are a lot of users stuck on 2.79 who are afraid of 2.80. I would be surprised to find anyone stuck on 2,78. If they are not on 2.79, then they are probably on some much older version of blender, 2.65 or something, which we are not supporting any way.

douglaskastle commented 4 years ago

on review, i don't think you are using the blender python, just the system. You will probably also have a problem with 2.79. This is where the local install would be very helpful

myselfhimself commented 4 years ago

Hello I have changed the mechanism of custom test script evaluation by blender's bundled python.

Regarding these log lines:

2020-03-25T18:18:18.8704570Z found bundled python: /Users/runner/runners/2.165.2/work/blender-addon-tester/blender-addon-tester/.blender_releases_cache/blender-2.78c/MacOS/../Resources/2.78/python
2020-03-25T18:18:18.8768200Z Running /Users/runner/runners/2.165.2/work/blender-addon-tester/blender-addon-tester/examples/testing-fake-addon/blender_advanced_load_pytest.py from Blender
2020-03-25T18:18:18.8768900Z No module named 'pytest'
2020-03-25T18:18:18.8979570Z Error: Not freed memory blocks: 1, total unfreed memory 0.000122 MB

they stem from the blender-evaluated python file

which is injected from the config dict here.

Good point about blender 2.78c. Putting Blender 2.79 instead.

myselfhimself commented 4 years ago

Blender 2.79's download fails, it seems that scraper cannot find the download anymore on the webpage: https://github.com/douglaskastle/blender-addon-tester/runs/534524536?check_suite_focus=true

douglaskastle commented 4 years ago

ok yes that look like you are calling the blender python

where is the pip install requirements happening, the failure complaint will be there by pytest is not present

douglaskastle commented 4 years ago

Blender 2.79's download fails, it seems that scraper cannot find the download anymore on the webpage: https://github.com/douglaskastle/blender-addon-tester/runs/534524536?check_suite_focus=true

There is always something like that, I think they have issues over on blender.org today

https://www.blendernation.com/2020/03/25/blender-org-server-issues/

douglaskastle commented 4 years ago

oh, make sure you do a failure test at some point. Change the rev in the test to be wrong and make sure all the passing tests fail. Propagating the failure out of python, to blender, to travis was lengthy enough. There is a series of tests possible around those failure modes. (also if pytest fails, it needs to error as this failure happens before blender is even called)

douglaskastle commented 4 years ago

also, there are failures sometimes due to source server problems, a sleep and retry should be implemented, actually that is its own issue

myselfhimself commented 4 years ago

The lack of match is due to blender 2.79 not existing anymore as a .zip or .dmg, just as .tar.gz Now only blender 2.79a and 2.79b exist as zip or dmg Related downloads list page is here: https://download.blender.org/release/Blender2.79/

myselfhimself commented 4 years ago

tar.gz support for macos archive added

myselfhimself commented 4 years ago

The MacOS archives inner files layout vary too much, this is very painful to code..:

myselfhimself commented 4 years ago

Linux and blender 2.79b do not get on well together: 2020-03-26T00:51:00.1362624Z /home/runner/work/blender-addon-tester/blender-addon-tester/.blender_releases_cache/blender-2.79b/blender: error while loading shared libraries: libGLU.so.1: cannot open shared object file: No such file or directory https://github.com/douglaskastle/blender-addon-tester/runs/535156496?check_suite_focus=true

linux and 2.78c as well (bundled python permissions)

myselfhimself commented 4 years ago

Nightlies have new compression formats:

2020-03-26T01:00:08.5386735Z Downloading https://builder.blender.org/download/blender-2.83-188ccfb0dd67-linux64.tar.xz
2020-03-26T01:00:08.5387157Z Error, unknown archive extension: {blender_zipfile}. Will not extract it.}
myselfhimself commented 4 years ago

tar.xz support added It seems we are now all green for all OSes, all pytest cases, all blender >=2.80 versions including nightlies, on both Travis CI and github actions

douglaskastle commented 4 years ago

Linux and blender 2.79b do not get on well together: 2020-03-26T00:51:00.1362624Z /home/runner/work/blender-addon-tester/blender-addon-tester/.blender_releases_cache/blender-2.79b/blender: error while loading shared libraries: libGLU.so.1: cannot open shared object file: No such file or directory https://github.com/douglaskastle/blender-addon-tester/runs/535156496?check_suite_focus=true

linux and 2.78c as well (bundled python permissions)

I had this problem on Linux too. To get around it I did an apt-get of blender, this installed blender and all its dependacies, which are what's missing.

I have no macosx experience, I don't know if there is a apt equivalent.

I would recommend getting 2.79 support, as this is a massive inflection point, there will be a group of people stuck on it for years.

However, that's up to you if you want to complete, if not, I'll take it over and try and get out working.

I would still try and keep 2.79 operational for windows and Linux and have a pass for macosx.

Thoughts?

myselfhimself commented 4 years ago

for clarification 2.78 and 2.79 do work on mac os x as shown here

I will try to add your 2.79 technique for linux only. For 2.78 I do not know yet. Thanks

douglaskastle commented 4 years ago

yes remove 2.78, thats fine

myselfhimself commented 4 years ago

Good for me