CI actions failing - Githubissues

jared321 commented 6 months ago

Some of the GitHub actions have failed.

jared321 commented 6 months ago

@asemposki @jcyannotty @ominusliticus Ideally, the test_and_deploy GH action should test prebuilt binary wheels rather than test a local clone so that the tests are true end-to-end tests of not only the code but also packaging. Clearly this will also test a general user's experience.

I also like being able to test my installation directly. For instance, I would typically like to run

$ python
>>> import Taweret
>>> Taweret.__version__
1.0.1
>>> Taweret.test()
...

I propose to

move the test folder into the distribution as Taweret/tests, which means that it (including the test data) would be in the module as Taweret.tests
add the test function to the package that would use pytest under the hood to discover and run tests as above
build the binary wheel, install it, and use the test function to check the wheels
add a tools/test_installation.py script that users could run to test their installation rather than run the above manually

Also, I'm having better luck building wheels with the source layout than the flat layout. I prefer the source layout in general. This would require moving all your files in the package, which might make it harder to trace the history of the files in git. OK if I do this as well?

How does this sound?

jcyannotty commented 6 months ago

@jared321 everything regarding the tests sounds good to me. Two follow up questions:

what does the Taweret binary wheel contain? What purpose will the Taweret wheel serve going forward?
I'm not opposed to moving towards the source layout if it makes things easier. Will the new src folder just contain the Taweret folder, or will it also contain other items? If it just contains the Taweret folder (similar to the link you provided), then what is the benefit of doing this reconfiguration?

jared321 commented 6 months ago

@jcyannotty

Taweret is now a pure Python package. Therefore, when you run python -m build, it creates a source distribution as usual as well as a universal binary wheel that just contains the Python code and should work on any system that has a compatible version of Python 3. If I bring in the test folder, they both also contain the test data. My understanding is that for pure Python packages, the two are essentially the same. However, the wheel has gone through some extra processing that makes it ready for immediate installation. I doubt that any one would notice a difference when installing the two. Python seems to prefer wheels in all cases for some reason.
Just the Taweret folder. Although some of the annoying intermediate folders (e.g., egg) get written there too. It has made it easier to build binary wheels and with less information in the pyproject.toml because setuptools is able to find everything it needs in this layout. They also say in the documentation that there are less chances for errors. I believe that be default "." is in the PYTHONPATH. Therefore, if you are in the root of the repository with a flat layout, the code in your clone is found and used automatically if you run python or any of its tools (e.g., pydoc) there. It even overrides the use of the installed version of the package if you have already pip installed it. This can cause issues and surprises sometimes. In the source layout, this only happens if you are in the src folder, where you normally wouldn't be.

I've been playing around with it so far on my branch. Feel free to poke around and let me know if you have more questions. https://github.com/bandframework/Taweret/tree/85_FixCI

ominusliticus commented 6 months ago

I like the combining of the Taweret module code and test code. For one, this will included the test in the code report as well.

@jcyannotty wheels are binary objects. That means that they are smaller in size (faster to download) and ready to use (shorter to costumer). The user's computer would have to build the source distribution (which could potentially take very long), as it really is just snapshot of the source tree.

As an example: the Windows's source code is around 300 GB in size with ~3.5 million files, while the operating system on your computer is ~30 GB

asemposki commented 6 months ago

This is all good with me as well. I don't see any issues with the source format (at least at present), and it would be great to be able to test the packaging and not just the local clone.

jared321 commented 3 months ago

The work done on 85_FixCI was brought into develop and then main. We have not had any issues with CI recently and the current set of actions will be reassessed as part of the upcoming release. Closed.

bandframework / Taweret

CI actions failing #85