SocketcandMedia Media class added for socketcand functionality

wiboticanders commented 11 months ago

Although this runs on PythonCAN, changing the PythonCANMedia class to function with socketcand is not very clean because socketcand has 2 additional variables compared to all the other PythonCAN compatible interfaces on pycyphal. Socketcand can be referenced here: https://github.com/linux-can/socketcand , it is used for remote CAN access of udp socket. As far as copyright goes, because this is very similar to the PythonCANMedia class and I reused code from it I believe that the copyright still applies.

wiboticanders commented 11 months ago

Hello, can't tell why the read the docs build failed. Maybe because I added new files?

maksimdrachov commented 11 months ago

Regarding testing, it might be useful to take a look at the following forum post:

https://forum.opencyphal.org/t/development-setup-pycyphal-ubuntu/1844

As it relates to your question:

You can run the docs build/test using the following command (after the setup is done):

nox --session docs

The unit tests can also be run for each Python version seperately:

nox --session test-3.10

(You can also select a subset of tests to be run, see link above: Section 3, Step 5)

If you have any questions regarding the setup: feel free to ask them in the forum post! :)

pavel-kirienko commented 11 months ago

can't tell why the read the docs build failed.

Please check the logs:

They say:

/home/docs/checkouts/readthedocs.org/user_builds/pycyphal/checkouts/306/pycyphal/transport/can/media/socketcand/_socketcand.py:docstring of pycyphal.transport.can.media.socketcand._socketcand.SocketcandMedia:18: WARNING: Literal block ends without a blank line; unexpected unindent.
/home/docs/checkouts/readthedocs.org/user_builds/pycyphal/checkouts/306/pycyphal/transport/can/media/socketcand/_socketcand.py:docstring of pycyphal.transport.can.media.socketcand._socketcand.SocketcandMedia.__init__:15: WARNING: Unexpected indentation.
/home/docs/checkouts/readthedocs.org/user_builds/pycyphal/checkouts/306/pycyphal/transport/can/media/socketcand/_socketcand.py:docstring of pycyphal.transport.can.media.socketcand._socketcand.SocketcandMedia.__init__:16: WARNING: Block quote ends without a blank line; unexpected unindent.

pavel-kirienko commented 11 months ago

Please try adding a simple integration test for this media implementation. You can make a new test case similar to the SocketCAN test that starts a local socketcand daemon and connects to it.

wiboticanders commented 11 months ago

I am having an issue where I cannot run any tests on my version. Not just new tests but any test. Here is the output from the command: 'pytest tests/transport/can/media/_socketcan.py'

ImportError while loading conftest '/home/anders/Documents/Projects/Pycyphal/pycyphal/tests/conftest.py'. tests/init.py:10: in from . import dsdl as dsdl tests/dsdl/init.py:5: in from ._util import make_random_object, are_close, expand_service_types tests/dsdl/_util.py:12: in import pydsdl E ModuleNotFoundError: No module named 'pydsdl'

Also when I run 'nox --session', demo always fails for the same reason:

Running session demo nox > Creating virtual environment (virtualenv) using python3 in .nox/demo nox > python -V nox > python -m pip install -e '.[transport-can-pythoncan,transport-serial,transport-udp]' nox > python -m pip install 'yakut ~= 0.11' nox > cd /home/anders/Documents/Projects/Pycyphal/pycyphal/.nox/demo/tmp nox > Copy: /home/anders/Documents/Projects/Pycyphal/pycyphal/demo/requirements.txt nox > Copy: /home/anders/Documents/Projects/Pycyphal/pycyphal/demo/launch.orc.yaml nox > Copy: /home/anders/Documents/Projects/Pycyphal/pycyphal/demo/uavcan nox > Copy: /home/anders/Documents/Projects/Pycyphal/pycyphal/demo/plant.py nox > Copy: /home/anders/Documents/Projects/Pycyphal/pycyphal/demo/setup.py nox > Copy: /home/anders/Documents/Projects/Pycyphal/pycyphal/demo/demo_app.py nox > Copy: /home/anders/Documents/Projects/Pycyphal/pycyphal/demo/public_regulated_data_types nox > Copy: /home/anders/Documents/Projects/Pycyphal/pycyphal/demo/README.md nox > Copy: /home/anders/Documents/Projects/Pycyphal/pycyphal/demo/custom_data_types nox > yakut orc launch.orc.yaml Error: Run yakut compile <path>/uavcan to compile DSDL namespace 'uavcan' 2023-07-19 15:26:41 0312703 WAR yakut.cmd.orchestrate: Call stack: 'launch.orc.yaml' $ 0 $ 0 PID=00312717 Stopping (was started 0.4 sec ago) 2023-07-19 15:26:41 0312703 WAR yakut.cmd.orchestrate: Call stack: 'launch.orc.yaml' $ 2 $ 0 PID=00312722 Stopping (was started 0.4 sec ago) nox > Command yakut orc launch.orc.yaml failed with exit code 1 nox > Session demo failed.

I have tried the solution offered here: https://github.com/OpenCyphal/yakut/issues/18 , but it does not work.

pavel-kirienko commented 11 months ago

The first command is not supposed to work. You can only invoke tests via Nox unless you manually activate the corresponding virtual environment created by Nox beforehand.

Please try simply running nox to see if the test suite can succeed in its entirety. It is best to close all CPU-intensive programs running on your PC while the test suite is running to minimize the risk of spurious failures.

wiboticanders commented 11 months ago

Thanks for the clarification.

I still receive the same error when running nox , specifically on the test demo. This is the error: Error: Run yakut compile <path>/uavcan to compile DSDL namespace 'uavcan'

pavel-kirienko commented 11 months ago

Are you sure you cloned the repository with all submodules? The option is --recursive.

wiboticanders commented 11 months ago

@pavel-kirienko, I'm still working on building the unit test, and I have a few questions. I have been looking at the SocketCANMedia and PythonCANMedia classes and test classes to create my test. I noticed for PythonCANMedia only the virtual interface is tested unless you are running linux, then there is the _unittest_can_pythoncan_socketcan which relies on a vcan. The same for SocketCANMedia where a vcan is used. When the tests are run, usually _unittest_can_pythoncan_socketcan is skipped, but it doesn't seem like the SocketCANMedia tests are skipped.

I'm not sure how much you have looked into socketcand, but to create a unit test I think I need to create a socketcand instance somewhere in the testing, similar to how the unit test for socketcan requires a vcan to be set up (the socketcand instance will also need to rely on the same or similar vcan).

I can't seem to find how the vcan is set up in the current tests to test SocketCANMedia, and how I would be able to create a socketcand instance either.

Also, noticed you are migrating off of appveyor and onto GH actions. Not sure how that effects my test implementation.

If you could point me in the right direction, that would be very helpful.

pavel-kirienko commented 11 months ago

@wiboticanders the vcan interfaces are configured here:

https://github.com/OpenCyphal/pycyphal/blob/01b9a9b57143bc916bfa2fa26102a787b4e558e9/tests/conftest.py#L50-L58

You can create a similar fixture to go along with your tests. Place it in the same file as the test itself. More about fixtures here: https://docs.pytest.org/en/6.2.x/fixture.html

Also, noticed you are migrating off of appveyor and onto GH actions. Not sure how that effects my test implementation.

It doesn't, but the CI is temporarily borken. We will fix it soon.

wiboticanders commented 10 months ago

Thanks for clarification on vcan.

I've run into a new problem surrounding creating a socketcand instance. Essentially, to install socketcand you have to clone the source code and build it from source, it doesn't come pre built. More info on building and installing it can be found here: https://github.com/linux-can/socketcand and here: https://python-can.readthedocs.io/en/master/interfaces/socketcand.html.

Because of this, currently I think I would need to run this in the appveyor.yml file:

sudo apt-get install -y autoconf
git clone https://github.com/linux-can/socketcand.git
cd socketcand
./autogen.sh
./configure
make

From what I can tell that would be very inefficient and it would run each time the file was called.

These are the other options I found to bypass this:

Get in contact with the can-utils package maintainers to see if they can add socketcand to the build (now it will come installed with ubuntu)
Update the python-can library to add some sort of virtual socketcand version
Create a prebuilt image of socketcand that can be included in pycyphal

Unless you see an efficient way to build socketcand from how pycyphal and socketcand are set up currently, I think these are my only options. Do you have any preferences on which method to use or any other ideas on implimenting this?

pavel-kirienko commented 10 months ago

From what I can tell that would be very inefficient and it would run each time the file was called.

This is perfectly fine. The codebase is extremely compact so it doesn't take much effort to build, please proceed with this plan. We will kindly ask you to migrate your changes to the CI workflow to GitHub actions when https://github.com/OpenCyphal/pycyphal/pull/303 is merged.

wiboticanders commented 10 months ago

Should I wait until 303 is merged to make my changes then?

pavel-kirienko commented 10 months ago

At this point, it is probably best to wait, yes.

wiboticanders commented 10 months ago

Should I build and install socketcand in the github worflow file? or should I build and install within my test file when I set up my environment.

pavel-kirienko commented 10 months ago

I think it should be done in the GitHub workflow file.

pavel-kirienko commented 10 months ago

Please merge master into your branch. We have successfully migrated from AppVeyor to GitHub Actions.

wiboticanders commented 10 months ago

When cloning the new changes and attempting to run tests, I am running into an error with all the python tests where it is failing to install the package cobs. Do you have any idea why this is happening and how to fix it?

pavel-kirienko commented 10 months ago

Let's see the error message.

wiboticanders commented 10 months ago

I'll show you the error message for the 3.10 test. It is failing on all versions for what looks like the same reason, all having to do with the cobs package.

nox > Running session test-3.10 nox > Re-using existing virtual environment at .nox/test-3-10. nox > python -V nox > Using the newest supported Python: True nox > python -m pip install -e '.[transport-can-pythoncan,transport-serial,transport-udp]' nox > Command python -m pip install -e '.[transport-can-pythoncan,transport-serial,transport-udp]' failed with exit code 1: Obtaining file:///home/anders/Documents/pycyphalGHA/pycyphal Installing build dependencies: started Installing build dependencies: finished with status 'done' Checking if build backend supports build_editable: started Checking if build backend supports build_editable: finished with status 'done' Getting requirements to build editable: started Getting requirements to build editable: finished with status 'done' Preparing editable metadata (pyproject.toml): started Preparing editable metadata (pyproject.toml): finished with status 'done' Collecting nunavut~=1.7 Using cached nunavut-1.9.0-3124554183-py3-none-any.whl (333 kB) Collecting numpy~=1.22 Using cached numpy-1.25.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB) Collecting libpcap<2.0.0,>=0.0.0b0 Using cached libpcap-1.11.0b7-py3-none-any.whl (849 kB) Collecting python-can[serial]~=4.0 Using cached python_can-4.2.2-py3-none-any.whl (257 kB) Collecting cobs~=1.1.4 Using cached cobs-1.1.4.tar.gz (20 kB) Preparing metadata (setup.py): started Preparing metadata (setup.py): finished with status 'done' Requirement already satisfied: pyserial~=3.5 in ./.nox/test-3-10/lib/python3.10/site-packages (from pycyphal==1.15.3) (3.5) Requirement already satisfied: setuptools>=63.2.0 in ./.nox/test-3-10/lib/python3.10/site-packages (from libpcap<2.0.0,>=0.0.0b0->pycyphal==1.15.3) (66.1.1) Collecting pkg-about>=1.0.7 Using cached pkg_about-1.0.8-py3-none-any.whl (5.7 kB) Collecting pyyaml Using cached PyYAML-6.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (705 kB) Collecting importlib-resources Using cached importlib_resources-6.0.1-py3-none-any.whl (34 kB) Requirement already satisfied: pydsdl~=1.16 in ./.nox/test-3-10/lib/python3.10/site-packages (from nunavut~=1.7->pycyphal==1.15.3) (1.20.1) Collecting packaging Using cached packaging-23.1-py3-none-any.whl (48 kB) Requirement already satisfied: msgpack~=1.0.0 in ./.nox/test-3-10/lib/python3.10/site-packages (from python-can[serial]~=4.0->pycyphal==1.15.3) (1.0.5) Collecting wrapt~=1.10 Using cached wrapt-1.15.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (78 kB) Collecting typing-extensions>=3.10.0.0 Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB) Collecting tomli>=2.0.1 Using cached tomli-2.0.1-py3-none-any.whl (12 kB) Collecting importlib-metadata>=4.12.0 Using cached importlib_metadata-6.8.0-py3-none-any.whl (22 kB) Collecting zipp>=0.5 Using cached zipp-3.16.2-py3-none-any.whl (7.2 kB) Building wheels for collected packages: cobs, pycyphal Building wheel for cobs (setup.py): started Building wheel for cobs (setup.py): finished with status 'error' error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully. │ exit code: 1 ╰─> [140 lines of output] running bdist_wheel running build running build_py creating build creating build/lib.linux-x86_64-cpython-310 creating build/lib.linux-x86_64-cpython-310/cobs copying python3/cobs/init.py -> build/lib.linux-x86_64-cpython-310/cobs creating build/lib.linux-x86_64-cpython-310/cobs/cobs copying python3/cobs/cobs/test.py -> build/lib.linux-x86_64-cpython-310/cobs/cobs copying python3/cobs/cobs/_cobs_py.py -> build/lib.linux-x86_64-cpython-310/cobs/cobs copying python3/cobs/cobs/init.py -> build/lib.linux-x86_64-cpython-310/cobs/cobs creating build/lib.linux-x86_64-cpython-310/cobs/cobsr copying python3/cobs/cobsr/test.py -> build/lib.linux-x86_64-cpython-310/cobs/cobsr copying python3/cobs/cobsr/_cobsr_py.py -> build/lib.linux-x86_64-cpython-310/cobs/cobsr copying python3/cobs/cobsr/init.py -> build/lib.linux-x86_64-cpython-310/cobs/cobsr creating build/lib.linux-x86_64-cpython-310/cobs/_version copying _version/init.py -> build/lib.linux-x86_64-cpython-310/cobs/_version running build_ext building 'cobs.cobs._cobs_ext' extension creating build/temp.linux-x86_64-cpython-310 creating build/temp.linux-x86_64-cpython-310/python3 creating build/temp.linux-x86_64-cpython-310/python3/src gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/include -I/usr/local/include/python3.10 -c python3/src/_cobs_ext.c -o build/temp.linux-x86_64-cpython-310/python3/src/_cobs_ext.o gcc -shared build/temp.linux-x86_64-cpython-310/python3/src/_cobs_ext.o -o build/lib.linux-x8664-cpython-310/cobs/cobs/cobs_ext.cpython-310-x86_64-linux-gnu.so building 'cobs.cobsr._cobsr_ext' extension gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/include -I/usr/local/include/python3.10 -c python3/src/_cobsr_ext.c -o build/temp.linux-x86_64-cpython-310/python3/src/_cobsr_ext.o gcc -shared build/temp.linux-x86_64-cpython-310/python3/src/_cobsr_ext.o -o build/lib.linux-x8664-cpython-310/cobs/cobsr/cobsr_ext.cpython-310-x86_64-linux-gnu.so /home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools. warnings.warn( installing to build/bdist.linux-x86_64/wheel running install running install_lib running bdist_wheel running build running build_py creating build/lib creating build/lib/cobs copying python3/cobs/init.py -> build/lib/cobs creating build/lib/cobs/cobs copying python3/cobs/cobs/test.py -> build/lib/cobs/cobs copying python3/cobs/cobs/_cobs_py.py -> build/lib/cobs/cobs copying python3/cobs/cobs/init.py -> build/lib/cobs/cobs creating build/lib/cobs/cobsr copying python3/cobs/cobsr/test.py -> build/lib/cobs/cobsr copying python3/cobs/cobsr/_cobsr_py.py -> build/lib/cobs/cobsr copying python3/cobs/cobsr/init.py -> build/lib/cobs/cobsr creating build/lib/cobs/_version copying _version/init.py -> build/lib/cobs/_version installing to build/bdist.linux-x86_64/wheel running install running install_lib Traceback (most recent call last): File "/tmp/pip-install-owjhgmt1/cobs_d2e28daa5a0a4fb1a2024ff6c09be8e5/setup.py", line 58, in setup(setup_dict) File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/init.py", line 108, in setup return distutils.core.setup(attrs) File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup return run_commands(dist) File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands dist.run_commands() File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands self.run_command(cmd) File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/dist.py", line 1213, in run_command super().run_command(command) File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/wheel/bdist_wheel.py", line 360, in run self.run_command("install") File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command self.distribution.run_command(command) File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/dist.py", line 1213, in run_command super().run_command(command) File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/command/install.py", line 68, in run return orig.install.run(self) File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/command/install.py", line 709, in run self.run_command(cmd_name) File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command self.distribution.run_command(command) File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/dist.py", line 1213, in run_command super().run_command(command) File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 987, in run_command cmd_obj.ensure_finalized() File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 111, in ensure_finalized self.finalize_options() File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/command/install_lib.py", line 17, in finalize_options self.set_undefined_options('install',('install_layout','install_layout')) File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 296, in set_undefined_options setattr(self, dst_option, getattr(src_cmd_obj, src_option)) File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 107, in getattr raise AttributeError(attr) AttributeError: install_layout. Did you mean: 'install_platlib'?

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/tmp/pip-install-owjhgmt1/cobs_d2e28daa5a0a4fb1a2024ff6c09be8e5/setup.py", line 63, in <module>
      setup(**setup_dict)
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/__init__.py", line 108, in setup
      return distutils.core.setup(**attrs)
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
      return run_commands(dist)
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
      dist.run_commands()
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
      self.run_command(cmd)
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/dist.py", line 1213, in run_command
      super().run_command(command)
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/wheel/bdist_wheel.py", line 360, in run
      self.run_command("install")
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/dist.py", line 1213, in run_command
      super().run_command(command)
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/command/install.py", line 68, in run
      return orig.install.run(self)
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/command/install.py", line 709, in run
      self.run_command(cmd_name)
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/dist.py", line 1213, in run_command
      super().run_command(command)
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
      cmd_obj.ensure_finalized()
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 111, in ensure_finalized
      self.finalize_options()
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/command/install_lib.py", line 17, in finalize_options
      self.set_undefined_options('install',('install_layout','install_layout'))
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 296, in set_undefined_options
      setattr(self, dst_option, getattr(src_cmd_obj, src_option))
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
      raise AttributeError(attr)
  AttributeError: install_layout. Did you mean: 'install_platlib'?
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for cobs Running setup.py clean for cobs Building editable for pycyphal (pyproject.toml): started Building editable for pycyphal (pyproject.toml): finished with status 'done' Created wheel for pycyphal: filename=pycyphal-1.15.3-0.editable-py3-none-any.whl size=4917 sha256=e830e055822c669a1489dbe5bb668fbdc5ec018cd8e30503a081c007adf9fd35 Stored in directory: /tmp/pip-ephem-wheel-cache-tfpxpv3l/wheels/c6/1c/07/e2b5bd36f1f37cf078bc992ff57778f2f4ae97daa1cec6fedc Successfully built pycyphal Failed to build cobs Installing collected packages: cobs, zipp, wrapt, typing-extensions, tomli, pyyaml, packaging, numpy, importlib-resources, python-can, nunavut, importlib-metadata, pycyphal, pkg-about, libpcap Running setup.py install for cobs: started Running setup.py install for cobs: finished with status 'error' error: subprocess-exited-with-error

× Running setup.py install for cobs did not run successfully. │ exit code: 1 ╰─> [106 lines of output] running install /home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools. warnings.warn( running build running build_py creating build/lib.linux-x86_64-cpython-310 creating build/lib.linux-x86_64-cpython-310/cobs copying python3/cobs/init.py -> build/lib.linux-x86_64-cpython-310/cobs creating build/lib.linux-x86_64-cpython-310/cobs/cobs copying python3/cobs/cobs/test.py -> build/lib.linux-x86_64-cpython-310/cobs/cobs copying python3/cobs/cobs/_cobs_py.py -> build/lib.linux-x86_64-cpython-310/cobs/cobs copying python3/cobs/cobs/init.py -> build/lib.linux-x86_64-cpython-310/cobs/cobs creating build/lib.linux-x86_64-cpython-310/cobs/cobsr copying python3/cobs/cobsr/test.py -> build/lib.linux-x86_64-cpython-310/cobs/cobsr copying python3/cobs/cobsr/_cobsr_py.py -> build/lib.linux-x86_64-cpython-310/cobs/cobsr copying python3/cobs/cobsr/init.py -> build/lib.linux-x86_64-cpython-310/cobs/cobsr creating build/lib.linux-x86_64-cpython-310/cobs/_version copying _version/init.py -> build/lib.linux-x86_64-cpython-310/cobs/_version running build_ext building 'cobs.cobs._cobs_ext' extension creating build/temp.linux-x86_64-cpython-310 creating build/temp.linux-x86_64-cpython-310/python3 creating build/temp.linux-x86_64-cpython-310/python3/src gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/include -I/usr/local/include/python3.10 -c python3/src/_cobs_ext.c -o build/temp.linux-x86_64-cpython-310/python3/src/_cobs_ext.o gcc -shared build/temp.linux-x86_64-cpython-310/python3/src/_cobs_ext.o -o build/lib.linux-x8664-cpython-310/cobs/cobs/cobs_ext.cpython-310-x86_64-linux-gnu.so building 'cobs.cobsr._cobsr_ext' extension gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/include -I/usr/local/include/python3.10 -c python3/src/_cobsr_ext.c -o build/temp.linux-x86_64-cpython-310/python3/src/_cobsr_ext.o gcc -shared build/temp.linux-x86_64-cpython-310/python3/src/_cobsr_ext.o -o build/lib.linux-x8664-cpython-310/cobs/cobsr/cobsr_ext.cpython-310-x86_64-linux-gnu.so running install_lib running install running build running build_py running install_lib Traceback (most recent call last): File "/tmp/pip-install-owjhgmt1/cobs_d2e28daa5a0a4fb1a2024ff6c09be8e5/setup.py", line 58, in setup(setup_dict) File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/init.py", line 108, in setup return distutils.core.setup(attrs) File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup return run_commands(dist) File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands dist.run_commands() File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands self.run_command(cmd) File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/dist.py", line 1213, in run_command super().run_command(command) File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/command/install.py", line 68, in run return orig.install.run(self) File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/command/install.py", line 709, in run self.run_command(cmd_name) File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command self.distribution.run_command(command) File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/dist.py", line 1213, in run_command super().run_command(command) File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 987, in run_command cmd_obj.ensure_finalized() File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 111, in ensure_finalized self.finalize_options() File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/command/install_lib.py", line 17, in finalize_options self.set_undefined_options('install',('install_layout','install_layout')) File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 296, in set_undefined_options setattr(self, dst_option, getattr(src_cmd_obj, src_option)) File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 107, in getattr raise AttributeError(attr) AttributeError: install_layout. Did you mean: 'install_platlib'?

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/tmp/pip-install-owjhgmt1/cobs_d2e28daa5a0a4fb1a2024ff6c09be8e5/setup.py", line 63, in <module>
      setup(**setup_dict)
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/__init__.py", line 108, in setup
      return distutils.core.setup(**attrs)
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
      return run_commands(dist)
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
      dist.run_commands()
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
      self.run_command(cmd)
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/dist.py", line 1213, in run_command
      super().run_command(command)
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/command/install.py", line 68, in run
      return orig.install.run(self)
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/command/install.py", line 709, in run
      self.run_command(cmd_name)
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/dist.py", line 1213, in run_command
      super().run_command(command)
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
      cmd_obj.ensure_finalized()
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 111, in ensure_finalized
      self.finalize_options()
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/command/install_lib.py", line 17, in finalize_options
      self.set_undefined_options('install',('install_layout','install_layout'))
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 296, in set_undefined_options
      setattr(self, dst_option, getattr(src_cmd_obj, src_option))
    File "/home/anders/Documents/pycyphalGHA/pycyphal/.nox/test-3-10/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
      raise AttributeError(attr)
  AttributeError: install_layout. Did you mean: 'install_platlib'?
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: legacy-install-failure

× Encountered error while trying to install package. ╰─> cobs

note: This is an issue with the package mentioned above, not pip. hint: See above for output from the failure. nox > Session test-3.10 failed.

pavel-kirienko commented 10 months ago

Please try installing COBS in a separate virtual environment and see what happens.
Try https://stackoverflow.com/questions/36296134/attributeerror-install-layout-when-attempting-to-install-a-package-in-a-virtual.

What operating system are you using?

wiboticanders commented 10 months ago

I am using debian 12. I tried it in a separate virtual env and cobs installed just fine.

wiboticanders commented 10 months ago

Also, if I just try and run pytest tests/, I get a different error: ImportError while loading conftest '/home/anders/Documents/myPycyphalGHA/pycyphal/tests/conftest.py'. tests/init.py:10: in from . import dsdl as dsdl tests/dsdl/init.py:5: in from ._util import make_random_object, are_close, expand_service_types tests/dsdl/_util.py:12: in import pydsdl E ModuleNotFoundError: No module named 'pydsdl'

I followed the directions here for setting up my testing environment: [https://forum.opencyphal.org/t/development-setup-pycyphal-ubuntu/1844]

wiboticanders commented 10 months ago

I am not sure why it is now failing the docs build, but I did manage to get my tests running and working otherwise.

pavel-kirienko commented 10 months ago

You should now be able to run the test suite by pushing a commit with the #test hashtag in the commit message.

pavel-kirienko commented 10 months ago

Simply connect to any of the existing ones.

On Wed, Aug 30, 2023, 03:49 wiboticanders @.***> wrote:

@.**** commented on this pull request.

In tests/transport/can/media/_socketcand.py https://github.com/OpenCyphal/pycyphal/pull/306#discussion_r1309483208:

if sys.platform.startswith("linux"):

import resource # pylint: disable=import-error

_logger.info("Limiting process memory usage to %.1f GiB", MEMORY_LIMIT / GIBIBYTE)

resource.setrlimit(resource.RLIMIT_AS, (MEMORY_LIMIT, MEMORY_LIMIT))

Set up virtual SocketCAN interfaces.

execute("sudo", "modprobe", "can")

execute("sudo", "modprobe", "can_raw")

execute("sudo", "modprobe", "vcan")

execute("sudo", "ip", "link", "add", "dev", "vcan3", "type", "vcan", ensure_success=False)

execute("sudo", "ip", "link", "set", "vcan3", "mtu", 72) # Enable both Classic CAN and CAN FD.

execute("sudo", "ip", "link", "set", "up", "vcan3")

Should I change it so that it adds one more vcan instance for mine to connect too? or simply connect to one of the ones previously instantiated

— Reply to this email directly, view it on GitHub https://github.com/OpenCyphal/pycyphal/pull/306#discussion_r1309483208, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZFIZD4D4U43L6ITNJ6E23XX2EX7ANCNFSM6AAAAAA2GWWAAM . You are receiving this because you were mentioned.Message ID: @.***>

pavel-kirienko commented 10 months ago

These steps are highly platform-dependent so they are better moved to the workflow file. Currently I can't even run the test on my PC because I'm using not an APT-based distro.

On Wed, Aug 30, 2023, 04:05 wiboticanders @.***> wrote:

@.**** commented on this pull request.

In tests/transport/can/media/_socketcand.py https://github.com/OpenCyphal/pycyphal/pull/306#discussion_r1309491411:

build and install socketcand

execute("sudo", "apt-get", "install", "-y", "autoconf")

execute("git", "clone", "https://github.com/linux-can/socketcand.git")

execute("./autogen.sh", cwd="socketcand")

execute("./configure", cwd="socketcand")

execute("make", cwd="socketcand")

execute("sudo", "make", "install", cwd="socketcand")

I might be mistaken but for local testing GH workflow isn't used, and nox is. So I think it would make more sense for me to put it in the nox file so that it can be run. I also thought it may be possible to put it the conftest.py file. Do you have a preference on any of these

— Reply to this email directly, view it on GitHub https://github.com/OpenCyphal/pycyphal/pull/306#discussion_r1309491411, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZFIZBUNMLGQRFXO4RYEU3XX2GV5ANCNFSM6AAAAAA2GWWAAM . You are receiving this because you were mentioned.Message ID: @.***>

wiboticanders commented 10 months ago

So possibly something like this within workflow: name: Setup socketcand run: | sudo apt-get install -y autoconf git clone https://github.com/linux-can/socketcand.git cd socketcand ./autogen.sh ./configure make sudo make install shell: bash

And then within the actual unit test I can run the daemon and take it down.

On Tue, Aug 29, 2023 at 6:16 PM Pavel Kirienko @.***> wrote:

These steps are highly platform-dependent so they are better moved to the workflow file. Currently I can't even run the test on my PC because I'm using not an APT-based distro.

On Wed, Aug 30, 2023, 04:05 wiboticanders @.***> wrote:

@.**** commented on this pull request.

In tests/transport/can/media/_socketcand.py https://github.com/OpenCyphal/pycyphal/pull/306#discussion_r1309491411:

build and install socketcand

execute("sudo", "apt-get", "install", "-y", "autoconf")

execute("git", "clone", "https://github.com/linux-can/socketcand.git")

execute("./autogen.sh", cwd="socketcand")

execute("./configure", cwd="socketcand")

execute("make", cwd="socketcand")

execute("sudo", "make", "install", cwd="socketcand")

I might be mistaken but for local testing GH workflow isn't used, and nox is. So I think it would make more sense for me to put it in the nox file so that it can be run. I also thought it may be possible to put it the conftest.py file. Do you have a preference on any of these

— Reply to this email directly, view it on GitHub https://github.com/OpenCyphal/pycyphal/pull/306#discussion_r1309491411,

or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAZFIZBUNMLGQRFXO4RYEU3XX2GV5ANCNFSM6AAAAAA2GWWAAM>

. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/OpenCyphal/pycyphal/pull/306#issuecomment-1698349736, or unsubscribe https://github.com/notifications/unsubscribe-auth/BA63ESEBHSL4EPSC2KR4FI3XX2H5ZANCNFSM6AAAAAA2GWWAAM . You are receiving this because you were mentioned.Message ID: @.***>

pavel-kirienko commented 10 months ago

So possibly something like this within workflow

Yes!

And then within the actual unit test I can run the daemon and take it down

wiboticanders commented 10 months ago

I put #test in my commit message, but it seems like it's still skipping the test and release tests. Also, assuming all works correctly in the workflow file, this should be close to a finalized and working version

pavel-kirienko commented 10 months ago

I think it should work now.

coveralls commented 10 months ago

coverage: 95.212% (-0.4%) from 95.569% when pulling 00baae945348d1c497b34b14b63e07576149bff2 on wiboticanders:master into 1eefb916614ea8a964bc63f04c259b1a3b0f8cb2 on OpenCyphal:master.

wiboticanders commented 10 months ago

Commit #[00baae9] revealed an error in the breakdown when the test fails. I was going through suggestions on the github site and not locally, and I thought it was all changes to the docs, so I committed the suggestions. One slipped past me that was a change to the interface return, which in turn broke the unit test because the returns did not match anymore. The problem stems from the way I collecting stderr and stdout after the yield, the communicate function never ended. So, that commit will not stop testing until whatever timeouts you have in place stop it. My apologies for overlooking that issue and pushing it, but I can't seem to stop it with my permissions so it has been running for a few hours.

wiboticanders commented 10 months ago

So new version should be 1.16.0?

OpenCyphal / pycyphal

SocketcandMedia Media class added for socketcand functionality #306

@.**** commented on this pull request.

Set up virtual SocketCAN interfaces.

@.**** commented on this pull request.

build and install socketcand

@.**** commented on this pull request.

build and install socketcand