Support for python3.11

[kishankarun]

Hi,

When I tried using the Redfish-Event-Listener with python 3.11 , I got the following error during installation of http-parser

#6 14.53 Failed to build http-parser
#6 14.67 Installing collected packages: uwsgi, ply, http-parser, XlsxWriter, urllib3, six, robotframework, pycparser, MarkupSafe, jsonpointer, itsdangerous, idna, decorator, click, charset-normalizer, y
#6 14.73   Running setup.py install for http-parser: started
#6 15.54   Running setup.py install for http-parser: finished with status 'error'
#6 15.55   error: subprocess-exited-with-error
#6 15.55
#6 15.55   × Running setup.py install for http-parser did not run successfully.
#6 15.55   │ exit code: 1
#6 15.55   ╰─> [27 lines of output]
#6 15.55       /tmp/pip-install-fc9hcutp/http-parser_3e7adc6962534c8189a5124f158888f0/setup.py:12: DeprecationWarning: the imp module is deprecated in favour of importlib and slated for removal in Pyths
#6 15.55         from imp import load_source
#6 15.55       running install
#6 15.55       /usr/local/lib/python3.11/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
#6 15.55         warnings.warn(
#6 15.55       running build
#6 15.55       running build_py
#6 15.55       creating build
#6 15.55       creating build/lib.linux-x86_64-cpython-311
#6 15.55       creating build/lib.linux-x86_64-cpython-311/http_parser
#6 15.55       copying http_parser/__init__.py -> build/lib.linux-x86_64-cpython-311/http_parser
#6 15.55       copying http_parser/_socketio.py -> build/lib.linux-x86_64-cpython-311/http_parser
#6 15.55       copying http_parser/http.py -> build/lib.linux-x86_64-cpython-311/http_parser
#6 15.55       copying http_parser/pyparser.py -> build/lib.linux-x86_64-cpython-311/http_parser
#6 15.55       copying http_parser/reader.py -> build/lib.linux-x86_64-cpython-311/http_parser
#6 15.55       copying http_parser/util.py -> build/lib.linux-x86_64-cpython-311/http_parser
#6 15.55       running build_ext
#6 15.55       building 'http_parser.parser' extension
#6 15.55       creating build/temp.linux-x86_64-cpython-311
#6 15.55       creating build/temp.linux-x86_64-cpython-311/http_parser
#6 15.55       gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iparser -I/usr/local/include/python3.11 -c http_parser/http_parser.c -o build/temp.linux-x86_64-cpython-311/http_parser/ho
#6 15.55       gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iparser -I/usr/local/include/python3.11 -c http_parser/parser.c -o build/temp.linux-x86_64-cpython-311/http_parser/parsero
#6 15.55       http_parser/parser.c:196:12: fatal error: longintrepr.h: No such file or directory
#6 15.55         196 |   #include "longintrepr.h"
#6 15.55             |            ^~~~~~~~~~~~~~~
#6 15.55       compilation terminated.
#6 15.55       error: command '/usr/bin/gcc' failed with exit code 1
#6 15.55       [end of output]
#6 15.55
#6 15.55   note: This error originates from a subprocess, and is likely not a problem with pip.
#6 15.55 error: legacy-install-failure
#6 15.55
#6 15.55 × Encountered error while trying to install package.
#6 15.55 ╰─> http-parser
#6 15.55
#6 15.55 note: This is an issue with the package mentioned above, not pip.
#6 15.55 hint: See above for output from the failure.

Could you please help here?

[mraineri]

Are you attempting to do some sort of building with Cython, or is this simply the result of running pip install -r requirements.txt?

I see there are a lot of issues with other Python projects with 3.11 with similar results where "longintrepr.h" is no longer in the C environment.

[kishankarun]

Hi @mraineri

I tried using pip install -r requirements.txt

Is there a work around or fix for the issue with 3.11 as you mentioned?

[mraineri]

I haven't found anything too definitive yet; unfortunately I do not have Python3.11 handy to test some of the things I've seen noted.

One thing that keeps coming up is the need to upgrade Cython packages; apparently this is used even if you're not freezing code for a standalone executable. At least on Ubuntu, that can be achieved with updating the build-essential package. You might also be able to update it with pip from pip3 install Cython -U. If you're willing to try this and letting us know if that helped you, that would be much appreciated.

[kishankarun]

Hi @mraineri ,

Thanks for the pointer.

I shall try doing this step and let you know...

In the meanwhile you could try with the docker container of the python 3.11 version, which I was trying with as well.

[mraineri]

Did a bit of digging and unfortunately there's no fix at the moment. The http-parser module has an open issue about this (which we're dependent upon): https://github.com/benoitc/http-parser/issues/94

If we need to solve this sooner than later, we might need to find an alternative to the http-parser module.

[kishankarun]

Did a bit of digging and unfortunately there's no fix at the moment. The http-parser module has an open issue about this (which we're dependent upon): benoitc/http-parser#94

If we need to solve this sooner than later, we might need to find an alternative to the http-parser module.

Thanks for the update @mraineri .

I shall check from my side as well. Shall update you in case I get it through.

Thanks for your support 😊👍

[kishankarun]

@mraineri can you please let me know how exactly is http-parser helping when we have default python socket and requests library available?

Could you please point me to some reference or documentation of the same?

[mraineri]

It's all used inside of the process_data method. When an event comes in, it passes the connection of the incoming request to an HttpStream object so we can extract HTTP fields and inspect the event data without having to perform HTTP parsing ourselves. Otherwise, we'd need to read the data coming over the socket and decode the payload ourselves per HTTP (mainly extracting HTTP headers, the method, and request body).

[kishankarun]

I had tried using the BaseHTTPRequestHandler.

Looks like it works

Could you please check and confirm if this would work here?

from http.server import BaseHTTPRequestHandler
from io import BytesIO

class HTTPRequest(BaseHTTPRequestHandler):
    def __init__(self, request_text):
        self.rfile = BytesIO(request_text)
        self.raw_requestline = self.rfile.readline()
        self.error_code = self.error_message = None
        self.parse_request()

    def send_error(self, code, message):
        self.error_code = code
        self.error_message = message

[mraineri]

It looks like we should be able to do something like that. Do you mind making a pull request with the changes and I can test it a bit on my end?

[kishankarun]

Hi @mraineri ,

I have created this pull request. https://github.com/DMTF/Redfish-Event-Listener/pull/29

Hope it is what is expected.

Please check and let me know.

Thanks in advance. 🙂 👍

[kishankarun]

It looks like we should be able to do something like that. Do you mind making a pull request with the changes and I can test it a bit on my end?

Hi @mraineri ,

After testing the code changes which I did, I found that I ended up modifying a great lot of code.

Instead I think making the http_parser library to be recompiled to support python 3.11 may be easier.

Let me try it out. I shall get back if I succeed. :)

[MeritedHobbit]

Hi @kishankarun ,

Did your change go in to http_parser lib?

[kishankarun]

Hi @kishankarun ,

Did your change go in to http_parser lib?

Hi @MeritedHobbit ,

It was not quite easy to dig in to http_parser lib.

I had a different way to deal with the problem...

I wrote myself a simple flask http listener, which served the purpose.

I used the HTTP URL to subscribe to the Redfish server and the events were received in the socket.

I would send you the implementation which I did and try to update the Redfish-Event-Listener to adapt to this way.

[MeritedHobbit]

@kishankarun ,

Do send what you have, we shall attempt integrating it with the Redfish Listener and validating the same.

[kishankarun]

@MeritedHobbit

This was my basic idea.

The application is basically an event listener.

This is the beginning.

from flask import Flask, jsonify

app = Flask(__name__)

@app.post('/event/')
def redfish_listener():
    data = request.json
    try:
        if isinstance(data, dict):
            if context:= data.get("Context"):
                print(context)
    except Exception as e:
        raise e

    return jsonify({"ReceivedMessage": data})

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5002)

We need to use the port parameter as per necessity.

To start with, I just started this flask application listener.

I sent a curl command to the RedFish server to subscribe for events with the appropriate values in the body. (This body of the subscription message should contain the information about the IP address of the machine in which the (above) listener is running, so that the Redfish server knows the IP and port to which it has to send the notification events)

When the subscription was successful, whatever events were generated by the redfish server were received on the socket (<IP address of the machine>:5002)

Please note, this is just the beginning of the task.

As of the original RedfishEventListener has a config file, where all the parameters can be mentioned and it has retry mechanisms as well.

This can also be implemented by adding the necessary logic, which I shall continue to work on.

Please let me know if you need any more inputs in this part.