Bad Performance

[sinainnet]

I am testing python interpreter performance against codon. The scenario is sending a message via the posix_ipc module (defined as a library). I am using the Posix message Queus. The test is like sending a message to a server and then receiving the response. The server just sends back the message that is received. I do this 1000,000 times and measure the time of 1000.000 send-receive! The execution time of the compiled codon (release build) program is always worse than the python program.

I will be happy if someone can help me through this.

[arshajii]

Hi @sinainnet -- can you please share more details and (ideally) an example to reproduce the issue? If you're using a Python library like posix_ipc then Codon will just invoke it through CPython, so the performance shouldn't be any different than calling it in Python as usual.

[sinainnet]

The codon code is like below:

from python import argparse
from python import os
from python import datetime
from python import numpy as np
from python import time
from python import threading
from python import posix_ipc
import sys
import python

@export
def processing(chunk_size: int, number_of_samples: int, verbose: bool):

    if verbose:
        print("\nCodon compiler: ")
        print f"\tChunk size: {chunk_size} Bytes"
        print f"\tSamples: {number_of_samples}"

    try:
        mq = posix_ipc.MessageQueue("/miji", flags=posix_ipc.O_CREAT)

        data = np.arange(chunk_size, dtype=np.uint8)

        start = time.time()
        for i in range(number_of_samples):
            mq.send(data)
            mq.receive()

        end = time.time()

        duration = (end - start) * 1e3
        print duration

        mq.close()

    except Exception as e:
        print("Exception encountered:", e)

And python code like below:

import argparse
import os
import datetime
import numpy as np
import time
import threading
import posix_ipc

def main(chunk_size, number_of_samples, verbose):

    if verbose:
        print("\nPython interpreter: ")
        print("\tChunk size: {} Bytes".format(chunk_size))
        print("\tNumber of samples: {}".format(number_of_samples))

    try:
        mq = posix_ipc.MessageQueue("/miji", flags=posix_ipc.O_CREAT)

        data = np.arange(chunk_size, dtype=np.uint8)

        start = time.time()
        for i in range(number_of_samples):
            mq.send(data)
            mq.receive()
        end = time.time()

        duration = (end - start) * 1e3  # ms
        print(duration)

        mq.close()

    except Exception as e:
        print("Exception encountered:", e)

[elisbyberi]

@sinainnet There is a similar case in Discussions. I believe it may be helpful: https://github.com/exaloop/codon/discussions/375#discussioncomment-5830841

[sinainnet]

@sinainnet There is a similar case in Discussions. I believe it may be helpful: #375 (comment)

@sinainnet There is a similar case in Discussions. I believe it may be helpful: #375 (comment)

In this thread, there is an open problem of using C codes in the codon. Actually, I had a plan to replace python 'poxic_ipc' with an equivalent c (or C++) implementation of MQ. The documents (https://docs.exaloop.io/codon/interoperability/cpp) are not quite helpful in this case. Is there any place that I can use to utilize the MQ that Implemented in C (and C++) in the codon?