errors: " helper.py TypeError: a bytes-like object is required, not 'str' " and " 'Namespace' object has no attribute 'timeit' "

[lmarval]

Hi, nice project. I came to learn a lot from it.

I got these errors when using %%cuda --name sourcecodename.cu --compile true 1) " helper.py TypeError: a bytes-like object is required, not 'str' " 2) " 'Namespace' object has no attribute 'timeit' "

I got to solve them as follows: 1)in v2.py, line 49, add: @argument("-t", "--timeit", action='store_true', help='flag to return timeit result instead of stdout')

2)in v2.py, line change tp: res.decode("utf8")

I was trying to use the plugin with opencv in Colab. I think I solved it. Allow me to share the solution here.

modified version of v2.py:

## include opencv
import os
import subprocess

from IPython.core.magic import Magics, cell_magic, magics_class
from IPython.core.magic_arguments import argument, magic_arguments, parse_argstring
from common import helper

compiler = '/usr/local/cuda/bin/nvcc'

result = subprocess.run(['pkg-config', '--cflags',  '--libs', 'opencv' ], stdout=subprocess.PIPE)
cv_string = result.stdout.decode('utf-8')

@magics_class
class NVCCPluginV2(Magics):

    def __init__(self, shell):
        super(NVCCPluginV2, self).__init__(shell)
        self.argparser = helper.get_argparser()
        current_dir = os.getcwd()
        self.output_dir = os.path.join(current_dir, 'src')
        if not os.path.exists(self.output_dir):
            os.mkdir(self.output_dir)
            print(f'created output directory at {self.output_dir}')
        else:
            print(f'directory {self.output_dir} already exists')

        self.out = os.path.join(current_dir, "result.out")
        print(f'Out bin {self.out}')

    @staticmethod
    def compile(output_dir, file_paths, out):
        res = subprocess.check_output( [compiler, '-I' + output_dir, file_paths, "-o", out, '-Wno-deprecated-gpu-targets',  *( cv_string[:-1].split(" ") ) ], stderr=subprocess.STDOUT)
        print(   [compiler, '-I' + output_dir, file_paths, "-o", out, '-Wno-deprecated-gpu-targets',  *( cv_string[:-1].split(" ") )   ] )
        print(res)
        print( type(res) )
        helper.print_out(res.decode("utf8"))

    def run(self, timeit=False):
        if timeit:
            stmt = f"subprocess.check_output(['{self.out}'], stderr=subprocess.STDOUT)"
            output = self.shell.run_cell_magic(
                magic_name="timeit", line="-q -o import subprocess", cell=stmt)
        else:
            output = subprocess.check_output(
                [self.out], stderr=subprocess.STDOUT)
            output = output.decode('utf8')

        helper.print_out(output)
        return None

    @magic_arguments()
    @argument('-n', '--name', type=str, help='file name that will be produced by the cell. must end with .cu extension')
    @argument('-c', '--compile', type=bool, help='Should be compiled?')
    @argument("-t", "--timeit", action='store_true',  help='flag to return timeit result instead of stdout')
    @cell_magic
    def cuda(self, line='', cell=None):
        args = parse_argstring(self.cuda, line)
        ex = args.name.split('.')[-1]
        if ex not in ['cu', 'h']:
            raise Exception('name must end with .cu or .h')

        if not os.path.exists(self.output_dir):
            print(f'Output directory does not exist, creating')
            try:
                os.mkdir(self.output_dir)
            except OSError:
                print(f"Creation of the directory {self.output_dir} failed")
            else:
                print(f"Successfully created the directory {self.output_dir}")

        file_path = os.path.join(self.output_dir, args.name)
        with open(file_path, "w") as f:
            f.write(cell)

        if args.compile:
            try:
                print("about to compile")
                self.compile(self.output_dir, file_path, self.out)
                print("about to run")
                output = self.run(timeit=args.timeit)
            except subprocess.CalledProcessError as e:
                print(output.decode("utf8"))
                helper.print_out(e. output.decode("utf8") )
                output = None
        else:
            output = f'File written in {file_path}'

        return output

    @cell_magic
    def cuda_run(self, line='', cell=None):
        try:
            args = self.argparser.parse_args(line.split())
        except SystemExit:
            self.argparser.print_help()
            return

        try:
            cuda_src = os.listdir(self.output_dir)
            cuda_src = [os.path.join(self.output_dir, x)
                        for x in cuda_src if x[-3:] == '.cu']
            print(f'found sources: {cuda_src}')
            self.compile(self.output_dir, ' '.join(cuda_src), self.out)
            output = self.run(timeit=args.timeit)
        except subprocess.CalledProcessError as e:
            helper.print_out(e.output.decode("utf8"))
            output = None

        return output

important changes are:

result = subprocess.run(['pkg-config', '--cflags',  '--libs', 'opencv' ], stdout=subprocess.PIPE)
cv_string = result.stdout.decode('utf-8')
. . .
    def compile(output_dir, file_paths, out):
        res = subprocess.check_output( [compiler, '-I' + output_dir, file_paths, "-o", out, '-Wno-deprecated-gpu-targets',  *( cv_string[:-1].split(" ") ) ], stderr=subprocess.STDOUT)

I tested it with the following cell (not a thoroughly test though)

%%cuda --name sourcecodename.cu --compile true

#include <opencv2/core/core.hpp>
#include <iostream>

using namespace cv;
using namespace std;
int main( int argc, char** argv )
{
    std::cout << cv::getBuildInformation() << std::endl;
    return 0;
}

[andreinechaev]

Hi @lmarval, great work. Looks like you found some points of improvements. A PR would be appreciated. Using additional libs is a great feature to have. Perhaps, we should extend the implementation to have the ability to list desired libs. I'd suggest introducing a new flag. Are you interested in contributing to the project? If so, please do, I will review any contribution ASAP. If not, not a problem, I will consider your proposal later. Thanks!

[cosminc98]

Hello @lmarval and thank you for your contribution! In the documentation of version 1.1.0 we added a notebook showing how to compile CUDA C++ with OpenCV by passing the necessary arguments to nvcc. As for the error, it is no longer an issue.