Problem with installation of CULanes dataset

[Wakarox]

Hello, I have a problem with the installation of the CULanes dataset. When i do the "make" command, i have an "error opencv2/core/core.hpp : No such file or directory". I have all the libraries on my computer but i am not using conda. Can someone help me ?

[voldemortX]

@Wakarox Check if you have the c++ OpenCV on your system?

[Wakarox]

I am using cv2. I commented the line 21 as the tutorial said.

[voldemortX]

I am using cv2. I commented the line 21 as the tutorial said.

that is weird. you sure it is not python opencv?

[Wakarox]

I have the error on the include of the library. I did an installation of opencv and have the files in /usr/local/includes. Maybe i have a problem with the path of the files ?

[voldemortX]

@Wakarox Can you display opencv version by pkg-config --modversion opencv ?

[Wakarox]

OpenCV is not found. I don't know why ...

[voldemortX]

OpenCV is not found. I don't know why ...

It seems you may have a faulty installation of OpenCV. Are you using ubuntu, I think it can be installed via apt-get.

[Wakarox]

I have pkg-config --modversion opencv4 which returns 4.5.5.

[voldemortX]

I have 3.2.0 on my own machine and it works.

[Wakarox]

Not me. I don't know why :/

[voldemortX]

@Wakarox Perhaps try install opencv 2.x?

[Wakarox]

culane_list_convertor.py show something when running it?

[voldemortX]

culane_list_convertor.py show something when running it?

nope. no printing means success, as in all Linux commands.

[Wakarox]

My error code : pytorch-auto-drive/tools/culane_evaluation$ make CXX src/lane_compare.cpp In file included from include/lane_compare.hpp:4, from src/lane_compare.cpp:8: include/spline.hpp:6:10: fatal error: opencv2/core/core.hpp: Aucun fichier ou dossier de ce type 6 | #include <opencv2/core/core.hpp> | ^~~~~~~ compilation terminated. make: *** [Makefile:40 : build/src/lane_compare.o] Erreur 1

[voldemortX]

My error code : pytorch-auto-drive/tools/culane_evaluation$ make CXX src/lane_compare.cpp In file included from include/lane_compare.hpp:4, from src/lane_compare.cpp:8: include/spline.hpp:6:10: fatal error: opencv2/core/core.hpp: Aucun fichier ou dossier de ce type 6 | #include <opencv2/core/core.hpp> | ^~~~~~~ compilation terminated. make: *** [Makefile:40 : build/src/lane_compare.o] Erreur 1

I reproduced this error with a newly installed opencv4 from source. And I can make with adding CXXFLAGS += -I/usr/local/include/opencv4, for instance at Line 22. Or if you have other place for your includes, change the -I path.

[Wakarox]

The include is now good, i found something like you said yesterday. But now, i have a problem with the shared libraries.

CXX/LD evaluate /usr/bin/ld: build/src/evaluate.o: undefined reference to symbol '_ZN2cv6imreadERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEi' /usr/bin/ld : /lib/x86_64-linux-gnu/libopencv_imgcodecs.so.4.2 : erreur lors de l'ajout de symboles : DSO missing from command line collect2: error: ld returned 1 exit status make: *** [Makefile:44 : evaluate] Erreur 1

[voldemortX]

The include is now good, i found something like you said yesterday. But now, i have a problem with the shared libraries.

CXX/LD evaluate /usr/bin/ld: build/src/evaluate.o: undefined reference to symbol '_ZN2cv6imreadERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEi' /usr/bin/ld : /lib/x86_64-linux-gnu/libopencv_imgcodecs.so.4.2 : erreur lors de l'ajout de symboles : DSO missing from command line collect2: error: ld returned 1 exit status make: *** [Makefile:44 : evaluate] Erreur 1

As far as I know, this might be caused by installing multiple versions of opencv. For instance, when I install opencv3 from apt and sudo install the opencv4 zip, I see a similar error.

[Wakarox]

So you suggest me to unistall opencv, and reinstall it ?

[voldemortX]

In my case I uninstalled the apt one. Or you can find a way to link that .so specifically, I think it is already the last step for this build.

[Wakarox]

I tried to unistall it, it didn't work, i always have the same issue... I am searching a way to link that .so but all i tried didn't work too

[voldemortX]

@Wakarox I think you can search for opencv installation issues in general, since this probably is not specific to these codes. If you really can't make it work, you can try use culane_evaluation_py instead. I believe LaneATT have a full python version. But it slightly misalign with the cpp version.

[Wakarox]

I have a new error when i changed the Makefile deleting the 2 lines of condition:

CXX/LD evaluate /usr/bin/ld : build/src/lane_compare.o : dans la fonction « LaneCompare::get_lanesimilarity(std::vector<cv::Point, std::allocator<cv::Point > > const&, std::vector<cv::Point, std::allocator<cv::Point_ > > const&) » : lane_compare.cpp:(.text+0x56e) : référence indéfinie vers « cv::_InputArray::_InputArray(cv::MatExpr const&) » /usr/bin/ld : build/src/lane_compare.o : dans la fonction « cv::MatExpr::operator cv::Mat() const » : lane_compare.cpp:(.text._ZNK2cv7MatExprcvNS_3MatEEv[_ZNK2cv7MatExprcvNS_3MatEEv]+0x2c) : référence indéfinie vers « cv::Mat::Mat() » collect2: error: ld returned 1 exit status make: *** [Makefile:43 : evaluate] Erreur 1

[voldemortX]

@Wakarox At this point, I'd suggest you clean your environment of any opencv related stuff, delete the build dir in culane_evaluation. Then install opencv via the opencv documentation, and use the last step sudo make install to install it in /usr. If you installed opencv4 (their latest version), add the Line 22 like I did and it should work fine.

Alternatively, apt install libopencv-dev might also work but I don't really know.

[Wakarox]

That's good now, i found a solution (adding the library in the Makefile). Now, i am blocked with the config of lane detection.

mokhtarbouain@iris-Precision-Tower-7810:~/Documents/Python_scripts/pytorch-auto-drive$ python3 main_landet.py --config=/home/mokhtarbouain/Téléchargements/resnet18-5c106cde.pth Traceback (most recent call last): File "main_landet.py", line 11, in from utils.args import parse_arg_cfg, read_config, map_states, add_shortcuts, cmd_dict File "/home/mokhtarbouain/Documents/Python_scripts/pytorch-auto-drive/utils/init.py", line 4, in from . import models File "/home/mokhtarbouain/Documents/Python_scripts/pytorch-auto-drive/utils/models/init.py", line 16, in from .swin import SwinTransformer File "/home/mokhtarbouain/Documents/Python_scripts/pytorch-auto-drive/utils/models/swin.py", line 13, in from timm.models.layers import DropPath, to_2tuple, truncnormal File "/home/mokhtarbouain/.local/lib/python3.8/site-packages/timm/init.py", line 2, in from .models import create_model, list_models, is_model, list_modules, model_entrypoint, \ File "/home/mokhtarbouain/.local/lib/python3.8/site-packages/timm/models/init.py", line 1, in from .cspnet import * File "/home/mokhtarbouain/.local/lib/python3.8/site-packages/timm/models/cspnet.py", line 20, in from .helpers import build_model_with_cfg File "/home/mokhtarbouain/.local/lib/python3.8/site-packages/timm/models/helpers.py", line 17, in from .layers import Conv2dSame, Linear File "/home/mokhtarbouain/.local/lib/python3.8/site-packages/timm/models/layers/init.py", line 7, in from .cond_conv2d import CondConv2d, get_condconv_initializer File "/home/mokhtarbouain/.local/lib/python3.8/site-packages/timm/models/layers/cond_conv2d.py", line 16, in from .helpers import to_2tuple File "/home/mokhtarbouain/.local/lib/python3.8/site-packages/timm/models/layers/helpers.py", line 6, in from torch._six import container_abcs ImportError: cannot import name 'container_abcs' from 'torch._six' (/home/mokhtarbouain/.local/lib/python3.8/site-packages/torch/_six.py)

[voldemortX]

@Wakarox Good to know you made opencv work!

It seems your timm version doesn't support your pytorch version. Do you have pytorch > 1.8? It looks the same issue as in this one:

https://github.com/voldemortX/pytorch-auto-drive/issues/37

[Wakarox]

I have Pytorch 1.10.2.

import torch print(torch.version) 1.10.2+cu102

[voldemortX]

I have Pytorch 1.10.2.

import torch print(torch.version) 1.10.2+cu102

try 1.6 or 1.8 instead. And of course the corresponding torchvision version.

[Wakarox]

Done. Now, i have another error, i am not sure but i think it's mmcv. mokhtarbouain@iris-Precision-Tower-7810:~/Documents/Python_scripts/pytorch-auto-drive$ python3 main_landet.py --config=/home/mokhtarbouain/Téléchargements/resnet18-5c106cde.pth Traceback (most recent call last): File "main_landet.py", line 11, in from utils.args import parse_arg_cfg, read_config, map_states, add_shortcuts, cmd_dict File "/home/mokhtarbouain/Documents/Python_scripts/pytorch-auto-drive/utils/init.py", line 2, in from . import losses File "/home/mokhtarbouain/Documents/Python_scripts/pytorch-auto-drive/utils/losses/init.py", line 3, in from .hungarian_loss import HungarianLoss File "/home/mokhtarbouain/Documents/Python_scripts/pytorch-auto-drive/utils/losses/hungarian_loss.py", line 10, in from ..models.lane_detection import cubic_curve_with_projection File "/home/mokhtarbouain/Documents/Python_scripts/pytorch-auto-drive/utils/models/init.py", line 7, in from . import segmentation File "/home/mokhtarbouain/Documents/Python_scripts/pytorch-auto-drive/utils/models/segmentation/init.py", line 5, in from .erfnet import ERFNet File "/home/mokhtarbouain/Documents/Python_scripts/pytorch-auto-drive/utils/models/segmentation/erfnet.py", line 10, in from ..common_models import non_bottleneck_1d File "/home/mokhtarbouain/Documents/Python_scripts/pytorch-auto-drive/utils/models/common_models/init.py", line 1, in from .blocks import * File "/home/mokhtarbouain/Documents/Python_scripts/pytorch-auto-drive/utils/models/common_models/blocks/init.py", line 1, in from .inverted_residual import InvertedResidual, InvertedResidualV3 File "/home/mokhtarbouain/Documents/Python_scripts/pytorch-auto-drive/utils/models/common_models/blocks/inverted_residual.py", line 3, in from ..plugins import SELayer File "/home/mokhtarbouain/Documents/Python_scripts/pytorch-auto-drive/utils/models/common_models/plugins/init.py", line 6, in from .feature_flip_fusion import FeatureFlipFusion, DCN_v2_Ref File "/home/mokhtarbouain/Documents/Python_scripts/pytorch-auto-drive/utils/models/common_models/plugins/feature_flip_fusion.py", line 3, in from mmcv.ops import ModulatedDeformConv2d, modulated_deform_conv2d # DCNv2 File "/home/mokhtarbouain/.local/lib/python3.8/site-packages/mmcv/ops/init.py", line 2, in from .active_rotated_filter import active_rotated_filter File "/home/mokhtarbouain/.local/lib/python3.8/site-packages/mmcv/ops/active_rotated_filter.py", line 8, in ext_module = ext_loader.load_ext( File "/home/mokhtarbouain/.local/lib/python3.8/site-packages/mmcv/utils/ext_loader.py", line 13, in load_ext ext = importlib.import_module('mmcv.' + name) File "/usr/lib/python3.8/importlib/init.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) ImportError: /home/mokhtarbouain/.local/lib/python3.8/site-packages/mmcv/_ext.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZNK2at10TensorBase8data_ptrIdEEPT_v

[voldemortX]

@Wakarox Yes it should be mmcv, but I never encountered this before. Maybe try find it in the mmcv repo? However, are you installing things directly to your system (.local)? That is highly not recommended. Please try use conda if possible and delete the python in .local.

Not considering the python install issue, do check if your mmcv-full is matched with CUDA and pytorch. Did you install mmcv with torch 1.10 before downgrading to below 1.8? In that case, maybe mmcv needs reinstall also

[Wakarox]

I tried reinstalling mmcv but the "requirements already satisfied". I am not working on conda because i will try (after lauching it and testing it) to add it in a ROS environnement.

[voldemortX]

I tried reinstalling mmcv but the "requirements already satisfied". I am not working on conda because i will try (after lauching it and testing it) to add it in a ROS environnement.

If you are using pip, first pip uninstall mmcv-full, then install the correct version.

ROS sounds exciting!

[Wakarox]

Hello, I don't have any errors now, now, i want to use a pre-trained weight, but i didn't saw how to do it in the README. Can you explain me pls ?

[voldemortX]

Hello, I don't have any errors now, now, i want to use a pre-trained weight, but i didn't saw how to do it in the README. Can you explain me pls ?

You can download the pre-trained weights available from the model zoo, where you can see blue [model]() as links. And test with them by --checkpoint=<path to .pt file>

[Wakarox]

It is good, i can try to train, but the problem now is the setup, because my graphic card is a Quadro M2000. I had an error when training : RuntimeError: CUDA out of memory. Tried to allocate 142.00 MiB (GPU 0; 3.94 GiB total capacity; 3.05 GiB already allocated; 95.50 MiB free; 3.13 GiB reserved in total by PyTorch) Can i do something to success the train ?

[voldemortX]

@Wakarox You should start from batch size 1 (better with mixed precision if possible), gradually increase it until your gpu memory is full. Then adjust learning rate, usually by linear (e. g. bs 20 to 4, lr should be divoded by 5) or sqrt (same example divide by sqrt(5)).

If even bs 1 fails, then probably you can't train this model with ease.

FYI, you can adjust lr and bs in config files or simply by --lr and --batch-size

[Wakarox]

Hello @voldemortX , That's good now, i want to visualize the result in real time, it's possible with your code ?

[voldemortX]

Hello @voldemortX , That's good now, i want to visualize the result in real time, it's possible with your code ?

By in real-time do you mean camera inputs? If so we do not support that currently. But we do support visualizing a folder of images or video, it would not be hard to adapt to camera input.

You can refer to: https://www.github.com/voldemortX/pytorch-auto-drive/tree/master/docs/VISUALIZATION.md

[Wakarox]

In ROS, i use rosbag to play a record of our vehicle. Now, i want to use these bags with your program.

[voldemortX]

@Wakarox Sorry I don't know much about rosbag, but if the input format does not match the current visualization format, you'll have to adjust the APIs yourself to connect these programs. Perhaps you will need to write a new vis Runner or expose some API functions from PytorchAutoDrive based on your use case.

If you made some addtions that would benefit ros integration, I'd be more than happy to take pull requests.

[Wakarox]

I will try to use only one config pretrained (resnet18_culane). I need to filter the files for this config. What files i will need to keep ?

[voldemortX]

I will try to use only one config pretrained (resnet18_culane). I need to filter the files for this config. What files i will need to keep ?

I think you need the entry file, transforms and model definitions (including the config). And lane runner & dataset if training or testing are still required. In fact, best way is find the important files and keep others through import.

[Wakarox]

Hello @voldemortX , I have a question for visualization, can i use just a test image without keypoints txt for checking the accuracy of the program ?

Edit : That's good, i just forgot to use --pred on my image in my line command. Now, let's go to try to add it in ROS ! Now, next question : can i have the equation of the lane detected by the code ?

[voldemortX]

Hello @voldemortX , I have a question for visualization, can i use just a test image without keypoints txt for checking the accuracy of the program ?

Edit : That's good, i just forgot to use --pred on my image in my line command. Now, let's go to try to add it in ROS ! Now, next question : can i have the equation of the lane detected by the code ?

by equation do you mean a curve equation? In that case LSTR have some sort of polynomial and BezierLaneNet has a Bezier curve at network output, which you can insert codes to save. Other methods do not have such things and you will need to fit a curve equation by yourself.

[Wakarox]

Hello @voldemortX Where can I find the coordinates of the points ? I am using a model without curve equation (resnet18_baseline_tusimple), i will add it myself.

[voldemortX]

@Wakarox Perhaps get them somewhere around here? Before they were drawn. https://github.com/voldemortX/pytorch-auto-drive/blob/76e83301e3ef1f245f9bc116566c017a816e229c/utils/vis_utils.py#L93

If you are using dataset inference, you can find things in ./output

[Wakarox]

@Wakarox Perhaps get them somewhere around here? Before they were drawn.

https://github.com/voldemortX/pytorch-auto-drive/blob/76e83301e3ef1f245f9bc116566c017a816e229c/utils/vis_utils.py#L93

If you are using dataset inference, you can find things in ./output

Hello, I am trying to add your code in ROS, but i need to know if all the parameters in this function are necessary. I want to use my own images with the predict function, but i want to keep the dataset module for training and testing on my own dataset. I modified some files to reduce the quantity of code, but it's not enough yet, and i am a little lost. If you need to see my change, i will send you my code. Best Regards,

[voldemortX]

@Wakarox Can I maybe see your change in git? I could suggest what more could be deleted when I go through them.

[Wakarox]

I just created a Git for this application : https://github.com/Wakarox/pytorch_auto_drive_ros I deleted a lot of semantic segmentation code and i just begin to modify lane detection. For the moment, i didn't push all the ROS environment, i will do it later.

[voldemortX]

@Wakarox It would be best if it can include the git history of pytorch_auto_drive so it can indicate which files are deleted already. Nevertheless, I will probably go through them before the weekend.

[Wakarox]

Hello, I forgot to push the downloaded version, but all you have on GitHub is for lane detection. I am trying to add your code on ROS, but i have errors that i am trying to debug.