Laser_mapping and laser_loc are broken

[pariaspe]

Laser_mapping and laser_loc are broken:

[ERROR] [1681486219.098906868]: [STDR_PARSER] Failed to load file '/catkin_ws/src/stdr_simulator/stdr_resources/resources/robots/amigobot.xml'
Error was 'Failed to open file'
If error was 'Error reading end tag' you have a malformed xml file
[robot_spawn_664947ce1ef7_110_3002388729644700876-5] process has died [pid 212, exit code 255, cmd /catkin_ws/devel/lib/stdr_robot/robot_handler add /catkin_ws/src/stdr_simulator/stdr_resources/resources/robots/amigobot.xml 4 8 0 __name:=robot_spawn_664947ce1ef7_110_3002388729644700876 __log:=/root/.ros/log/430e9e8a-dad9-11ed-a20e-0242ac110002/robot_spawn_664947ce1ef7_110_3002388729644700876-5.log].
log file: /root/.ros/log/430e9e8a-dad9-11ed-a20e-0242ac110002/robot_spawn_664947ce1ef7_110_3002388729644700876-5*.log

Also at laser_loc:

Exception in thread Thread-11:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/RoboticsAcademy/exercises/static/exercises/laser_loc/web-template/gui.py", line 184, in run
    self.gui.update()
  File "/RoboticsAcademy/exercises/static/exercises/laser_loc/web-template/gui.py", line 78, in update
    self.payload["pos_vertices"], self.payload["sonar_sensor"] = self.map.setSonarValues()
  File "/RoboticsAcademy/exercises/static/exercises/laser_loc/web-template/map.py", line 198, in setSonarValues
    hipotenusa = distance/(math.cos(cone/2))
TypeError: unsupported operand type(s) for /: 'list' and 'float'

Originally posted by @pariaspe in https://github.com/JdeRobot/RoboticsAcademy/issues/1984#issuecomment-1508805066

[pariaspe]

Laser mapping:

[Bhardwaj-Himanshu]

Hi @pariaspe,

Successfully reproduced the issue,here-

Could you elaborate on how one can start working on this issue?

After looking at the issue I think This has something to do with catkin_dependencies or catkin_config_file or even maybe ros not installed on the system, which could not be the case as we are trying and testing other exercises which make use of it and work perfectly

Let me know about your thoughts on this and if you are working on anything? Thanks

[pariaspe]

Hey @Bhardwaj-Himanshu ,

I'm not working on it right now, I ain't time. But I have an idea on what it might be happening.

There are al least two different issues:

1. amigobot.xml malformed. Take a look at the file, it may have a unclosed tag or the some XML specification may be missing.
2. On map.py there is a python mistake where a list is divided by a float which is not supported.

[Bhardwaj-Himanshu]

Thanks for the feedback @pariaspe,

On taking a look at map.py It has these many division processes going on-

#these were int values but I am unsure about self.RTz command
RTz = self.RTz(pi/2, 50, 70, 0)
base = radius/2
alfa = math.asin((base)/radius)
hipotenusa = distance/(math.cos(cone/2))

# I suspect it is from division below this comment
orientation = self.sonares[i][2] - cone/2
orientation = self.sonares[i][2] + cone/2
orientation = self.sonares[i][2] + cone/2
orientation = self.sonares[i][2] - cone/2

On taking a look at amigobot.xml, which I could not find the file but found, amigobot_without_gui.launch, amigobot_launch The syntax of both feels good to me as code editor didn't raise any red underlines as well So it is most probably some XML specification missing!

Let me know your thoughts on this and how can I proceed onto next?

[pariaspe]

The traceback tells you where the error is happening:

hipotenusa = distance/(math.cos(cone/2))

On taking a look at amigobot.xml, which I could not find the file but found, amigobot_without_gui.launch, amigobot_launch The syntax of both feels good to me as code editor didn't raise any red underlines as well So it is most probably some XML specification missing!

Those files are cloned from here. I've just realized that amigobot.xml file is missing (check it on the repo), raising the error "Failed to open file". Could you try to launch the exercise without line 12 in amigobot_without_gui.launch?

[Bhardwaj-Himanshu]

Thanks for all the clues @pariaspe,

After your suggestions, I commented out the

<!-- <node name="$(anon robot_spawn)" pkg="stdr_robot" type="robot_handler" args="add $(find stdr_resources)/resources/robots/amigobot.xml 4 8 0"/>-->

from both the files amigobot_without_gui.launch and amigobot_launch, I know you said about just one, but just testing in the moment.

I then build both the BASE and RADI images of the docker container and named it as test. Now before I run the docker container using-

docker run --rm -it -p 8000:8000 -p 2303:2303 -p 1905:1905 -p 8765:8765 -p 6080:6080 -p 1108:1108 -p 7163:7163 test_image_id

I try to install the requirements.txt using pip install -r requirements.txt which always ends on the error below as-

ERROR: Could not find a version that satisfies the requirement catkin-pkg-modules==0.5.2 (from versions: 66.0.0, 66.0.2)
ERROR: No matching distribution found for catkin-pkg-modules==0.5.2

Which denotes--> I have version 66.0.2 installed on system but I need 0.5.2--> so when I manually tweak the requirements.txt file catkin-pkg-modules==0.5.2 it again ends on error.

Could you tell the possible reason why?

[pariaspe]

Hey @Bhardwaj-Himanshu , Everything went well while running? You don't need to install requirements.txt it is done while building the RADI.

[Bhardwaj-Himanshu]

Yep, the both the images build without any raising any errors. Here it is, just made it some hours ago-

REPOSITORY                       TAG       IMAGE ID       CREATED         SIZE
jderobot/robotics-academy        test      29581ccacfa4   15 hours ago    12GB

Let me know, if you want me to attach the log file for it, and what do you think about the issue!

[pariaspe]

So, where is the issue? You just have to run that docker image and see if the error persists

[Bhardwaj-Himanshu]

Hmm, I think I am stuck on installing django or other bootstrap dependencies or libraries.

Here is the output of django version checker command-

python -m django --version
4.2.1

and here is a log I am running into when I run docker run -p 8000:8000 test_image_id INPUT

sudo docker run --rm -it -p 8000:8000 -p 2303:2303 -p 1905:1905 -p 8765:8765 -p 6080:6080 -p 1108:1108 -p 7163:7163 29581ccacfa4            

OUTPUT

BASE_DIR /RoboticsAcademy
2023-05-03 18:09:10,938 [MainThread  ] [INFO ] (root)  Starting RAM consumer in <websocket_server.websocket_server.WebsocketServer object at 0x7fac0c459f40>:7163
2023-05-03 18:09:10,938 [MainThread  ] [INFO ] (websocket_server.websocket_server)  Listening on port 7163 for clients..
2023-05-03 18:09:10,939 [MainThread  ] [INFO ] (websocket_server.websocket_server)  Starting WebsocketServer on thread Thread-1.
BASE_DIR /RoboticsAcademy
Watching for file changes with StatReloader
Performing system checks...

Exception in thread django-main-thread:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/django/template/utils.py", line 69, in __getitem__
    return self._engines[alias]
KeyError: 'django'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/django/template/backends/django.py", line 128, in get_package_libraries
    module = import_module(entry[1])
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 848, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/local/lib/python3.8/dist-packages/rest_framework/templatetags/rest_framework.py", line 12, in <module>
    from rest_framework.renderers import HTMLFormRenderer
  File "/usr/local/lib/python3.8/dist-packages/rest_framework/renderers.py", line 17, in <module>
    from django.http.multipartparser import parse_header
ImportError: cannot import name 'parse_header' from 'django.http.multipartparser' (/usr/local/lib/python3.8/dist-packages/django/http/multipartparser.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.8/dist-packages/django/utils/autoreload.py", line 64, in wrapper
    fn(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/django/core/management/commands/runserver.py", line 133, in inner_run
    self.check(display_num_errors=True)
  File "/usr/local/lib/python3.8/dist-packages/django/core/management/base.py", line 485, in check
    all_issues = checks.run_checks(
  File "/usr/local/lib/python3.8/dist-packages/django/core/checks/registry.py", line 88, in run_checks
    new_errors = check(app_configs=app_configs, databases=databases)
  File "/usr/local/lib/python3.8/dist-packages/django/contrib/admin/checks.py", line 78, in check_dependencies
    for engine in engines.all():
  File "/usr/local/lib/python3.8/dist-packages/django/template/utils.py", line 94, in all
    return [self[alias] for alias in self]
  File "/usr/local/lib/python3.8/dist-packages/django/template/utils.py", line 94, in <listcomp>
    return [self[alias] for alias in self]
  File "/usr/local/lib/python3.8/dist-packages/django/template/utils.py", line 85, in __getitem__
    engine = engine_cls(params)
  File "/usr/local/lib/python3.8/dist-packages/django/template/backends/django.py", line 24, in __init__
    options["libraries"] = self.get_templatetag_libraries(libraries)
  File "/usr/local/lib/python3.8/dist-packages/django/template/backends/django.py", line 42, in get_templatetag_libraries
    libraries = get_installed_libraries()
  File "/usr/local/lib/python3.8/dist-packages/django/template/backends/django.py", line 116, in get_installed_libraries
    return {
  File "/usr/local/lib/python3.8/dist-packages/django/template/backends/django.py", line 116, in <dictcomp>
    return {
  File "/usr/local/lib/python3.8/dist-packages/django/template/backends/django.py", line 105, in get_template_tag_modules
    for name in get_package_libraries(pkg):
  File "/usr/local/lib/python3.8/dist-packages/django/template/backends/django.py", line 130, in get_package_libraries
    raise InvalidTemplateLibrary(
django.template.library.InvalidTemplateLibrary: Invalid template library specified. ImportError raised when trying to load 'rest_framework.templatetags.rest_framework': cannot import name 'parse_header' from 'django.http.multipartparser' (/usr/local/lib/python3.8/dist-packages/django/http/multipartparser.py)
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/django/template/utils.py", line 69, in __getitem__
    return self._engines[alias]
KeyError: 'django'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/django/template/backends/django.py", line 128, in get_package_libraries
    module = import_module(entry[1])
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 848, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/local/lib/python3.8/dist-packages/rest_framework/templatetags/rest_framework.py", line 12, in <module>
    from rest_framework.renderers import HTMLFormRenderer
  File "/usr/local/lib/python3.8/dist-packages/rest_framework/renderers.py", line 17, in <module>
    from django.http.multipartparser import parse_header
ImportError: cannot import name 'parse_header' from 'django.http.multipartparser' (/usr/local/lib/python3.8/dist-packages/django/http/multipartparser.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "RoboticsAcademy/manage.py", line 22, in <module>
    main()
  File "RoboticsAcademy/manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.8/dist-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.8/dist-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.8/dist-packages/django/core/management/base.py", line 412, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.8/dist-packages/django/core/management/commands/runserver.py", line 74, in execute
    super().execute(*args, **options)
  File "/usr/local/lib/python3.8/dist-packages/django/core/management/base.py", line 458, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.8/dist-packages/django/core/management/commands/runserver.py", line 111, in handle
    self.run(**options)
  File "/usr/local/lib/python3.8/dist-packages/django/core/management/commands/runserver.py", line 118, in run
    autoreload.run_with_reloader(self.inner_run, **options)
  File "/usr/local/lib/python3.8/dist-packages/django/utils/autoreload.py", line 671, in run_with_reloader
    start_django(reloader, main_func, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/django/utils/autoreload.py", line 660, in start_django
    reloader.run(django_main_thread)
  File "/usr/local/lib/python3.8/dist-packages/django/utils/autoreload.py", line 343, in run
    autoreload_started.send(sender=self)
  File "/usr/local/lib/python3.8/dist-packages/django/dispatch/dispatcher.py", line 176, in send
    return [
  File "/usr/local/lib/python3.8/dist-packages/django/dispatch/dispatcher.py", line 177, in <listcomp>
    (receiver, receiver(signal=self, sender=sender, **named))
  File "/usr/local/lib/python3.8/dist-packages/django/template/autoreload.py", line 43, in watch_for_template_changes
    for directory in get_template_directories():
  File "/usr/local/lib/python3.8/dist-packages/django/template/autoreload.py", line 16, in get_template_directories
    for backend in engines.all():
  File "/usr/local/lib/python3.8/dist-packages/django/template/utils.py", line 94, in all
    return [self[alias] for alias in self]
  File "/usr/local/lib/python3.8/dist-packages/django/template/utils.py", line 94, in <listcomp>
    return [self[alias] for alias in self]
  File "/usr/local/lib/python3.8/dist-packages/django/template/utils.py", line 85, in __getitem__
    engine = engine_cls(params)
  File "/usr/local/lib/python3.8/dist-packages/django/template/backends/django.py", line 24, in __init__
    options["libraries"] = self.get_templatetag_libraries(libraries)
  File "/usr/local/lib/python3.8/dist-packages/django/template/backends/django.py", line 42, in get_templatetag_libraries
    libraries = get_installed_libraries()
  File "/usr/local/lib/python3.8/dist-packages/django/template/backends/django.py", line 116, in get_installed_libraries
    return {
  File "/usr/local/lib/python3.8/dist-packages/django/template/backends/django.py", line 116, in <dictcomp>
    return {
  File "/usr/local/lib/python3.8/dist-packages/django/template/backends/django.py", line 105, in get_template_tag_modules
    for name in get_package_libraries(pkg):
  File "/usr/local/lib/python3.8/dist-packages/django/template/backends/django.py", line 130, in get_package_libraries
    raise InvalidTemplateLibrary(
django.template.library.InvalidTemplateLibrary: Invalid template library specified. ImportError raised when trying to load 'rest_framework.templatetags.rest_framework': cannot import name 'parse_header' from 'django.http.multipartparser' (/usr/local/lib/python3.8/dist-packages/django/http/multipartparser.py)

and here is a list of installed images on machine as well-

sudo docker images
REPOSITORY                       TAG       IMAGE ID       CREATED         SIZE
jderobot/robotics-academy        test      29581ccacfa4   16 hours ago    12GB
jderobot/robotics-applications   base      79c116bb3333   3 days ago      8.59GB
hello-world                      latest    feb5d9fea6a5   19 months ago   13.3kB

What I could identify that the system is missing, django, bootstrap and rest frameworks respectively, but there is django installed on the system! Could be an error on the machine side as well, but I use this cloud VM instance for every docker image build, as I am bottlenecked by memory on my local machine.

I have tried running jderobot/roboticsacademy:latest a few days back and it worked fine! Let me know your thoughts on this!

[pariaspe]

Hi, it seems that your BASE was not well generated. That dependency missing is installed here. So, make sure that your dockerfile have that line.

[Bhardwaj-Himanshu]

Hi, I am attaching a screenshot of what did dockerfile.base look like before-

So the line was already added, but I'll try making the base image again tommorow most probably, power cut issues;(

[pariaspe]

Yes, that line is the one I am referring to. Ok, no problem, no need to hurry :+1:

[Bhardwaj-Himanshu]

Totally skipped to look at the missing django=="version" in the above picture, thanks for pointing it out to me as well!

Also, for today I know I could be asking something very stupid, but I can't seem to find the scripts folder in the roboticsacademy cloned directory from github. Did a fresh clone after yesterday, and I know this could be stupid, but I am totally missing on did we do something to build the scripts folder?

and I think both the RADI as well as BASE image should be fine after making the change-

RUN pip3 install websocket_server posix-ipc django==4.1.7 djangorestframework==3.13.1 django-webpack-loader==1.5.0 django-cors-headers==3.14.0

[pariaspe]

Yes, we are restructuring a bit the repositories before gsoc23 kicks off. You can find the Dockerfiles here. You can build the RADI like always.

[Bhardwaj-Himanshu]

Ok pariaspe, we have a good news..........................that "exercise is still broken"->sorry for the PJ here!

I commented out the line 12 as mentioned by you from both the files amigobot.launch and amigobot_without_gui- and the error output for both laser_loc and laser mapping looks like this-

The BASE and RADI image build were both successful, and I do have one more doubt, do we have to commit changes before making the Images, just saving the file should be okay right?

Let me know what do you think about this-

Also to the GSOC point, my portfolio was rejected so if you could provide me the selected proposals or the profile of people who are selected that would be very helpful for me!

Also, keeping the gsoc tag aside, If I could contribute to the "contaniersing of visual circuit in any way" as I am interested in devops stuff, how things break and work! I know this won't be allowed as this could hinder the progress meter of the candidate,but looking as an open source organisation if I could would matter a lot to me! Thanks

[pariaspe]

Yes, you have to commit the changes and clone a version of the repository (your forked repo or any branch) with them on the RADI. If not, you will be not modifying files inside the docker.

We will announce them soon on twitter :+1: About the contribution, better ask the mentors of that project or open a discussion explaining your proposal

[Bhardwaj-Himanshu]

Ah, I'll give it a shot after commiting the changes as well then, but do I need to clone it again? Cause Saving the changes first + then commiting them should work right?

Also, I'll keep an eye out for twitter too then, thanks👍

[pariaspe]

Are you talking about git commit or docker commit? You can do both, but I was talking about git commit the changes, push them on your forked repo and clone it inside the RADI while building the Dockerfile.

[Bhardwaj-Himanshu]

Yep I am talking about the git commit exactly, so I should first commit, push and then try to build the RADI image. Gotcha!

[Bhardwaj-Himanshu]

Ah, I think it's been a long time! Pardon me had a lot going on my side, but active back again!--just moving homes and exams!

I am now making the RADI/BASE image for this issue, and will update you by evening! Thanks @pariaspe

So here is the update as I keep on switching distros and machines, the build error now ends on-

Let me know your thoughts on this, or should I physically download a copy of roboticsacademy:latest docker image from docker store before building the RADI image or BASE infact for that matter!

[pariaspe]

Hey @Bhardwaj-Himanshu , things have changed a lot in these last two weeks. We split the repository, so Dockerfiles are now also split. Take a look at this guide to build new RADI.

[Bhardwaj-Himanshu]

➜ scripts git:(#2008) sudo ./build.sh "#2008" noetic-devel main noetic test

Running this for a branch name gives me a "invalid use of -t flag error!"

[pariaspe]

Hey, use master branch and choose an alphanumeric tag name

[pawanw17]

Hi @Bhardwaj-Himanshu the script and its usage has been updated, use the latest master, and this should help:

https://github.com/JdeRobot/RoboticsAcademy/blob/master/docs/generate_a_mini_radi.md#usage

[AADI-234]

hello Is this issue resolved or not I'm new to github i want to contribute by solving issues related to DL, computer vision task and python. Can someone help me out in this @pawanw17 OR @Bhardwaj-Himanshu

[pariaspe]

Hi @AADI-234 , First step will be testing if the exercise has still the issue or not. Try to download latest RADI (or build a local miniRADI) and running the exercise.

[jmplaza]

Hi @AADI-234,

this issue is still open. Those exercises are broken. They are not related to DL neither Computer Vision. They are based on STDR simulator, not in Gazebo. Have you checked their user documentation? (laser Mapping, laser_loc)

In order to effectively contribute to RoboticsAcademy, first start using it :-) For instance the exercises FollowLine, ObstacleAvoidance, VacuumCleaner... And study the source code, feel free to ask here (or in Discussions) any arising question!. Then you will become familiar with its software base and you will be closer to create your own Pull Requests solving the issues.

You will find more context in this paper.

Cheers,

[damodardatta]

Hi, i'm not able to find laser Mapping, laser_loc in the latest RADI image list of exercises. Do i need to look for someother RADI image? or am i going wrong somewhere?

[pariaspe]

They might have been hidden since they were not working. You can make them available again by creating new entries to the database. Take a look at this guide.

[jmplaza]

Yes, they are deprecated. They both are based in the STDR Simulator and are now quite outdated with the latest frontend, REACT based. They require a major refactoring to be updated. Some information about them is available here (in Spanish, sorry).

And currently we are focused on the migration to RoboticsAcademy 5.2 for the core list exercises.