search

Tobias-Fischer / rt_gene

RT-GENE: Real-Time Eye Gaze and Blink Estimation in Natural Environments
http://www.imperial.ac.uk/personal-robotics
Other
368 stars 68 forks source link

Problems related to ROS TCP/IP #112

Closed kanos-taeyeon closed 2 years ago

kanos-taeyeon commented 2 years ago

I am trying to use the code with interest in how gaze estimation is performed through RT-GENE. I've tried a lot through the guidelines, but I can't solve this problem. Could you help me with this problem?

It's an error that I want to solve. Running roslaunchrt_gene estimate_gaze.launch yields the following output values.

tae@tae:~$ roslaunch rt_gene estimate_gaze.launch ... logging to /home/tae/.ros/log/8c54b444-4d2c-11ec-8884-7c10c926a764/roslaunch-tae-10590.log Checking log directory for disk usage. This may take a while. Press Ctrl-C to interrupt Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://192.168.0.83:41503/

SUMMARY

PARAMETERS

NODES /gaze/ gaze_estimate_gaze_twoeyes (rt_gene/estimate_gaze.py) gaze_extract_landmarks_new (rt_gene/extract_landmarks_node.py)

ROS_MASTER_URI=http://192.168.0.83:11311

process[gaze/gaze_extract_landmarks_new-1]: started with pid [10620] process[gaze/gaze_estimate_gaze_twoeyes-2]: started with pid [10621] Traceback (most recent call last): File "/home/tae/catkin_ws/src/rt_gene/rt_gene/scripts/estimate_gaze.py", line 182, in [os.path.join(rospkg.RosPack().get_path("rt_gene"), model_file) for model_file in rospy.get_param("~model_files")]) File "/home/tae/catkin_ws/src/rt_gene/rt_gene/scripts/estimate_gaze.py", line 48, in init from rt_gene.estimate_gaze_tensorflow import GazeEstimator File "/home/tae/catkin_ws/src/rt_gene/rt_gene/src/rt_gene/estimate_gaze_tensorflow.py", line 4, in import tensorflow as tf File "/home/tae/.local/lib/python2.7/site-packages/tensorflow/init.py", line 28, in from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-import File "/home/tae/.local/lib/python2.7/site-packages/tensorflow/python/init.py", line 52, in from tensorflow.core.framework.graph_pb2 import * File "/home/tae/.local/lib/python2.7/site-packages/tensorflow/core/framework/graph_pb2.py", line 7, in from google.protobuf import descriptor as _descriptor File "/home/tae/.local/lib/python2.7/site-packages/google/protobuf/descriptor.py", line 113 class DescriptorBase(metaclass=DescriptorMetaclass):

If you look at the logs, the following logs are stored.

[rospy.internal][ERROR] 2021-11-24 13:35:04,520: Unable to initiate TCP/IP socket to 192.168.0.83:46193 (http://192.168.0.83:41085/): Traceback (most recent call last):

File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_base.py", line 562, in connect

self.read_header()

File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_base.py", line 657, in read_header

self._validate_header(read_ros_handshake_header(sock, self.read_buff, self.protocol.buff_size))

File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_base.py", line 600, in _validate_header

raise TransportInitError("remote error reported: %s"%header['error'])

TransportInitError: remote error reported: node shutting down

roslaunch rt_gene start_webcam.Launch I think it's done normally.

tae@tae:~$ roslaunch rt_gene start_webcam.launch ... logging to /home/tae/.ros/log/8c54b444-4d2c-11ec-8884-7c10c926a764/roslaunch-tae-10427.log Checking log directory for disk usage. This may take a while. Press Ctrl-C to interrupt Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://192.168.0.83:39221/

SUMMARY

PARAMETERS

NODES / ros_static_transform_publisher (tf2_ros/static_transform_publisher) uvc_camera (uvc_camera/uvc_camera_node)

auto-starting new master process[master]: started with pid [10465] ROS_MASTER_URI=http://192.168.0.83:11311

setting /run_id to 8c54b444-4d2c-11ec-8884-7c10c926a764 process[rosout-1]: started with pid [10491] started core service [/rosout] process[ros_static_transform_publisher-2]: started with pid [10498] process[uvc_camera-3]: started with pid [10499] [ INFO] [1637761427.596010349]: camera calibration URL: file:///home/tae/catkin_ws/src/rt_gene/rt_gene/webcam_configs/webcam_blue_26010230.yaml opening /dev/video0 pixfmt 0 = 'YUYV' desc = 'YUYV 4:2:2' discrete: 640x480: 1/30 discrete: 352x288: 1/30 discrete: 320x240: 1/30 discrete: 176x144: 1/30 discrete: 160x120: 1/30 pixfmt 1 = 'MJPG' desc = 'Motion-JPEG' discrete: 640x480: 1/30 discrete: 352x288: 1/30 discrete: 320x240: 1/30 discrete: 176x144: 1/30 discrete: 160x120: 1/30 int (Brightness, 0, id = 980900): -16 to 16 (1) int (Contrast, 0, id = 980901): 1 to 32 (1) int (Saturation, 0, id = 980902): 0 to 60 (1) int (Hue, 0, id = 980903): -45 to 45 (1) bool (White Balance Temperature, Auto, 0, id = 98090c): 0 to 1 (1) int (Gamma, 0, id = 980910): 100 to 200 (10) menu (Power Line Frequency, 0, id = 980918): 0 to 2 (1) 0: Disabled 1: 50 Hz 2: 60 Hz int (White Balance Temperature, 16, id = 98091a): 2800 to 6500 (1850) int (Sharpness, 0, id = 98091b): 0 to 15 (1) int (Backlight Compensation, 0, id = 98091c): 0 to 2 (1) menu (Exposure, Auto, 0, id = 9a0901): 0 to 3 (1) int (Exposure (Absolute), 16, id = 9a0902): 2 to 20 (2) bool (Exposure, Auto Priority, 0, id = 9a0903): 0 to 1 (1) int (Pan (Absolute), 0, id = 9a0908): -36000 to 36000 (7200) int (Tilt (Absolute), 0, id = 9a0909): -36000 to 36000 (7200) Setting auto_focus is not supported Setting focus_absolute is not supported

To solve this problem, can I get advice on what I should try?

Tobias-Fischer commented 2 years ago

Hi,

Could you please confirm that a simple import tensorflow works in your python environment? The traceback seems to point to some issues in your tensorflow installation.

kanos-taeyeon commented 2 years ago

Thank you for your help.

There was a package in the tensorflow that was incompatible with the version of python2.7, and I downgrade this package to solve this problem. (pip install protobuf==3.17.3)

However, gaze estimation does not work in the webcam environment yet. Can you tell me the output values of the 3 launchs? (roscore, roslaunch rt_gene start_webcam.launch, roslaunch rt_gene estimate_gaze.launch) I'm going to find a problem with my environment by comparing it with my output value.

Tobias-Fischer commented 2 years ago

@ahmed-alhindawi - could you please help out? I don't have an installation around ..

kanos-taeyeon commented 2 years ago

Currently, I think the launching file works without errors.

However, real-time gaze estimation on the first page is not working. (https://github.com/Tobias-Fischer/rt_gene/blob/master/assets/dataset_video.gif)

The webcam screen isn't coming out.

What should I check? Could you help me with this part?

I am currently using Microsoft webcam.

ahmed-alhindawi commented 2 years ago

Sorry for the late reply.

Could you check that your webcam works under ROS and that the output topics are being routed to the correct inputs required by RT-GENE?

I've checked on a fresh-install in a conda-environment and everything is working as is.

I'll be closing this issue but feel free to continue using it and I'll try to help along

kanos-taeyeon commented 2 years ago

Thank you for your help. What output value should I provide?

If possible, could you provide more information about the process below?

Requirements for live gaze estimation (webcam / RGB only) -Calibrate your camera (http://wiki.ros.org/camera_calibration). -Save the resulting *.yaml file to $(rospack find rt_gene)/webcam_configs/. -Change the entry for the camera_info_url in the $(rospack find rt_gene)/launch/start_webcam.launch file.