search

OpenPTrack / open_ptrack_v2

OPT v2 "Gnocchi" - adding object tracking and pose recognition
BSD 3-Clause "New" or "Revised" License
220 stars 65 forks source link

Is it possible to run the code in a single computer? #71

Closed JimZhou-001 closed 5 years ago

JimZhou-001 commented 5 years ago

Hi!

Thank you for providing such a great program.

Is it possible to run the code in a single computer?

Looking forward to your reply.

jburkeucla commented 5 years ago

Hi Jim, Yes, you can run detection and tracking (and actually all modules) on a single computer as long as it has sufficient GPU resources. The single camera examples are a good place to start- for example, see https://github.com/OpenPTrack/open_ptrack_v2/wiki/Person-Single-Camera

Jeff

From: Jim Zhou notifications@github.com Reply-To: OpenPTrack/open_ptrack_v2 reply@reply.github.com Date: Sunday, July 14, 2019 at 9:36 PM To: OpenPTrack/open_ptrack_v2 open_ptrack_v2@noreply.github.com Cc: Subscribed subscribed@noreply.github.com Subject: [OpenPTrack/open_ptrack_v2] Is it possible to run the code in a single computer? (#71)

Hi!

Thank you for providing such a great program.

Is it possible to run the code in a single computer?

Looking forward to your reply.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/OpenPTrack/open_ptrack_v2/issues/71?email_source=notifications&email_token=AADFGR6535B5DPEEVPAHPGTP7P5F7A5CNFSM4IDTDSV2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4G7EFBSA, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AADFGR7HGO6RKR2UUIQJHG3P7P5F7ANCNFSM4IDTDSVQ.

JimZhou-001 commented 5 years ago

@jburkeucla Thank you very much. I mean several Kinect2 cameras connected on one computer. Is there any simple tutorial?

jburkeucla commented 5 years ago

Multiple Kinect2s is possible if you have the GPU resources; you have to manually specify the imager serial number in the detection startup. I don’t believe we have a tutorial but will check.

From: Jim Zhou notifications@github.com Reply-To: OpenPTrack/open_ptrack_v2 reply@reply.github.com Date: Sunday, July 14, 2019 at 10:41 PM To: OpenPTrack/open_ptrack_v2 open_ptrack_v2@noreply.github.com Cc: Jeff Burke jburke@remap.ucla.edu, Mention mention@noreply.github.com Subject: Re: [OpenPTrack/open_ptrack_v2] Is it possible to run the code in a single computer? (#71)

@jburkeuclahttps://github.com/jburkeucla Thank you very much. I mean several Kinect2 cameras connected on one computer. Is there any simple tutorial?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/OpenPTrack/open_ptrack_v2/issues/71?email_source=notifications&email_token=AADFGR4QUDRDUUSTFQ6AY4DP7QEZ5A5CNFSM4IDTDSV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZ4YE3I#issuecomment-511279725, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AADFGR2APMERZQNYM3RZD3DP7QEZ5ANCNFSM4IDTDSVQ.

JimZhou-001 commented 5 years ago

Hello!

I tried to install it on Ubuntu 16.04, NVIDIA RTX 2080, but failed. Is it because RTX 2080 doesn't support nvidia-384 driver?

jburkeucla commented 5 years ago

The 2080 should be supported – what does nvidia-smi return? Are you trying to run from a docker image or source? How is it failing?

From: Jim Zhou notifications@github.com Reply-To: OpenPTrack/open_ptrack_v2 reply@reply.github.com Date: Tuesday, July 16, 2019 at 8:10 PM To: OpenPTrack/open_ptrack_v2 open_ptrack_v2@noreply.github.com Cc: Jeff Burke jburke@remap.ucla.edu, Mention mention@noreply.github.com Subject: Re: [OpenPTrack/open_ptrack_v2] Is it possible to run the code in a single computer? (#71)

Hello!

I tyied to install it on Ubuntu 16.04, NVIDIA RTX 2080, but failed. Is it because RTX 2080 doesn't support nvidia-384 driver?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/OpenPTrack/open_ptrack_v2/issues/71?email_source=notifications&email_token=AADFGR2MNJEPKM2WPYRWD5DP72EQDA5CNFSM4IDTDSV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2C4PPQ#issuecomment-512083902, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AADFGR7RY3TU6YOAKPIAPPLP72EQDANCNFSM4IDTDSVQ.

JimZhou-001 commented 5 years ago

Thank you.

Now I have built the software environment successfully. But I meet a new problem. How can I do ( multi-kinect2 ) extrinsic calibration in one computer?

Can anyone help me?

JimZhou-001 commented 5 years ago

It seems that I can't add two kinects to one computer.

When I add the second one, the first becomes [ INFO] [1564541428.495658310]: [Kinect2Bridge::main] color processing: ~-nanms (~-nanHz) publishing rate: ~0Hz [ INFO] [1564541431.497053986]: [Kinect2Bridge::main] color processing: ~-nanms (~-nanHz) publishing rate: ~0Hz [ INFO] [1564541434.501169380]: [Kinect2Bridge::main] color processing: ~-nanms (~-nanHz) publishing rate: ~0Hz [ INFO] [1564541437.504666227]: [Kinect2Bridge::main] color processing: ~-nanms (~-nanHz) publishing rate: ~0Hz

And the second one shows `[kinect02_bridge-2] restarting process process[kinect02_bridge-2]: started with pid [11400] [ INFO] [1564541316.503123226]: Loading nodelet /kinect02_bridge of type kinect2_bridge/kinect2_bridge_nodelet to manager kinect02 with the following remappings: [ INFO] [1564541316.597298713]: [Kinect2Bridge::initialize] parameter: base_name: kinect02 sensor: default fps_limit: -1 calib_path: /home/sdu123456/workspace/ros/src/iai_kinect2/kinect2_bridge/data/ use_png: false jpeg_quality: 90 png_level: 1 depth_method: default depth_device: -1 reg_method: default reg_device: -1 max_depth: 12 min_depth: 0.1 queue_size: 5 bilateral_filter: true edge_aware_filter: true publish_tf: false base_name_tf: kinect02 worker_threads: 4 exposure_method0 exposure_compensation0 pseudo_exposure_time_ms33 pseudo_exposure_time_ms33 integration_time_ms33

[Info] [CudaDepthPacketProcessorImpl] device 0: GeForce RTX 2080 Ti @ 1545MHz Memory 11018MB [Info] [CudaDepthPacketProcessorImpl] selected device 0 [Info] [Freenect2Impl] enumerating devices... [Info] [Freenect2Impl] 25 usb devices connected [Info] [Freenect2Impl] found valid Kinect v2 @10:3 with serial 500301542742 [Info] [Freenect2Impl] found valid Kinect v2 @4:2 with serial 009249464747 [Info] [Freenect2Impl] found 2 devices [ INFO] [1564541316.609962584]: [Kinect2Bridge::initDevice] Kinect2 devices found: [ INFO] [1564541316.610035089]: [Kinect2Bridge::initDevice] 0: 500301542742 (selected) [ INFO] [1564541316.610093630]: [Kinect2Bridge::initDevice] 1: 009249464747 [Info] [Freenect2DeviceImpl] opening... [Error] [protocol::UsbControl] failed to claim interface with ControlAndRgbInterfaceId(=0)! LIBUSB_ERROR_BUSY Resource busy. Try debugging with environment variable: export LIBUSB_DEBUG=3 . [Info] [Freenect2DeviceImpl] closing... [Error] [protocol::CommandTransaction] bulk transfer failed: LIBUSB_ERROR_IO Input/Output Error [Error] [protocol::CommandTransaction] bulk transfer failed: LIBUSB_ERROR_IO Input/Output Error [Info] [Freenect2DeviceImpl] deallocating usb transfer pools... [Info] [Freenect2DeviceImpl] closing usb device... [Info] [Freenect2DeviceImpl] closed [Error] [Freenect2Impl] failed to open Kinect v2: @10:3 [ INFO] [1564541316.733787973]: [Kinect2Bridge::initDevice] no device connected or failure opening the default one! [ERROR] [1564541316.733864317]: [Kinect2Bridge::start] Initialization failed! [FATAL] [1564541316.829631745]: Failed to load nodelet '/kinect02_bridgeof typekinect2_bridge/kinect2_bridge_nodeletto managerkinect02' [kinect02_bridge-2] process has died [pid 11400, exit code 255, cmd /opt/ros/kinetic/lib/nodelet/nodelet load kinect2_bridge/kinect2_bridge_nodelet kinect02 __name:=kinect02_bridge __log:=/home/sdu123456/.ros/log/4ac1674c-b33d-11e9-8cfd-54bf64a0f5c5/kinect02_bridge-2.log]. log file: /home/sdu123456/.ros/log/4ac1674c-b33d-11e9-8cfd-54bf64a0f5c5/kinect02_bridge-2*.log`

How can I solve this problem?

JimZhou-001 commented 5 years ago

I can't detect all the Kinect v2s. Why?

[Info] [CudaDepthPacketProcessorImpl] device 0: GeForce RTX 2080 Ti @ 1545MHz Memory 11018MB [Info] [CudaDepthPacketProcessorImpl] selected device 0 [Info] [Freenect2Impl] enumerating devices... [Info] [Freenect2Impl] 24 usb devices connected [Info] [Freenect2Impl] found valid Kinect v2 @10:3 with serial 500301542742 [Info] [Freenect2Impl] found 1 devices [ INFO] [1564542655.565172025]: [Kinect2Bridge::initDevice] Kinect2 devices found: [ INFO] [1564542655.565245613]: [Kinect2Bridge::initDevice] 0: 500301542742 (selected)

bsubbaraman commented 5 years ago

Have you set up your camera_network.yaml file with the serial number of each Kinect? If not, please see this section of the wiki.

JimZhou-001 commented 5 years ago 
enable_people_tracking: true
enable_pose: true
enable_object: true
# Camera network parameters
network:
  - pc: "PC_ONLY"
    sensors:
      - type: kinect2
        id: "kinect01"
        serial: "500301542742"
        people_detector: hog
      - type: kinect2
        id: "kinect02"
        serial: "004481451147"
        people_detector: hog
      - type: kinect2
        id: "kinect03"
        serial: "019938543547"
        people_detector: hog
      - type: kinect2
        id: "kinect04"
        serial: "009249464747"
        people_detector: hog

# Checkerboard parameters
checkerboard:
  rows: 5
  cols: 7
  cell_width: 0.03
  cell_height: 0.03

@bsubbaraman Thank you. Is it right?

JimZhou-001 commented 5 years ago

Anyway, I can only run one Kinect v2 at one time.

JimZhou-001 commented 5 years ago

[Error] [protocol::CommandTransaction] bulk transfer failed: LIBUSB_ERROR_IO Input/Output Error [Error] [protocol::CommandTransaction] bulk transfer failed: LIBUSB_ERROR_IO Input/Output Error

JimZhou-001 commented 5 years ago

Hello, I define kinect01 to serial 500... but it automatically select the first one ...

[ INFO] [1564551925.694413728]: [Kinect2Bridge::initDevice] Kinect2 devices found: 
[ INFO] [1564551925.694489813]: [Kinect2Bridge::initDevice]   0: 019938543547 (selected)
[ INFO] [1564551925.694542352]: [Kinect2Bridge::initDevice]   1: 500301542742
[ INFO] [1564551925.694587573]: [Kinect2Bridge::initDevice]   2: 009249464747
[ INFO] [1564551925.694625023]: [Kinect2Bridge::initDevice]   3: 004481451147
JimZhou-001 commented 5 years ago

Hello! @bsubbaraman @jburkeucla @SimeonOA

I connect 4 kinect v2s to one computer. But every time it can only select the first one. How can I deal with it? Camera Network Calibration

By the way, I can run 4 kinects with 4 Protonect (libfreenect2 example) programs. But it fail with your project.

Thank you.

logging to /home/sdu123456/.ros/log/fd69dc2e-b353-11e9-9d9c-54bf64a0f5c5/roslaunch-123456-21612.log
Checking log directory for disk usage. This may take awhile.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://101.76.223.95:32883/

SUMMARY
========

PARAMETERS
 * /kinect01_bridge/analog_gain: 2.0
 * /kinect01_bridge/base_name: kinect01
 * /kinect01_bridge/base_name_tf: kinect01
 * /kinect01_bridge/bilateral_filter: True
 * /kinect01_bridge/calib_path: /home/sdu123456/w...
 * /kinect01_bridge/depth_device: -1
 * /kinect01_bridge/depth_method: default
 * /kinect01_bridge/edge_aware_filter: True
 * /kinect01_bridge/exposure_compensation: 0.0
 * /kinect01_bridge/exposure_method: 0
 * /kinect01_bridge/fps_limit: -1.0
 * /kinect01_bridge/integration_time_ms: 33.0
 * /kinect01_bridge/jpeg_quality: 90
 * /kinect01_bridge/max_depth: 12.0
 * /kinect01_bridge/min_depth: 0.1
 * /kinect01_bridge/png_level: 1
 * /kinect01_bridge/pseudo_exposure_time_ms: 33.0
 * /kinect01_bridge/publish_tf: False
 * /kinect01_bridge/queue_size: 5
 * /kinect01_bridge/reg_device: -1
 * /kinect01_bridge/reg_method: default
 * /kinect01_bridge/respawn: True
 * /kinect01_bridge/sensor: 
 * /kinect01_bridge/start_manager": True
 * /kinect01_bridge/use_machine: True
 * /kinect01_bridge/use_nodelet": True
 * /kinect01_bridge/use_png: False
 * /kinect01_bridge/worker_threads: 4
 * /kinect01_points_xyzrgb_hd/queue_size: 5
 * /kinect01_points_xyzrgb_qhd/queue_size: 5
 * /kinect01_points_xyzrgb_sd/queue_size: 5
 * /rosdistro: kinetic
 * /rosversion: 1.12.14

NODES
  /
    kinect01 (nodelet/nodelet)
    kinect01_bridge (nodelet/nodelet)
    kinect01_broadcaster (tf/static_transform_publisher)
    kinect01_points_xyzrgb_hd (nodelet/nodelet)
    kinect01_points_xyzrgb_qhd (nodelet/nodelet)
    kinect01_points_xyzrgb_sd (nodelet/nodelet)

ROS_MASTER_URI=http://localhost:11311

process[kinect01-1]: started with pid [21629]
process[kinect01_bridge-2]: started with pid [21630]
process[kinect01_points_xyzrgb_sd-3]: started with pid [21631]
process[kinect01_points_xyzrgb_qhd-4]: started with pid [21632]
[ INFO] [1564553162.413175038]: Loading nodelet /kinect01_bridge of type kinect2_bridge/kinect2_bridge_nodelet to manager kinect01 with the following remappings:
[ INFO] [1564553162.421856783]: waitForService: Service [/kinect01/load_nodelet] has not been advertised, waiting...
process[kinect01_points_xyzrgb_hd-5]: started with pid [21645]
[ INFO] [1564553162.437399061]: Initializing nodelet with 20 worker threads.
process[kinect01_broadcaster-6]: started with pid [21684]
[ INFO] [1564553162.447359664]: waitForService: Service [/kinect01/load_nodelet] is now available.
[ INFO] [1564553162.566830843]: [Kinect2Bridge::initialize] parameter:
        base_name: kinect01
           sensor: default
        fps_limit: -1
       calib_path: /home/sdu123456/workspace/ros/src/iai_kinect2/kinect2_bridge/data/
          use_png: false
     jpeg_quality: 90
        png_level: 1
     depth_method: default
     depth_device: -1
       reg_method: default
       reg_device: -1
        max_depth: 12
        min_depth: 0.1
       queue_size: 5
 bilateral_filter: true
edge_aware_filter: true
       publish_tf: false
     base_name_tf: kinect01
   worker_threads: 4
exposure_method0
exposure_compensation0
pseudo_exposure_time_ms33
pseudo_exposure_time_ms33
integration_time_ms33

[Info] [CudaDepthPacketProcessorImpl] device 0: GeForce RTX 2080 Ti @ 1545MHz Memory 11018MB
[Info] [CudaDepthPacketProcessorImpl] selected device 0
[Info] [Freenect2Impl] enumerating devices...
[Info] [Freenect2Impl] 27 usb devices connected
[Info] [Freenect2Impl] found valid Kinect v2 @10:7 with serial 019938543547
[Info] [Freenect2Impl] found valid Kinect v2 @8:5 with serial 500301542742
[Info] [Freenect2Impl] found valid Kinect v2 @6:4 with serial 009249464747
[Info] [Freenect2Impl] found valid Kinect v2 @4:4 with serial 004481451147
[Info] [Freenect2Impl] found 4 devices
[ INFO] [1564553162.929316554]: [Kinect2Bridge::initDevice] Kinect2 devices found: 
[ INFO] [1564553162.929393402]: [Kinect2Bridge::initDevice]   0: 019938543547 (selected)
[ INFO] [1564553162.929440278]: [Kinect2Bridge::initDevice]   1: 500301542742
[ INFO] [1564553162.929475943]: [Kinect2Bridge::initDevice]   2: 009249464747
[ INFO] [1564553162.929513769]: [Kinect2Bridge::initDevice]   3: 004481451147
[Info] [Freenect2DeviceImpl] opening...
[Info] [Freenect2DeviceImpl] transfer pool sizes rgb: 20*16384 ir: 60*8*33792
[Info] [Freenect2DeviceImpl] opened
[ INFO] [1564553163.040266328]: [Kinect2Bridge::initDevice] starting kinect2
[Info] [Freenect2DeviceImpl] starting...
[Info] [Freenect2DeviceImpl] submitting rgb transfers...
[Info] [Freenect2DeviceImpl] submitting depth transfers...
[Info] [Freenect2DeviceImpl] started
[ INFO] [1564553163.392867100]: [Kinect2Bridge::initDevice] device serial: 019938543547
[ INFO] [1564553163.392946694]: [Kinect2Bridge::initDevice] device firmware: 4.0.3916.0
[Info] [Freenect2DeviceImpl] stopping...
[Info] [Freenect2DeviceImpl] canceling rgb transfers...
[Info] [Freenect2DeviceImpl] canceling depth transfers...
[Info] [Freenect2DeviceImpl] stopped
[ERROR] [1564553163.805687287]: [Kinect2Bridge::loadCalibrationDepthFile] can't open calibration depth file: /home/sdu123456/workspace/ros/src/iai_kinect2/kinect2_bridge/data/019938543547/calib_depth.yaml
[ WARN] [1564553163.805776587]: [Kinect2Bridge::initCalibration] using defaults for depth shift.
[ INFO] [1564553163.846671477]: [DepthRegistration::New] Using OpenCL registration method!
[ INFO] [1564553163.846755926]: [DepthRegistration::New] Using OpenCL registration method!
[ INFO] [1564553163.857542009]: [DepthRegistrationOpenCL::init] devices:
[ INFO] [1564553163.857731409]: [DepthRegistrationOpenCL::init]   0: GeForce RTX 2080 Ti
[ INFO] [1564553163.858216924]: [DepthRegistrationOpenCL::init] selected device: GeForce RTX 2080 Ti
[ INFO] [1564553164.004596112]: [DepthRegistrationOpenCL::init] devices:
[ INFO] [1564553164.004701868]: [DepthRegistrationOpenCL::init]   0: GeForce RTX 2080 Ti
[ INFO] [1564553164.004756285]: [DepthRegistrationOpenCL::init] selected device: GeForce RTX 2080 Ti
[ INFO] [1564553164.174292035]: [Kinect2Bridge::main] waiting for clients to connect
[ INFO] [1564553164.359695198]: [Kinect2Bridge::callbackStatus] client connected. starting device...
[Info] [Freenect2DeviceImpl] starting...
[Info] [Freenect2DeviceImpl] submitting rgb transfers...
[Info] [Freenect2DeviceImpl] started
[Info] [RgbPacketStreamParser] packetsize or sequence doesn't match!
[ INFO] [1564553167.719550216]: [Kinect2Bridge::main] color processing: ~13.5662ms (~73.7125Hz) publishing rate: ~25.2559Hz
JimZhou-001 commented 5 years ago

Hello!

I have solved this problem.

I meet this problem because your code is for one-to-one connection. When multiple sensors are connected to one PC, we should distinguish them by (Kinect) sensor serial. Previously I add <arg name="sensor" value="$(arg sensor_id)" /> to each sensor_<sensor_id>.launch and detection_node_<sensor_id>.launch manually. And it works for me. Then I try to fix the bug from your code. So here comes Support multiple sensors.

Thank you for your nice tool.