search

uzh-rpg / rpg_dvs_ros

ROS packages for DVS
MIT License
291 stars 152 forks source link

Downgrade DAVIS revision #77

Closed lee-sangil closed 5 years ago

lee-sangil commented 5 years ago

Hi,

Is there any method to downgrade DAVIS revision from 16 to 9912? or specifically 9912.

I had reflashed DAVIS device from 9912 to 16 for executing jAER toolkit, but this ROS package requires 16 revision.

Thanks.

YongjianDeng commented 5 years ago

Hi,

Is there any method to downgrade DAVIS revision from 16 to 9912? or specifically 9912.

I had reflashed DAVIS device from 9912 to 16 for executing jAER toolkit, but this ROS package requires 16 revision.

Thanks.

Hi,

Have you already solved this problem? Because I am now facing the same issue as yours.

BW

Thanks

llongi commented 5 years ago

Hi, just to clarify the issue: all current DAVIS cameras and iniVation software (libcaer 3.0, jAER 1.7, Flashy) do require logic version 16. Our DAVIS346 RED cameras only work with version 16. The current RPG ROS driver uses an old libcaer version (2.something) that only supports the old logic version 9912. The correct solution here is to fix the ROS driver so it uses the latest libcaer 3.0 release.

TSRyan commented 5 years ago

I ran into this issue as well. As you suggested, @llongi , I installed libcaer version 3.0.0, but I encountered errors when trying to rebuild the davis_ros_driver package:

$ catkin build davis_ros_driver

...

/home/tryan/catkin_ws/src/rpg_dvs_ros/davis_ros_driver/src/driver.cpp: In member function ‘void davis_ros_driver::DavisRosDriver::changeDvsParameters()’:
/home/tryan/catkin_ws/src/rpg_dvs_ros/davis_ros_driver/src/driver.cpp:303:70: error: ‘DAVIS_CONFIG_APS_FRAME_DELAY’ was not declared in this scope
                 caerDeviceConfigSet(davis_handle_, DAVIS_CONFIG_APS, DAVIS_CONFIG_APS_FRAME_DELAY, current_config_.frame_delay);
                                                                      ^
/home/tryan/catkin_ws/src/rpg_dvs_ros/davis_ros_driver/src/driver.cpp:307:70: error: ‘DAVIS_CONFIG_IMU_RUN’ was not declared in this scope
                 caerDeviceConfigSet(davis_handle_, DAVIS_CONFIG_IMU, DAVIS_CONFIG_IMU_RUN, current_config_.imu_enabled);
                                                                      ^
...

Some of the configuration parameters appear to have changed (compare davis.h between versions of libcaer), so the driver node itself will need to be modified to deal with the new parameters (e.g., DAVIS_CONFIG_APS_FRAME_MODE) and remove obsolete code (e.g., handling DAVIS_CONFIG_APS_FRAME_DELAY). I haven't had time to make the modifications, but I'm posting this in case it sheds some light for someone else.

TSRyan commented 5 years ago

I have the DAVIS ROS driver working with libcaer 3 now. After updating to libcaer 3 and removing references to frame_delay and imu_run, I implemented support for frame_mode and frame_interval, which replaced frame_delay. Incidentally, without frame_mode implemented, the APS picture did not display properly; it looked like an interleaved double image. If anyone wants specifics, let me know.

llongi commented 5 years ago

Your branch with libcaer3 support looks good. To properly handle the IMU change too, simply replace the one call to set IMU_RUN with three calls that set IMU_RUN_ACCELEROMETER, IMU_RUN_TEMPERATURE and IMU_RUN_GYROSCOPE to the same value. That gives you back the same exact behavior as before.

It should work also by not setting frame_mode, as the default value should be 0 automatically (ie. frame_default). Can you give more details on this? What camera model are you using? And it didn't work without setting FRAME_MODE, but when you do set it, then it works with any of the three possibilities? Or does one of them fail too?

Thanks for your work! -- Luca Longinotti (llongi)

Software Engineer iniVation AG - https://inivation.com/ Zurich, Switzerland Office: +41 44 500 3214

Support - https://inivation.com/support/

-----Original Message----- From: TSRyan notifications@github.com Reply-To: uzh-rpg/rpg_dvs_ros < reply@reply.github.com

To: uzh-rpg/rpg_dvs_ros rpg_dvs_ros@noreply.github.com Cc: Luca Longinotti luca.longinotti@inivation.com, Mention < mention@noreply.github.com> Subject: Re: [uzh-rpg/rpg_dvs_ros] Downgrade DAVIS revision (#77) Date: Wed, 07 Nov 2018 19:37:09 +0000

I have the DAVIS ROS driver working with libcaer 3 now. After updating to libcaer 3 and removing references to frame_delay and imu_run, I implemented support for frame_mode and frame_interval, which replaced frame_delay. Incidentally, without frame_mode implemented, the APS picture did not display properly; it looked like an interleaved double image. If anyone wants specifics, let me know. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

TSRyan commented 5 years ago

I'll add the IMU_RUN replacements for a PR if you'd like.

As for FRAME_MODE, it did default to 0, but that mode did not display properly (and still doesn't). It does however display properly in grayscale and original modes. I'm using a DAVIS346 red color

Default: frame_mode_default Note how the image appears enlarged (sort of) and the chair back appears in both the left and right sides of the frame. It seems like the pixels are not arranged correctly.

Grayscale: frame_mode_grayscale

Original: frame_mode_original

llongi commented 5 years ago

Ahhh I see the problem. davis_ros_driver/src/driver.cpp line 667 has: msg.encoding = "mono8"; but the 'default' output of a color sensor is a color image, so the encoding is wrong and it gets the sizes wrong too (because RGB is three values per pixel, MONO only one). The fix is to check if the number of channels in the cAER frame is not 1, and in that case properly format the color image in the way ROS expects it. I don't have any experience with how ROS handles frame though and can't help there. -- Luca Longinotti (llongi)

Software Engineer iniVation AG - https://inivation.com/ Zurich, Switzerland Office: +41 44 500 3214

Support - https://inivation.com/support/

-----Original Message----- From: TSRyan notifications@github.com Reply-To: uzh-rpg/rpg_dvs_ros < reply@reply.github.com

To: uzh-rpg/rpg_dvs_ros rpg_dvs_ros@noreply.github.com Cc: Luca Longinotti luca.longinotti@inivation.com, Mention < mention@noreply.github.com> Subject: Re: [uzh-rpg/rpg_dvs_ros] Downgrade DAVIS revision (#77) Date: Wed, 07 Nov 2018 21:22:36 +0000

I'll add the IMU_RUN replacements for a PR if you'd like. As for FRAME_MODE, it did default to 0, but that mode did not display properly (and still doesn't). It does however display properly in grayscale and original modes. I'm using a DAVIS346 red color Default:

Note how the image appears enlarged (sort of) and the chair back appears in both the left and right sides of the frame. It seems like the pixels are not arranged correctly. Grayscale:

Original:

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

TSRyan commented 5 years ago

I noticed that, too. I'll look into it further.

TSRyan commented 5 years ago

I got the default frame mode to work with the color data; it now publishes an rgb8-encoded ROS image message when it sees 3 channels, and mono8 for single-channel frame events. Apparently, dvs_renderer converts it to grayscale anyway, but if you use image_view to look at the published message directly from the driver, it is in color as expected.

On a side note, I added the skip, polarity, and ROI (both DVS and APS) filter functionalities and made them dynamically reconfigurable.

YongjianDeng commented 5 years ago

I got the default frame mode to work with the color data; it now publishes an rgb8-encoded ROS image message when it sees 3 channels, and mono8 for single-channel frame events. Apparently, dvs_renderer converts it to grayscale anyway, but if you use image_view to look at the published message directly from the driver, it is in color as expected.

On a side note, I added the skip, polarity, and ROI (both DVS and APS) filter functionalities and made them dynamically reconfigurable.

Hi TSRyan,

I am newer in ROS, cannot figure out the driver stuff, would you tell me some detail about how to transform the dependency from library 1.0 to library 3.0?

Thank you a lot

Tony

TSRyan commented 5 years ago

The exact changes I made are in my fork of this repo. The main things to get up and running are:

I added support to make the hardware filters dynamically reconfigurable as well, but that's not strictly necessary. If you don't want to bother figuring out how to code only the changes you need, you can just clone my repo instead of the main one in step 6 of the normal install instructions:

git clone https://github.com/TSRyan/rpg_dvs_ros.git

instead of

git clone https://github.com/uzh-rpg/rpg_dvs_ros.git

NOTE: my updates are in the update/use_libcaer3_with_davis346 branch of my repo--not master, so you will need to make sure you check out the proper branch. The rest of the instructions still apply. I hope this helps!

EDIT: formatting for clarity EDIT: add note indicating appropriate branch of my repo.

llongi commented 5 years ago

Branch looks good to me. Just a comment on the hot-pixel filter auto-train: you don't have to send values manually, libcaer does that for you. The moment you do configSet(AUTO_TRAIN, true), it starts looking at events, learns the 8 most active pixels, and if they are above 5 KHz it sends the commands to the hardware to filter them out. So really there's nothing to do manually there.

TSRyan commented 5 years ago

I'm not 100% sure I understand what you mean by sending the values manually. I do send the config values to the hardware manually when AUTO_TRAIN is false (I did this to confirm that known pixels were being filtered), but when AUTO_TRAIN is true, I pull them from the hardware and update the config to match (so you can see the automatic changes in rqt_reconfigure). Actually, in doing so, I noticed that the auto train output seems to have the column and rows backwards (trying to set the row values to 346 and columns to 260), or maybe I do, but I haven't found the error. In any case, if I have misunderstood or should handle it differently, please let me know.

EDIT: I just realized my confusion may better expressed as a question: Are you referring to the manual mode of hot-pixel filtering (AUTO_TRAIN=false), or am I still manually sending values (unintentional) when AUTO_TRAIN=true?

llongi commented 5 years ago

Ahh my fault, sorry. When quickly going over the code I thought I saw configSet() when pixel-filter enabled, but it's actually configGet(). So your code is actually sound. You might want to wait some time before doing the configGet()s though, as the automatic training takes a moment and the results are only available after it's done. You can poll the value of AUTO_TRAIN repeatedly, when it switches back to 'false' on its own, that means the training is done and the new values sent to the device. -- Luca Longinotti (llongi)

Software Engineer iniVation AG - https://inivation.com/ Zurich, Switzerland Office: +41 44 500 32 14

Support - https://inivation.com/support/

-----Original Message----- From: TSRyan notifications@github.com Reply-to: uzh-rpg/rpg_dvs_ros <reply+0046590e548a082d8f261d1484c76cf825d583a6cd26442792cf000000011804 044e92a169ce162328a2@reply.github.com> To: uzh-rpg/rpg_dvs_ros rpg_dvs_ros@noreply.github.com Cc: Luca Longinotti luca.longinotti@inivation.com, Mention <mention@n oreply.github.com> Subject: Re: [uzh-rpg/rpg_dvs_ros] Downgrade DAVIS revision (#77) Date: Wed, 14 Nov 2018 07:42:06 -0800

I'm not 100% sure I understand what you mean by sending the values manually. I do send the config values to the hardware manually when AUTO_TRAIN is false (I did this to confirm that known pixels were being filtered), but when AUTO_TRAIN is true, I pull them from the hardware and update the config to match (so you can see the automatic changes in rqt_reconfigure). Actually, in doing so, I noticed that the auto train output seems to have the column and rows backwards (trying to set the row values to 346 and columns to 260), or maybe I do, but I haven't found the error. In any case, if I have misunderstood or should handle it differently, please let me know. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

TSRyan commented 5 years ago

No worries; thanks for checking. I had assumed auto-train was continuous instead of one-shot, oops. I added a while loop to check for AUTO_TRAIN = false every 100 ms before updating the pixel coordinates and auto_train in current_config_ as you suggested.

llongi commented 5 years ago

Looks good to me. :+1: If it's all working for you, I suggest you do a PR, to me it looks ready for more testing and merging then.

YongjianDeng commented 5 years ago

The exact changes I made are in my fork of this repo. The main things to get up and running are:

If you don't want to bother figuring out how to code only the changes you need, you can just clone my repo instead of the main one in step 6 of the normal install instructions:

git clone https://github.com/TSRyan/rpg_dvs_ros.git

instead of

git clone https://github.com/uzh-rpg/rpg_dvs_ros.git

The rest of the instructions still apply. I hope this helps!

Hi TSRyan,

Thanks you a lot for your generous help.

BW,

Tony

superhector2000 commented 5 years ago

Hi TSRyan, When I clone using "git clone https://github.com/TSRyan/rpg_dvs_ros.git ", it downloads the main repo and doesn't have your changes. So I download it as a Zip file. It turns out it compiles fine but when I tried running the camera it shows me the same error as the DAVIS revision from 16 to 9912, but instead from 16 to 18. What can I do about it? Inivation doesn't seem to have previous firmware versions available.

2018-12-04 08:47:00 (TZ-0500): ERROR: DAVIS ID-1: Device firmware version incorrect. You have version 5; but version 4 is required. Please update by following the Flashy documentation at 'https://inivation.com/support/software/reflashing/'. 2018-12-04 08:47:00 (TZ-0500): ERROR: DAVIS ID-1: Device logic version incorrect. You have version 18; but version 16 is required. Please update by following the Flashy documentation at 'https://inivation.com/support/software/reflashing/'. 2018-12-04 08:47:00 (TZ-0500): CRITICAL: DAVIS ID-1: Failed to open device, see above log message for more information (errno=-6).

TSRyan commented 5 years ago

Hi @superhector2000, My mistake. You need to clone the update/use_libcaer3_with_DAVIS346 branch of my repo--not the master branch. The errors you're getting are actually from libcaer. My update branch uses libcaer 3.0.0, but there is now a 3.1.0. I can't say for sure how your device will interact with libcaer as I only have experience with my own device, but if you still get similar errors after switching branches, I would start looking at that aspect of things.

I will update my previous comments to reflect the appropriate branch of my repo to clone. Sorry for the confusion.

superhector2000 commented 5 years ago

I am assuming that your device has logic version 16? I have a new DAVIS346 red, that comes with version logic version 18 :( I just tried changing the libcaer to 3.1.0 but when I run roslaunch dvs_renderer renderer_mono.launch, the app just hangs.

TSRyan commented 5 years ago

Yeah, my DAVIS is logic version 16. I don't know what changes there are in 18 or if/how they may affect the driver.

Did you try roslaunch dvs_renderer davis_mono.launch? That's the example launch file suggested in the README. I don't think renderer_mono.launch brings up the device driver--or are you launching the driver elsewhere?

llongi commented 5 years ago

Logic 18 fixes a problem related to IMU RUN controls, fixes corrupt data with DAVIS BLUE, and enables IMU on DAVIS RED. From a usage point of view it's an easy upgrade from libcaer 3.0 (logic 16) to 3.1 (logic 18). There are no changes to commands or configurations or data formats. Simply build the ROS driver against 3.1.0 and it should work correctly.

superhector2000 commented 5 years ago

Sorry, I meant 'roslaunch dvs_renderer davis_mono.launch'. Still got the error of requiring logic version 16 while I have 18.

superhector2000 commented 5 years ago

It is working! Thanks for your help Luca and TSRyan! You were right Luca, the driver didn't affect anything else. So here is what I did to fix it:

  1. Modify the CMakeLists.txt to download 3.1.0
  2. Do a fresh build of the ros packages (rm -rf devel/ build/ and follow steps 7 and on) Thanks!