Incomplete installation process on v0.11.0 - Message generation failing

[wayneparrott]

I while installing rclnodejs v0.11.0 from npm the message generation process is aborting when it encounters a missing file. This results in only a subset of js file being generated and the interface.d.ts is not generated. Here's the informational portion of stdout during the installation process.

Note: the same issue applies when running rclnodejs#regenerateAll()

We need to fix this immediate issue and post a 0.11.1 update as well as extend testing to detect message generation issues.

> rclnodejs@0.11.0 postinstall /home/wayne/dev/workspaces/ros-lidar/test1/node_modules/rclnodejs
> node scripts/generate_messages.js

Start JavaScript message generation...
Caught error: Error: Cannot find module '../../generated/action_tutorials_interfaces/action_tutorials_interfaces__msg__FibonacciActionGoal.js'
npm WARN typescript-hello-world@1.0.0 No repository field.

+ rclnodejs@0.11.0
added 55 packages from 36 contributors and audited 77 packages in 51.97s
found 0 vulnerabilities

[wayneparrott]

How come we didn't find this bug when testing?

I checked the CI logs and for example the travis ci is using the same Linux version as I am but it is not reproducing this problem when it runs the installation step. And in my dev environment I had an older copy of the generated message files prior to our removal of processing action files. So my local env did not reflect a proper user environment. Once I removed the generated folder from my dev environment I am able to replicate the failure.

[wayneparrott]

Added try/catch around tsd message file process to ignore errors as it is a non-critical failure. The output will appear as follows:

Start JavaScript message generation...
Unable to generate tsd info: Cannot find module '../../generated/action_tutorials_interfaces/action_tutorials_interfaces__msg__FibonacciActionGoa
l.js'
Unable to generate tsd info: Cannot find module '../../generated/action_tutorials_interfaces/action_tutorials_interfaces__msg__FibonacciFeedback.
js'
Unable to generate tsd info: Cannot find module '../../generated/action_tutorials_interfaces/action_tutorials_interfaces__msg__FibonacciGoal.js'
Unable to generate tsd info: Cannot find module '../../generated/action_tutorials_interfaces/action_tutorials_interfaces__msg__FibonacciResult.js
'
Unable to generate tsd info: Cannot find module '../../generated/example_interfaces/example_interfaces__msg__FibonacciActionGoal.js'
Unable to generate tsd info: Cannot find module '../../generated/example_interfaces/example_interfaces__msg__FibonacciFeedback.js'
Unable to generate tsd info: Cannot find module '../../generated/example_interfaces/example_interfaces__msg__FibonacciGoal.js'
Unable to generate tsd info: Cannot find module '../../generated/example_interfaces/example_interfaces__msg__FibonacciResult.js'
Unable to generate tsd info: Cannot find module '../../generated/tf2_msgs/tf2_msgs__msg__LookupTransformActionGoal.js'
Unable to generate tsd info: Cannot find module '../../generated/tf2_msgs/tf2_msgs__msg__LookupTransformFeedback.js'
Unable to generate tsd info: Cannot find module '../../generated/tf2_msgs/tf2_msgs__msg__LookupTransformGoal.js'
Unable to generate tsd info: Cannot find module '../../generated/tf2_msgs/tf2_msgs__msg__LookupTransformResult.js'
Generation complete.

[minggangw]

I checked the CI logs and for example the travis ci is using the same Linux version as I am but it is not reproducing this problem when it runs the installation step. And in my dev environment I had an older copy of the generated message files prior to our removal of processing action files.

The test on travis-ci runs in docker, so I think each run should be without any cache, e.g. the generated folder.

and from the error message:

Caught error: Error: Cannot find module '../../generated/action_tutorials_interfaces/action_tutorials_interfacesmsgFibonacciActionGoal.js'

I suspect the reason is that some unexisting .js files are required, which is triggered by https://github.com/RobotWebTools/rclnodejs/blob/892e5392074933a4ae7593213cd0145e6d9ae86c/rostsd_gen/index.js#L75 but the existence of a file is verified at https://github.com/RobotWebTools/rclnodejs/blob/892e5392074933a4ae7593213cd0145e6d9ae86c/lib/interface_loader.js#L95

Would you please confirm which line throws the exception and whether the required file exists?

[wayneparrott]

I believe the error is resulting just after the line #95 when any action.js file is loaded. https://github.com/RobotWebTools/rclnodejs/blob/892e5392074933a4ae7593213cd0145e6d9ae86c/lib/interface_loader.js#L96 I'll dig into a little deeper in a few hours. But I can see that the action.js file exists and then in the require(filePath) the exception is being thrown. An example file is the generated/tf2_msgs/tf2_msgsactionLookupTransformActionResult.js.

[minggangw]

The file tf2_msgs__action__LookupTransformActionResult.js. was generated by https://github.com/RobotWebTools/rclnodejs/blob/4f55598239aaac6b2097ba046fd8405157842a41/rosidl_gen/index.js#L56 But as the generation for .action files has been removed (ac497ef13aeb562c06e75985e2005446b75a77b0), I am wondering how this js file is created?

Or you can debug if the following function does get called (I install rclnodejs by npm, but cannot reproduce) https://github.com/RobotWebTools/rclnodejs/blob/4f55598239aaac6b2097ba046fd8405157842a41/rosidl_gen/index.js#L26

[wayneparrott]

OK. After reinstalling my ros2-eloquent installation I am NOT able to replicate this issue when installing from npm (v.0.11.0). The issue resulted from my ros2 environment having action pkgs that were created prior to removal of action processing. So my ros2 installation was invalid for testing. I should have caught this when I noticed that the travis.ci was not generating the error. My position now is that we should close this issue as WORKSFORME and close the PR.

[minggangw]

That sounds reasonable. Because when generating from .action, several .msg files were created in the past. For example, if you have an action file named LookupTransform.action, and the previous generator is going to be created:

• LookupTransformResult.msg
• LookupTransformGoal.msg
• LookupTransformFeedback.msg
• LookupTransformAction.msg
• LookupTransformActionFeedback.msg
• LookupTransformActionGoal.msg
• LookupTransformActionResult.msg

and these .msg files are written under subfolder named msg of LookupTransform.action (this behavior caused issue https://github.com/RobotWebTools/rclnodejs/issues/510 due to permission error). After ac497ef, although the .action is ignored while generating the JavaScript message files, the .msg files already generated from .action are still there. Thus, the generator will still generate .js files for them. That's is the reason why you still can find tf2_msgs__action__LookupTransformActionResult.js on your local disk.

Meanwhile, I don't understand the JS file collected by https://github.com/RobotWebTools/rclnodejs/blob/df09ca482affdda20192bda561f29961f7889c25/rostsd_gen/index.js#L63-L65 was reported to be not found later at https://github.com/RobotWebTools/rclnodejs/blob/df09ca482affdda20192bda561f29961f7889c25/rostsd_gen/index.js#L74

[minggangw]

Finally, I think I found the root cause. The reason is the .msg file created by the old generator which generates for .action will trigger to create JS files, e.g. action_tutorials_interfaces__action__FibonacciActionGoal.js and that file will try to require action_tutorials_interfaces__msg__FibonacciActionGoal.js (this is the output error)