Message exchange between a ROS node and multi Windows programs

jinno-oit commented 3 years ago

https://github.com/oit-ipbl/Integration/blob/65982735fe2b98f40ee0595164d11fdb4b9c33f8/win_multi/win_multi.md についてです．

show_hand_game_win.py このプログラムはmediapipesを利用して，コンピュータが指示した手をユーザがカメラに翳すゲームです

コンピュータ－＞ROSの方が良い？

start_game functionでは，ROSとの間でメッセージの送受信が行われる．例えば，message_from_ros = ros_bridge_tcp.wait_response(pub_msg, hand_types, timeout=30) では，pub_msgをROSに送信し，hand_typesに含まれる文字列がROSから返ってくることを最大30秒待ち，返り値をmessage_from_rosに保存する

ここのところで，第二引数についてもう少し説明した方が良いかもしれません．・第二引数は文字列のリストで，その中に格納されている文字列と完全一致する場合のみ受け取ります（丁寧すぎる？）・Noneにするとどのような内容でも受け取るようにできます（ただし，予期せぬメッセージを受け取って先に進んでしまう可能性がある） ※ちなみに，Message exchange between a Windows program and a ROS nodeの方では，ros_bridge_tcp.wait_response(pub_msg, timeout=10)のように書かれてもいるようで，Noneと入れなくても同じことができそうです．どちらかに統一しておいてあげた方が良いと思います．Noneと入れた方が安心な気はします．

現在の仕様では，start_on_windows_single.pyはwhile True:でROSからのメッセージ送信を無限ループで待機するようになっている

これにプラスして，無限ループで待機状態のときにROSから"The end"というメッセージが届くと無限ループを終了するということを追加するのが良いと思います．

start_on_win_single.py や start_on_ros_single.py などで，「呼び出す処理を追加する」となっていますが，配布するプログラムでは削除しておきますか？

show_hand_game_ros.py と show_hand_game_win.py のやりとり部分にコメントで番号を付けてあげると横に並べてみたときに良いのかもしれません．また，画像処理部分が長いとタイムアウトになる可能性があることも伝えたいところですが，図を作るのが良いかもですね．ちょっと考えてみます．

たまに正常に動作しない場合があります．その場合はWindows Terminalを再起動してみよう．

この現象は，配っているポータル環境のVSCodeを使うときに発生するので，特定のプログラムで発生するわけではなさそうです．うちの学生はbasicを試しているときになっていました．

bright_dark_game についても，show_hand_gameのような説明があると良いと思います．

「このプログラムは，ROSが指示する"bright"と"dark"に合わせて，カメラの映像を明るくしたり暗くしたりするゲームです．明るくしたり暗くしたりする方法は，例えば，カメラを照明に向けたり，カメラを手で覆ったりです」

また，判定基準はユーザーの環境によるので，プログラムを読んで自分で調整してみると良いかもしれません．とか書いておくと良いかもしれません．

Exercise (integration 2) Add navigation function between game A and game B. See Robot control 3.

game A と game B をそれぞれ，show_hand_game bright_dark_game に変えればOK

Challenge (integration 3)

確かにチャレンジは次のTeam exerciseに託して良いと思います．

igaki commented 3 years ago

@jinno-oit これどこですかね？

start_on_win_single.py や start_on_ros_single.py などで，「呼び出す処理を追加する」となっていますが，配布するプログラムでは削除しておきますか？

jinno-oit commented 3 years ago

@igaki 以下の二つの箇所で，「***を呼び出す処理を追加してコマンドを実施する」となっていますが，配布しているプログラムでは既にコードが追加されていると思います．

start_on_windows_single.py show_hand_game_win.pyが正常に実行できるかを確認できたら，start_on_windows_single.pyにshow_hand_game_win.pyを呼び出す処理を追加し，下記コマンドを実行してROSとの通信を確認すると良い現在の仕様では，start_on_windows_single.pyはwhile True:でROSからのメッセージ送信を無限ループで待機するようになっている

start_on_ros_single.py show_hand_game_ros.pyが正常に実行できるかを確認できたら，start_on_ros_single.pyにshow_hand_game_ros.pyを呼び出す処理を追加し，下記コマンドを実行してWindows側プログラム(show_hand_game_win.py)との通信を確認すると良いもし，roslaunch oit_stage_ros navitation.launchを実行していない場合は実行し，別のターミナルでコマンドを実行する

igaki commented 3 years ago

@jinno-oit ナビゲーションプログラム追加しました．ロボットの移動部分を別ファイルにしたんですが，これ，単独で動作するようなmain()追加する書き方ご存じないでしょうか．．？

jinno-oit commented 3 years ago

@igaki ありがとうございます．以下で動きました．現在の move_robot_ros.py には import os が足りないようです．

def demo():
    script_name = os.path.basename(__file__)
    rospy.init_node(os.path.splitext(script_name)[0])
    rospy.sleep(0.5)  # rospy.Time.now() returns 0, without this sleep.

    rospy.loginfo("%s:Started", rospy.get_name())
    process(2.0, 2.5, -90)
    rospy.loginfo("%s:Exiting", rospy.get_name())

if __name__ == '__main__':
    try:
        demo()
    except Exception as e:
        rospy.logerr("%s:%s", rospy.get_name(), str(e))
        exit(1)

igaki commented 3 years ago

@jinno-oit ありがとうございます．コード修正してコミットしておきます．

jinno-oit commented 3 years ago

とりあえず，ざっと担当分を英訳しました．

Objectives

Let's Make multiple image processing programs on windows and a ROS node which can communicate with each other via TCP/IP.

This page explains how to communicate between many image processing programs on Windows and the ROS node.

Prerequisite

You have to finish all of robots, image processing, and Message exchange between a Windows program and a ROS node

Practice1(Single image processing program and a ros node)

Make a Windows side python program

Save the following two files into C:\oit\py21\code on Windows.
- If you want to download the files, click the following links and then download from a Raw button.
- start_on_windows_single.py
- This program is the module that can call the all image processing programs on Windows and runs first. When executed, this program waits for messages from the ROS and calls the image processing program according to the received message.
- show_hand_game_win.py
- This program runs an image processing game while communicating with the ROS.

show_hand_game_win.py

This program runs the image processing game show hand game in which the player shows own left or right hand to the camera according to the ROS message. This program uses mediapipe to recognize hands.
You can test only this image processing game without communication with the ROS by using the following command.
- When executed, the command calls demo function in the show_hand_game_win.py, and runs demo of show hand game without communication with the ROS.
- We strongly recommend to implement the demo function like this even if you create new original image processing games.

python show_hand_game_win.py

This program's start_game function is called from start_on_windows_single.py when the recieved ROS message is "[shg]start show hand game".
- start_game function communicates with the ROS by sending/recieving message.
- For example, message_from_ros = ros_bridge_tcp.wait_response(pub_msg, hand_types, timeout=30) runs the following three steps
  1. It sends message pub_msg to the ROS
  2. It waits up to timeout=30 seconds for a message that matching the string in hand_types to recieve. (* hand_types is the list, and it has any strings)
  3. It store the recieved message in the message_from_ros if the message is recieved
- It is strongly recommended to prefix the messages sent/received with the prefix corresponding to the image processing game which is communicating with the ROS.
```
def start_game(topic_name_from_win, ros_bridge_tcp):
# definition of message types receiving from ros
hand_types = ["[shg]left", "[shg]right"]
```
Send start message and wait hand type selected by ROS

pub_msg = { "op": "publish", "topic": topic_name_from_win, "msg": {"data": "[shg]OK_start"} } message_from_ros = ros_bridge_tcp.wait_response(pub_msg, hand_types, timeout=30) if message_from_ros: print("\nReceive from ROS:", message_from_ros, "\n")

Decide your hand

window_message = "show only your " + message_from_ros + " hand!" left, right = show_hand(window_message)

Judge

result = judge_game(left, right, message_from_ros) pub_msg = { "op": "publish", "topic": topic_name_from_win, "msg": {"data": result} }

Send game result to ROS.

print(result)

In this program, ros never return to the following wait_response

ros_bridge_tcp.wait_response(pub_msg, ["[shg]OK_result"], timeout=5)

start_on_windows_single.py

After confirming that show_hand_game_win.py runs normally at alone, Confirm that the game show_hand_game called from start_on_windows_single.py runs normally while communicating with the ROS, as following command.
- Now, start_on_windows_single.py waits in the infinite loop for a message from the ROS with while True:, and exits the infinite loop when the message "The end" is recieved from ROS.

python start_on_windows_single.py

If you want to add a new image processing program that communicates with ROS (e.g. show_hand_game_win.py), you have to edit the following part of start_on_windows_single.py.
- if message['msg']['data'] == "[shg]start show hand game": is the conditional expression that will be True when the message [shg]start show hand game is received from the ROS.
- shg.start_game(topic_name_from_win, ros_bridge_tcp) calls start_game function of the image processing program (i.e. show_hand_game_win.py) that communicates with ROS.
- import show_hand_game_win as shg is also required.

    while True:
        messages = ros_bridge_tcp.wait() #Wait for message from ROS and assign the response into 'messages'
        for message in messages:
            # If message['msg']['data'] is "[shg]start show hand game", then shg.star_game is invoked
            if message['msg']['data'] == "[shg]start show hand game":
                shg.start_game(topic_name_from_win, ros_bridge_tcp)
            if message['msg']['data'] == "The end":
                pub_msg = {
                    "op": "publish",
                    "topic": topic_name_from_win,
                    "msg": {"data": "Every game has finished!"}
                }
                print("Every game has finished!")
                # Send message "Every game has finished!" to ros.
                ros_bridge_tcp.wait_response(pub_msg, ["Every game has finished"], timeout=5)
                break
        else:
            continue
        break

Make a ROS node

Open ~/catkin_ws/ by Visual Studio Code editor, and add the following files into ~/catkin_ws/src/oit_pbl_ros_samples/scripts/. See Developing inside the ROS container with VSCode.
Save the following two files into ~/catkin_ws/src/oit_pbl_ros_samples/scripts/ on ROS.
- If you want to download the files, click the following links and then download from a Raw button.
- start_on_ros_single.py
- This program runs on the ROS container, and then calls show_hand_game_ros.py which send the message to the image processing program on Windows.
- show_hand_game_ros.py
- This is the ROS program which communicates with the image processing game (i.e. show_hand_game_win.py) on the Windows.

show_hand_game_ros.py

play_show_hand_game function of this program communicats with show_hand_game_win.py on Windows.
Although this program is generally called from start_on_ros_signal.py as module, you can also run this program alone.
- In order to run this program alone, the following steps are required on the ROS before running the command.
- Allow the parmission for execution and run the launch program.
```
chmod u+x show_hand_game_ros.py
roslaunch oit_stage_ros navitation.launch
```
- Next, run the image processing program on Windows.
```
python start_on_windows_single.py
```
When run alone by using the following command, this program will run the game show hand game while communicating with Windows and exit without stopping the infinite loop on the Windows side.
```
rosrun oit_pbl_ros_samples show_hand_game_ros.py
```
The communication process between the ROS and Windows in show_hand_game_ros.py is as follows. Please read and understand the comments carefully.

def play_show_hand_game():
    rospy.sleep(3)
    node_name = rospy.get_name()
    # Prepare to play show hand game
    # Specify topic names to commnicate with show hand game
    to_win_pub = rospy.Publisher("/from_ros", String, queue_size=1)
    messenger = RosWinMessenger(to_win_pub, "/from_windows")
    # Start game sequence
    rospy.loginfo("%s:Try to start show hand game", node_name)

    # Send game start signal to Windows, and wait Windows side response.
    message_from_win = messenger.wait_response(
        "[shg]start show hand game", ["[shg]OK_start"], timeout=30)
    if message_from_win:
        rospy.loginfo("%s:Receive from win:%s",
                          node_name, message_from_win)
    else:
        rospy.logerr("%s:Timeout. can't start show hand game on windows", node_name)
        return "[shg]timeout"

    # Select robot's hand_type
    hand_types = ["[shg]left", "[shg]right"]
    hand_type = random.choice(hand_types)
    rospy.loginfo("%s:Robot selects '%s'", node_name, hand_type)

    # Send hand_type chosen by robot to windows show hand game, and wait game result
    message_from_win = messenger.wait_response(
        hand_type, ["[shg]correct", "[shg]wrong"], timeout=30)
    if message_from_win:
        rospy.loginfo("%s:Receive from win:%s",
                    node_name, message_from_win)
    else:
        rospy.logerr(
            "%s:Timeout. can't get show hand game result on windows", node_name)
        return "[shg]timeout"
    rospy.sleep(3)
    return message_from_win

start_on_ros_single.py

After confirming that show_hand_game_ros.py runs normally, Confirm that both start_on_ros_single.py and show_hand_game_ros.py runs normally, as following command.
- if you did not run the launch program, allow the parmission for execution and run the launch program on other terminal.
```
chmod u+x show_hand_game_ros.py
rosrun oit_pbl_ros_samples start_on_ros_single.py
```
If you want to add a new ROS program (e.g. show_hand_game_ros.py) that communicates with Windows, you have to edit the following part of start_on_ros_single.py.
- result = shgr.play_show_hand_game() calls show_hand_game_ros.py as module.
- import show_hand_game_ros as shgr is also required.

def process():
    rospy.sleep(10)
    node_name = rospy.get_name()

    print("---shg---")
    result = shgr.play_show_hand_game()
    rospy.sleep(5)
    end_game()
    rospy.loginfo("/* GAME:%s */", result)

igaki commented 3 years ago

@jinno-oit ありがとうございます！ Markdownファイルお願いしますーm(__)m

jinno-oit commented 3 years ago

なるほど，ここだとRawで見られないのですね．SlackのDMに投げておきます．m( )m

KMiyawaki commented 3 years ago

@igaki @jinno-oit https://github.com/oit-ipbl/Integration/blob/singletopic2/win_multi/win/start_on_windows_single.py#L39 で、Win側からはEvery game has finished!を送信し、ROSからのメッセージEvery game has finishedを待つ、というようにビックリマークの有無がありますが、使い分けている、ということですか？あと、StartとFinishのシグナルはもう少し簡単に（一単語に）してもよさそうな気がします。

igaki commented 3 years ago

@KMiyawaki finishedメッセージは統一しようと思います．それと，ご指摘の，シグナルを簡単にしたほうが良いという主旨はなんでしょう？ @jinno-oit 先生と相談したときにはOKとかだけだと，他のゲーム等のメッセージと重複するかもという懸念があったので[shg]といったprefixをつけることにしました．で，長さを短くすること自体は問題ないのですが，開始・終了が明確に分かるほうが良いのかなという意図で今のメッセージにしています．

KMiyawaki commented 3 years ago

@igaki 先生、 @jinno-oit 先生、シグナルを簡単にするのは単に間違いを防ぐためです。受信先識別のマークは当然必要だと思います。スペースありの文章よりは一単語の方がよさそうだと思いましたが、どちらでも結構です。

igaki commented 3 years ago

@KMiyawaki 長かったりスペースがあったりするとコーディング時に入力ミスが発生するのではということですね．確かにそのとおりのような気もしてきました（実際!のありなし間違えてましたし）．ちょっと考えてみます． @jinno-oit

oit-ipbl / Integration