search

DigiRL-agent / digirl

Official repo for paper DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning.
Apache License 2.0
200 stars 16 forks source link

The model produces untranslatable actions #14

Open mousewu opened 1 month ago

mousewu commented 1 month ago

I try to reproduce evalution only results using the checkpoints you provided. The actions generated by the model (general-off2on-digirl) cannot be decoded.

Copying the AVD folder from /home/xyz/.android/avd/test_Android.avd to /home/xyz/.android/avd/test1.avd Starting the Emulator executing command /home/xyz/.android/emulator/emulator -avd test1 "-no-audio" "-skip-adb-auth" "-no-boot-anim" "-gpu" "auto" "-no-snapshot-save" -port 5554 connected! Saving observation! history: [] prompt: Previous Actions: Goal: What's a good restaurant in Seattle? Getting observation from emulator 0: {'prompt': "Previous Actions: Goal: What's a good restaurant in Seattle?", 'image_feature': tensor([-0.3622, 2.5739, -0.0125, ..., 0.2995, 1.8789, -0.5472]), 'task': "What's a good restaurant in Seattle?", 'image_path': '/home/xyz/logs/ckpts/general-off2on-digirl/images/test1/1723621834.599621_0.png', 'video_path': None} Environment stpes 1 getting actions! raw action: foloseste cauta incadr solutie incadr diferenta diferenta multumescgardinenbouchage expoziți insotit rosii foloseste ediți incadrprezinta senzati rosii gasest ediți expoziți Craciun Craciun edițiDatoritaDatorita tarziu duhovnic Clickfunnel Clickfunnelbouchage incalzirebouchage edițisuntetimunicipiul clientii duhovnic psiho duhovnic placut psiho serrurier foloseste Clickfunnel inregistrat folosestesunteti Clickfunnel diferenta rosii multumescDatorita senzatisunteti clientii duhovnic serrurier senzati solutieinformatiile inregistrat placut Clickfunnel inregistrat list index out of range Failed to translate action: foloseste cauta incadr solutie incadr diferenta diferenta multumescgardinenbouchage expoziți insotit rosii foloseste ediți incadrprezinta senzati rosii gasest ediți expoziți Craciun Craciun edițiDatoritaDatorita tarziu duhovnic Clickfunnel Clickfunnelbouchage incalzirebouchage edițisuntetimunicipiul clientii duhovnic psiho duhovnic placut psiho serrurier foloseste Clickfunnel inregistrat folosestesunteti Clickfunnel diferenta rosii multumescDatorita senzatisunteti clientii duhovnic serrurier senzati solutieinformatiile inregistrat placut Clickfunnel inregistrat, terminating the environment Saving observation! history: [AndroidAction(action_type=<ActionType.TaskImpossible: 7>, touch_point=None, lift_point=None, typed_text=None)] prompt: Previous Actions: "action_type": "STATUS_TASK_COMPLETE", "touch_point": "[-1.0, -1.0]", "lift_point": "[-1.0, -1.0]", "typed_text": "" Goal: What's a good restaurant in Seattle? skipping evaluation due to same images Previous Actions: "action_type": "STATUS_TASK_COMPLETE", "touch_point": "[-1.0, -1.0]", "lift_point": "[-1.0, -1.0]", "typed_text": "" Goal: What's a good restaurant in Seattle?

BiEchi commented 1 month ago

The previous action is already STATUS_TASK_COMPLETE, so your process should already be ended. Did you modify this logic?

BiEchi commented 4 weeks ago

Closed due to inactivity.

BiEchi commented 6 days ago

There might be some problems with the config.json file and I just deleted it. I did not observe strange things using the checkpoints stored on Google Drive. Would you like to delete the config.json file and try again? @mousewu

This issue might be a relevant one with #16 .