The Cradle framework is a first attempt at General Computer Control (GCC). Cradle supports agents to ace any computer task by enabling strong reasoning abilities, self-improvment, and skill curation, in a standardized general environment with minimal requirements.
I don't understand where your universal computer control is universal. Moreover, the model still calls GPT-4, which is not local. Universal computer control should use a local large language model. Then, the computer operation corresponding to the shortcut keys directly interacts with the model. It can be multimodal, or an action selection agent that has mixed multimodal functions. I can only say that it is too simplistic at present. It seriously does not match the title, and the direction is a bit off.
I don't understand where your universal computer control is universal. Moreover, the model still calls GPT-4, which is not local. Universal computer control should use a local large language model. Then, the computer operation corresponding to the shortcut keys directly interacts with the model. It can be multimodal, or an action selection agent that has mixed multimodal functions. I can only say that it is too simplistic at present. It seriously does not match the title, and the direction is a bit off.