OpenInterpreter / 01

The #1 open-source voice interface for desktop, mobile, and ESP32 chips.
https://01.openinterpreter.com/
GNU Affero General Public License v3.0
4.98k stars 533 forks source link

consistent Cross-Platform App/GUI #106

Open Bratzmeister opened 8 months ago

Bratzmeister commented 8 months ago

Is your feature request related to a problem? Please describe. I've seen many people ask for frontends/apps here and in other places for specific devices and OS. In your Roadmap you also mention a secluded app which I think is problematic and not scaleable.

Describe the solution you'd like A cross-platform framework like Google's Flutter is a very good solution imo. I made my own experiences with it and it's super easy and the results are awesome.

Describe alternatives you've considered Theres also other similar approaches like Kivy but I think Flutter is the best at the time of writing.

Additional context flutter.dev tutorial this one is also pretty cool to mess around with and learn about possible widgets etc: https://flutterflow.io/

mysticaltech commented 8 months ago

Flutter would definitely be ideal. The client needs to be a wrapper to the python one, but it's completely doable, something like this would help: https://github.com/IVLIVS-III/dart_python_ffi

n7trd commented 8 months ago

If the aim is to create UIs/Apps that can be deployed across various types of screens, Flutter is an excellent choice to consider. While some may argue in favor of React Native for mobile devices, Flutter has distinct advantages when it comes to desktop platforms.

However, where Flutter truly stands out is in its ability to create custom user interfaces for devices that lack a default UI framework. For instance, you could use Flutter to build a tailor-made interface for a RPi or any other device you can think of. https://www.youtube.com/watch?v=_61l_DJpChU

Bratzmeister commented 8 months ago

yeah you guys put it in great words :) I had just seen that @tyfiero @KillianLucas @MikeBirdTech are working on a react native app but since there is no UI for desktop, why not make one consistent app for all platforms and screens. PS: flutter can also compile to JS for using it in a browser

mysticaltech commented 8 months ago

@tyfiero @KillianLucas @MikeBirdTech IMHO, @Bratzmeister is right, react native is not the right call. You will get way more sway from Flutter. And tools like FlutterFlow can do most of the work for you, via AI, no need to bother. Compile once for all platforms, literally.

mysticaltech commented 8 months ago

Nothing comes even close to targetting so many platforms with one codebase. And that's just the official support, community packages supports all the rest like watches OS, Raspberry, etc.

Flutter is THE code once, run everywhere platform.

Screenshot from 2024-03-24 05-45-39

sidevesh commented 8 months ago

Smartwatches seem to me like the most ideal form factor for an 01 app, you can have the same press a button speak and then hear a response experience on a device that you already have literally strapped on your wrists at all time.

Flutter does have some unofficial support for Wear OS, no support for Apple Watch though, I think a native app for these devices would make more sense anyways to be as efficient as possible as well as access to all the APIs unique to these devices,

@tyfiero @KillianLucas @MikeBirdTech I will be looking into building a Wear OS app and would love to upstream it if y'all think it makes sense for the project.

rrrodzilla commented 8 months ago

I pretty much bring nothing to this table. So, I'll just play devil's advocate, given the strong support for Flutter in this discussion. Flutter does seem like the best option unless the target platforms need features that require custom platform channels. In that case, you'd need to write custom native code for each platform, complicating the architecture and undermining Flutter's advantage of a single codebase.

Also, I'm unsure about the project's guiding principles, but if making it accessible to developers is important, requiring them to learn Flutter and Dart could deter many, potentially excluding a significant portion of the developer community.

In this situation, initially adopting Kivy's markup and Python approach could better serve community growth, allowing for a gradual introduction to more advanced architectures for those interested.

That being said, another consideration I didn't see mentioned in this thread is any consideration for binary size given the device-based deployment targets. Both Flutter and Kivy+Python can have relatively bloated binary sizes (even with tree shaking in Flutter). Less resource consumption means lower BOM costs and more room for better user experiences on resource constrained hardware.

...my unsolicited 2 cents.

Bratzmeister commented 8 months ago

don't forget we are talking about gui frontend here and it's going to be very simple. you don't actually need to learn a lot of dart for flutter and it's like any framework that you need to learn. Also the build system and dependency management is super easy and painless with dart.

rrrodzilla commented 8 months ago

Well that's fair. But if it's truly that simple of a GUI and simplicity is a key driving metric, why the whole need for the Flutter SDK when a simple pip install kivy and a few lines of Python will do for a basic cross platform GUI?

Also, xcode on Mac is a hard requirement for ~developing~ deploying to MacOS desktop with Flutter. But, for example, if you're only building for the desktop, you don't need to bother with xcode to ~create~ deploy a desktop app to Mac with Kivy; it would win here as well in a simplicity scorecard.

Bratzmeister commented 8 months ago

you just run binary, no installing what so ever. you also dont need xcode to develop for mac. you can develop for all platforms and generate binaries for all platforms from lets say vscode.

tyfiero commented 7 months ago

I am open to a cross-platform flutter app for 01 if someone in the community wants to help build it. I have no experience with Flutter or Dart, nor does anyone on the core team, so we won't be able to support its development from a technical standpoint.

But again, I'd love to see an 01 flutter app built by the community if anyone is interested in building one!