search

tensorflow / swift

Swift for TensorFlow
https://tensorflow.org/swift
Apache License 2.0
6.13k stars 608 forks source link

Python interoperability for FastAPI? #543

Closed INF800 closed 4 years ago

INF800 commented 4 years ago

Hi, with IBM ditching kitura project I have second thoughts about using vapour for REST API service. So, the first most option that comes to my mind is Python Interoperability and FastAPI.

  1. Can I know what are pros and cons for using FastAPI for exposing swift models for inference?
  2. Will there be any bottleneck because of interoperability?
  3. FastAPI is pretty fast but it's host language - Python is slow. I remember developers mentioning that Python interoperability project converts python code to native swift/assembly language. Consequently, FastAPI must become faster.. will it?
kongzii commented 4 years ago

Perhaps I do not understand, but why do you want to use FastAPI and Vapor together? Theoretically, you should be able to use FastAPI/Flask/Something in Swift but I would not recommend it. Python interoperatibility is meant for tools like pandas, for experimenting.

By the way, I have been using S4TF + Vapor for some time now and it is mostly OK, but sometimes you come across weird bugs. Because Vapor is built on stable swift toolchain but S4TF is based on latest, nightly, toolchains and therefore there is some incompatibility.

Also, python interoperability does not converts python code into native swift. It basically "just" calls python interpreter in the background.

BradLarson commented 4 years ago

Also, regarding Kitura, I'll point out that it's being transitioned to a community project, so I wouldn't totally write that out either. Most of the server-side Swift for TensorFlow projects I've seen have used Vapor or even SwiftNIO directly, however.

INF800 commented 4 years ago

I see. I had misconception that that python code is converted to native code. Thanks for clearing it out @kongzii.

INF800 commented 4 years ago

@BradLarson currently I am hosting my models using TFLite using FastAPI (something like this). Would I get almost same inference time when using server-side-swift? Or put it in other words, what is the BEST approach to deploy S4TF models on cloud?

saeta commented 4 years ago

Hi @rakesh4real !

This is a great question, but this is not something the core S4TF team is going to focus on in the short term. As a result, if folks in the community would like to put together some best practices to share, that would be great and we'd be happy to review!

In the meantime, we'll close this issue as it's not a focus of the team.

All the best, -Brennan