Reduce Streamlit dependency for better service integration

[vuhoanglam]

Description: Our library currently has a tight coupling with Streamlit across multiple functions and modules. While this works well for Streamlit-based applications, it limits the library's usability in other contexts, such as gRPC services, CLI applications, or integration with web servers written in other languages (e.g., Golang). We need to make our library more flexible to accommodate various use cases without sacrificing its current functionality.

Current behavior: Many functions in our library directly use Streamlit for:

• Status updates and progress indicators
• Error handling and display
• User input collection
• Result visualization and display

This tight coupling makes it challenging to use these functions in non-Streamlit environments and limits the library's overall flexibility.

Proposed change: Implement a dependency injection pattern to abstract UI interactions across the library. This would involve:

1. Creating an abstract UIInterface class with methods for common UI interactions (status updates, error handling, input collection, result display, etc.).
2. Implementing a StreamlitUI class that uses Streamlit and implements the UIInterface.
3. Modifying affected functions and classes to accept a UIInterface parameter instead of directly using Streamlit.
4. Providing a NullUI or MockUI implementation for non-UI contexts or testing.

Benefits:

• Maintains current functionality for Streamlit users.
• Allows the library to be used in diverse contexts (e.g., gRPC services, CLI applications, web servers in other languages).
• Improves testability by allowing UI interactions to be mocked.
• Enhances overall flexibility and reusability of the library.
• Facilitates easier integration with other services and frameworks.

[AJaySi]

Hello,

1). There is/was a CLI version, then came streamlit and now we are moving to next.js(react et al). You are absolutely right, decoupling interfaces from main LLM logic should have been the way to go. Bear with me as I explain some of the decision we made last 6 months.

In short: It would be better to implement API endpoints from Alwrty which can be consumed by any & everyone. What do you think ?

---

Explanations: (Not so short)

a). CLI version: As a linux cli guy myself, it was my first choice, but our audience are content creators & digital marketing, small business owners, startups, who arent CLI literate. Alwrity needs a clickty web interface for empowering its target audience. Empower, democratize AI for them, abstract prompting, AI tooling & saving them from monthly subscriptions. For non-tech end users, prompting is as complex as programming, as such that is the main focus and not making a sound library. Good prompting outputs good content with web researched context was the focus.

b). Streamlit UI : I am illiterate in UI, streamlit is our first foray (even our website is shi**ty) and having spent some time, its limitations to create a content lifecycle platform is evident. Designing UI for target audience who are used to inputs & clicks to generate good quality AI content. We had hoped getting community help but that hasnt arrived. We shelved the CLI as managing two interface would be difficult.

c). Nextjs UI: Motivation : While everything is a mess, our vision is going to moon and back. Alwrity is/will-be, one stop content lifecycle platform. Content planning, Content generation, Content editing/Review, Content publishing/scheduling, Content monitoring, analytics and iterate. Content implies text, image, video, audio and platforms are website, linkedin, facebook etc.

Roadmap: For the next 3 months, we plan to integrate and provide the following: 1). AI powered Editor built-in : TipTap.dev 2). Content publish : Will need to find cheaper 3rd party alternatives, for now, build our own later. 3). Content analytics across platform: Will need to find cheaper 3rd party alternatives, for now, build our own later. 4). SEO audits & Actionable insights. 5). llamaindex, embedchain, crewai will give capabilities of querying whole website, local documents and provide insights for generating new content, automate content lifecyle, analytics etc.

Presently, we are focusing on designing a UI that is simple, intitutive enough and work back to backend code. The backend's main code are prompts in this age.

A subtle design philosophy: I used to write OO with Perl(when there is None), implied it with python on linux, as end user was tech-savvy. With Alwrity, python functions made sense as standalone application which people could modularize with methods and what-not. Also, around 70% code is AI generated and human validated functionalities. LLM work best at function level rather than giving whole libraries of classes. For extendibility, enhancement for others & by LLM, alwrity became classless.

[vuhoanglam]

I completely understand your decision. However, even if we provide end users with a Streamlit-based version, we should still separate the interface from the logic.

I have experience in both FE and BE, and I want to use Alwrity for my CMS (written in Vue.js with a Golang backend), so I'm currently creating a headless version of Alwrity that doesn't depend on Streamlit.

I'm considering implementing either FastAPI so the Vue app can call Alwrity directly, or implementing gRPC, or using a message system like Kafka for better scale. However, since content generation is a long-running process, minimizing latency and payload size using gRPC seems a bit overkill.

Regardless of which approach we choose, separating the logic from Streamlit is the first thing we need to do. We can still maintain Streamlit and Next.js (in the future).

Although I have experience with React, for medium-sized projects, I prefer Vue because of its better performance and server-side rendering for SEO.

Fortunately, Tiptap is also my choice. I'm currently migrating my CMS from WangEditor to Tiptap.

In the future, we could also vectorize websites's article to support semantic search, suggest internal linking and related articles while using editor ( Can use LlamaIndex or build our own from scratch).

[AJaySi]

Great insights. Help me with below points:

1). Agreed on decoupling logic from interface. Help us prioritizing the below. Your experience will surely help to expedite point2 and point3 which will give us time for decoupling logic from interface.

Alwrity should first have decent UI for web users, for which we are thinking of getting done with a template and writing logic with typescript.

2). This is what we are planning and in process of buying this template to help us with web presence, boilerplate and start integrating features into it. Whats your advice ? We are very open to your suggestions and you should lead us here.

3). Present Focus: Have decent website presence, rank for keywords and where our end users can give us feedback on content quality & Ease of use. This gives us opportunity to let the end user drive our feature development efforts. Make some online noise for alwrity.

4). Basically, I am asking you this: How do we first given priority to a decent UI and then also work on decoupling ?

[vuhoanglam]

1. If you're planning to offer Alwrity as a SaaS (which I think is the best approach, as only publishers with a tech background like us would clone the project and start a Streamlit app, then fill in various tokens to make it run), you'll need a tech stack that's better in terms of performance and maintenance. Consider using a workflow engine and data orchestration. I've used about 27 workflow engines, and Dagster (which supports serverless), Prefect, Mage-AI, and Windmill (im currently making alwrity working on this) are the best options (not Airflow). A decent UI won't work without a REST/HTTP API backend. While you could use Nginx as a reverse proxy to expose the Streamlit app (and improve its appearance with CSS), that's not a common approach. Each task is a long-running process, and we need to update task progression and provide retry/cancel mechanisms for the UI. I'd advise against marketing Alwrity until the project is sufficiently mature, or users will come and quickly leave.
2. If you're considering buying a template from ThemeForest, I'd recommend sticking with utility CSS frameworks like Tailwind CSS or UnoCSS (the best, but not as popular). Why? Because we're all "sloth-driven developers" (lol), and AI UI/UX optimizers (like Anthropic's model) work best with Tailwind CSS. Moreover, Tailwind CSS is optimal for modern, component-based web apps. Trust me on this.
3. Additionally, if you're starting with React/Vue, choose a template with the components we need or use a UI library like Refine.dev for React or PrimeVue for Vue. Converting from HTML templates to components is time-consuming. Having an attractive and next-gen website early on is a good direction, but that's up to you.
4. If you choose a suitable workflow engine, decoupling logic will be straightforward - it might just involve creating tasks and inserting Python/TypeScript/R/Lua code into them. Lastly: To provide Alwrity as a service, you need to consider use cases like two users simultaneously researching one keyword and writing long-form content. How will you handle both sessions concurrently, address history viewing needs, and what if they search for the same keyword, etc.? Also, requiring users to register for multiple third-party tokens can be discouraging. Consider registering and calculating a universal computation unit, then charging fees or providing free access with limits.

Very last: We should stick with Python because almost all NLP libraries are readily available in it. Additionally, there are numerous Jupyter notebooks and Google Colab resources available for reference.

[AJaySi]

Thank you for your insights. You have cleared a lot of confusions and provided future directions.

1). I will spend time and research/understand more on windmill workflow engine, Tailwind CSS, refine.dev, primeVue and ask more relevant questions soon. I did like coolify, strapi and such similar directions.

2). Alwrity target audience wont clone & enter 20 API keys, hence a webapp. Streamlit was temporary to atleast show some UI to encourage adoption and not black & white console. Alwrity's target audience is not on github and they only trust 'attractive & next-gen' websites.

3). Performance, multi-tenancy & complex usecases are features that will be built later. There are two solutions(read procrastination) for immediate requirements:

a). Our end-user do not fully realize the worth of AI or feel competitive or only good programmers can prompt better. They are web searching for very small AI tools like title, meta description, instagram generator, blog, social media posts et al. This is the set of simple tools to provide them and handhold them into complex all-in-one content platform. The roadmap i mentioned above is months away.

For next 2-3 months, we only provide these simple tools without fee and get them used to SaaS webapp with an AI enabled editor (without collaboration) and allow publishing. Focusing mainly of single users and encouraging Devs to use on localhost.

b). Next phase : Build multitenancy, collaboration with teams with Open-source cloud managed APIs. Pay for it till we make our own. I had mentioned some these services above.

Lets stick to python.

Is there a chance we can collaborate on Alwrity, so that you can make it better for all ?

Motivation: Its only to empower content writers with AI and not something for big companies only.

[vuhoanglam]

sure, i will update my progress in trying to get alwrity running with one of the above workflow engines

[AJaySi]

Great News. Thank you. Alwrity now !

So, are we going with FastAPI ? Performance is at par with nodejs, go for python is awesome.

I will update my progress with the new UI, #62 and AI editor.

Novel Editor is based on Tiptap for react apps and it has AI editing available. TipTap AI editing is 'coming soon'. The drawback with Novel being dependent on AI vercel SDK. I am looking into tiptap to provide AI editing and checking out how difficult/easy it would.

[vuhoanglam]

Don't worry. The AI feature of Novel is simpler than you think, it's all contained in this file and can be easily replaced with a FastAPI Python backend

https://github.com/steven-tey/novel/blob/main/apps/web/app/api/generate/route.ts

[AJaySi]

Ok, so novel is not tightly coupled. Lets go on the novel pursuit. I did not want to include AI vercel SDK, we can write most of the AI capabilities ourselves. One of the reasons, i dont like langchain and avoided llamaindex till now.

BTW: AI-editor & AI-excel are great projects to undertake as all commands for these can be replaced with AI, at least the complex excel formulae & editing options.

[vuhoanglam]

Hey bro. I have an update. Do u have discord

[AJaySi]

Hello, Nope, no discord or any other social media logins. It seems your workflows are in place. Congrats.

[vuhoanglam]

okay