Web/App Generators? e.g: websim.ai, tabby, io.google, v0.dev, claude3.5

[ndrean]

Take a look at this.

It creates for you a website. Yet another AI assisted tech?
Yes, perhaps, but honestly, its pretty good! I mean, it populates based on the theme you give, and understands your orders. If you say, "I want a web site to promote herbal teas (why not!) with pictures inspired by Zen philosophy and related to Japan, and presenting my products in cards, and only zooming on the picture when you pass over the pciture, it does all this. You can create games etc. You can check what other created, and I checked DOOM. You can download the code, check the HTML, CSS. Pretty good! For the lest gifted like me, I can say that it gives quickly a solid foundation to prototype something.

Disclaimer: I don't make any money from this, its free, so you must be the product, but I did not understood how the creators think they will make money, nor how they can finance the server costs to make this work.

https://websim.ai/

[nelsonic]

Had a quick look. Could be promising. I've been saying that "AI" will replace all Dev for a while ... The allure of "AI" building a website/app for us is too tempting for many people to resist. 🤖 For now it's just templates and generators (i.e. smoke and mirrors). 🪞 I wish I could agree with you that an "AI"-generated site provides a "solid foundation" ... 💭 Sadly, it's more of a "Tofu-dreg" that only the "AI" can maintain. And if people don't learn how to extend and maintain their idea it dies.

[ndrean]

@nelsonic you are full of clever citations!!! Tofu-dreg!!! Indeed, its kind of, but I have not talent and no patience for the CSS side, so this would give me a kind of base. Not expecting more.

I know a lot sof people use copilot, but unfortunately, I don't have a free one. But I found https://tabby.tabbyml.com/. Seems like Yet another copilot, free (for the moment). I always wonder how people can afford to pay servers for this. What is the product?

[nelsonic]

Copilot is good for certain tasks but I found it gets in my way a lot too so I've disabled it. 🙅 I was an early beta tester and am still hopeful for it. 🤞 However it will rapidly become a "crutch" for people who think they are "10x Engineers" because they appear to be producing lots of code ... but if they don't understand it and nobody else in their team can either ... 😬

[ndrean]

I don't have this problem 😄. I don't expect it to code for me, but instead I looked into this because I thought - like I imagine 99% of the people - that it could be nice (?) for:

• helping at naming variables or functions,
• pasting some code and ask for some explanation (I tried with mine).
• proposing some skeleton of code when you use some case or switch, things that are repetitive,
• maybe help and understand errors
• ... I am targeting wildly used languages, such as Javascript or Go. I don't expect anything for Elixir.

So , tabby ? Maybe. Tools seems to pop-up all the time and improve quite a lot.

I've been saying that "https://github.com/nelsonic/nelsonic.github.io/issues/733" will replace all Dev for a while ...

Since you can import the server messages in a LiveView app, you can also potentially debug Elixir errors? No idea if Elixir knowledge is effectively developed by the LLM.

[nelsonic]

https://v0.dev

[ndrean]

A thread from Elixirforum (which neither uses an Elixir based framework for its forum by the way).

[nelsonic]

Thanks for sharing the link to the Forum topic. 🔗 🤖 💭 https://github.com/paul-gauthier/aider + https://www.anthropic.com/news/claude-3-5-sonnet looks like a good combo. 👌 Have you tried it?

I don't have time to dig deeper right now, but it looks promising! 💭

[ndrean]

Have you tried it?

Yes, and made me think of a wise response:

I've been saying that "https://github.com/nelsonic/nelsonic.github.io/issues/733" will replace all Dev for a while ...🤔

More seriously, it is good in some sense. An Elixir-RAG that digested the whole Elixir GitHub might be good, but not quite yet. My example if you have patience and time to lose. I want to do a rather simple task: compare image captioning techs running pre-trained models, either server-side and in the browser. I was lazy so asked for some JS code for ml5.js, mediaPipe and transformer.js to implement it in the browser. I had a dilemma: I needed the models to evaluate the same image that I got randomly from "picsum". Should I fetch it from the browser, send to the server to run the model, and in parallel run the client-side models, or should I get it from the server, run the model and in parallel send to the browser and run the client-side models. It responded with a server-side request (I specified Req). Pretty good, Bumblebee, Exla, but failed to convert the image into a tensor, and forgot to declare the Nx backend (and so did I...) . Then it converted the image into base64, to populate the <img src=...> that is rendered via an assign server-side, and then run the models on the DOM element. It was better than me because it reminded me to pass the assigns in the mount as I did not do it and nothing rendered. However, this means that the image data are sent via the LiveSocket as a string. I asked Claude who will transform an image into base64 and increase the payload of 30% as well as latency? I suggested solutions such as:

• file upload,
• pubsub the binary from the LiveView to a Channel, and send a binary via the userSocket (or whatever name you give), and directly use "onmessage" on the userSocket to get a binary payload.
• setup a raw WebSocket directly instead of the Channel overhead, directly -30% data transfered. It get mixed up, so I showed the solution using Phoenix.Socket.Transport.

A little complain. The Bumblebee library wants a tensor; honestly, when do you have a tensor under the hand? Most probably a file, or a buffer. Compared to the client-side, they all simply get a DOM element - say <img> - as input and work on it. So more than the response of Claude, I thought that indeed, Bumblebee was kinda rough so I excused Claude.

Then for some reason, I could not render anymore: I had an error "no root layout" ?!! I had no idea of what was happening. Yes, I compared file by file to try understand but I lost patience and it did not help me.So many layers of Phoenix niceties that I gave up! => I created a brand new project, copied the code back and it worked back. I was hoping Claude to be more clever than me, but no, my brutal force worked and was faster. Arguably, I did not learn anything by not finding the problem but Claude neither.

So the code Claude produced is pretty good in some sense. Arguably, my subject is not rocket science. Also I don't have the feeling of working with a senior dev, but rather with a good helper. I also tried in parallel Chatgpt as for some reason I seem to have access to v4. I would say that ChatGpt seemed less accurate. I corrected the code for mediaPipe and ML5 as it seemed to use an older API. Saying that, I almost used "him" instead of "it" when I talk about Claude of ChatGPT. I'm a bit schizophrenic, aren't I?😄 I hope I corrected.

Now, I was thinking of letting Claude produce a kind of Dwyl post on using Websockets and Channel to pass binary data. If you are interested of having this, I will do it (mostly because I will forget how to do it!). But I will continue as long as it is free because I enjoyed it! Almost never got stuck with a stupid problem.

[ndrean]

Correction! Claude IS good. An example. I want to secure a raw WebSocket connection by passing a user token and get it checked when connecting. This is basic. But I have a problem when I declare "check_origin: false". Sobelow does not like this. I needed help for this, as I am 100% sure I will to it wrong with the common issue with runtime vs compile-time configuration. Just hard. For example, betweenSystem.get_env and System.fetch_env!, one work, not the other, Application.compile_env etc... Unless you are versed into this, it is hell. Well Claude solved this for me with NimbleOptions and it seems to work!

[ndrean]

So I played with Claude. He is my wonderful week-end work, LiveView and passing binary data with Channels or raw WebSockets. At least I learnt something!

https://github.com/ndrean/phoenix-websockets

I would say: "don't rely on it". I asked for some JS code to perhaps zip or deflate images I sent. It happily proposed pako. Then I suggested to use the native compression stream, but the JS code wasn't good. The good part is that the code is not difficult. I ran it but discovered that compressing an image is useless, as JPEG is already compressed. So I asked why I hardly saw a difference and Claude responded that indeed JPEG is maybe not a good fit.... a bit of a waste of time wasting, isn't it? I asked Claude to explain a bit the code, but most was paraphrasing. Some good bits however.

I may try to run this on Fly, just to see how this demo works.

If you think it is valuable work for you to have this as a repo in Dwyl, let me know.

[nelsonic]

Very interesting! How much of this was Claude output? 💭

[ndrean]

Claude produced the solution when I wanted to send a "large file from the server to the browser. The problem was to pass indexes.

But don't copy bindly....

Claude produced this server-side:

File.stream!("channel.jpg", 1024 * 10)
    |> Stream.with_index()
    |> Enum.each(fn {chunk, index} ->
      push(socket, "new chunk", {:binary, <<index::32, chunk::binary>>})

https://github.com/ndrean/phoenix-websockets/blob/d2b1d332ba05f21a4611cf1a7462f4f04905d4d9/lib/ws_web/channels/image_channel.ex#L24 You see the trick: the JS should read the data as an array of bytes, and read the 4 first bytes....to glue everything properly by index

and produced this client-side:

let view = new DataView(payload); 
let index = view.getInt32(0);
let chunk = payload.slice(4);
imageChunks[index] = chunk;

https://github.com/ndrean/phoenix-websockets/blob/d2b1d332ba05f21a4611cf1a7462f4f04905d4d9/assets/js/channelHook.js#L18

Th rest is "standard", as Channels document passing binary data (I found another clue in the Elixirforum regarding Phoenix.Socket.Transport, a response from J Valim).

Claude refactored and corrected my JS code when sending big files from the browser to the server:

https://github.com/ndrean/phoenix-websockets/blob/d2b1d332ba05f21a4611cf1a7462f4f04905d4d9/assets/js/channelHook.js#L74

[ndrean]

By Charlie Holtz

[nelsonic]

Yeah, it's only going to get better. Have you tried Melty? 💭

[ndrean]

No, not yet. I have only a small M1 with 8GB of RAM. I put it here in order to come back later. One day perhaps, an Ubuntu PC with 32GB of RAM and a solid GPU....

[ndrean]

And waiting for the AI bubble to burst to get a cheap second hand GPU 🤔 Seriously, so many apps using AI/GPU! How can they pay the bill? Not talking about who makes the money, but just thinking of all the electricity you must produce for air conditioning and powering all this. Norway should be top1 in the world for green washing this: no air conditioning 😄 and incredible amount of hydropower for free.

[nelsonic]

Which GPU will you get? 💭 and is your electricity price cheap enough to power it? ⚡ 💸

Yeah, the energy demands for all these chips is insane! 10-20x more energy per query than a regular Google search! Norway, Iceland and Greenland are all well-positioned for building epic data centers.

When the Ai Bubble 🫧 bursts, and it will ... the whole economy is coming crashing down hard 📉 😬 There's been waaaay too much CapEx on Ai hardware in the last few years. 💰 🔥

It's incredible how intel (INTC) has missed the boat on "Ai" while NVDA has sky-rocketed! 📈 🚀 🤯

The "investors" (speculators) who rode this wave and got out before the bubble burst are laughing so hard on their yachts ... 💭

[ndrean]

In France, its around 0.25Eur.

What seems surprising is how Apple tackles this trend. Once again, it is a closed ecosystem. You can't plug a GPU into your mac. I know there is Metal and Core ML but I am not sure how you can leverage them.

So I asked to my new friend (I have no idea if ChatGPT is hallucinating or not...):

Question: Can I use Apple Core ML with the Elixir language on an M-serie?

Response:

---

Using Apple’s Core ML directly in the Elixir language isn't straightforward because Core ML is a framework designed for use with languages like Swift and Objective-C, which are native to Apple's ecosystem. Elixir, being a functional language that runs on the Erlang virtual machine (BEAM), doesn't have direct bindings to Core ML. However, you can still integrate Core ML models in an Elixir-based project by using a few different approaches:

Approach 1: Using Ports or NIFs

1. Ports:

   • Ports allow Elixir to communicate with external programs written in other languages like Swift or Objective-C.
   • You could write a small Swift/Objective-C application that uses Core ML to perform machine learning tasks, and then use Elixir to interact with that application via a port.

2. NIFs (Native Implemented Functions):

   • NIFs are a way to run C code directly in the Elixir VM. However, NIFs are generally not recommended for heavy computations or tasks that could block the VM.
   • You could theoretically write a NIF that bridges to a Core ML model, but this would involve writing a lot of C and potentially Swift/Objective-C code.

Approach 2: External Microservice

1. Microservice Architecture:

   • Another approach is to create a separate microservice in Swift (or another language that can use Core ML) and then interact with it from your Elixir application via HTTP or a similar protocol.
   • This service would load and run the Core ML models, and your Elixir application would send data to it and receive predictions.

2. Inter-process Communication:

   • You could also use a messaging system like RabbitMQ or Redis to communicate between your Elixir app and a separate Core ML service.

Approach 3: Using Core ML via Python

1. Use a Python Bridge:
   • Since Python has many machine learning libraries and can interact with Core ML (using the coremltools library), you could use Elixir’s :erlang.port_open to run Python scripts from Elixir.
   • The Python script would handle the Core ML model, and Elixir would handle the rest of the application.

Summary

While Elixir doesn’t natively support Core ML, you can integrate Core ML functionality into your Elixir projects by:

• Using ports or NIFs to call out to Swift/Objective-C code.
• Setting up a microservice in Swift or Python that your Elixir app can communicate with.
• Leveraging Python as an intermediary between Elixir and Core ML.

This allows you to benefit from the M-series chips and Core ML while developing in Elixir, albeit with some additional complexity.

[nelsonic]

https://support.apple.com/en-us/102363

[ndrean]

Yes, with Intel chip, but this is old. Even I have a Silicon chip! 😁

Of course, I am focussing on the dev mode, not the production mode where OSX does not exist anymore.

But I am still puzzled:

[nelsonic]

Meanwhile ... https://github.com/nelsonic/nelsonic.github.io/issues/912#issuecomment-2328366890 🤯

[ndrean]

But everything is not lost! see the current work

[nelsonic]

@ndrean recommend watching: https://youtu.be/U_cSLPv34xk 📺 Thoughts? 💭

[ndrean]

FULLY AGREE with him.

But, saying that, who cares if I agree with him! 😀

Why are there so many experts/gurus on this new AI paradigm? "everybody is a CEO, and everyone is lying" ... to themselves firstly 🫢 But wait, why??

It is hard to say smart things, so let's stick to small facts.

ML is good for a few scoped tasks, but certainly not at developing a full project, at the time of writing (saved 😬). Firstly it invents a lot. A good parametrised generator can do a better job. Then, you are also limited with the length of the context you pass to an LLM, and the responses suffer from this.

However, its capacity of "inventing" can be very useful. Aren't (pretty? very?) good at hallucinate things for you like images. For example, ML is also known to be quite good at finding clusters, similarities, discovering patterns when using various algorithms. ML is known to be good in recognition (OCR, captioning...) and fields related to technical document search: law, patent, technical documentation.....Even mathematicians use them. In general, fields where you might need an advanced technical workforce exploring data in a semantical way, not in a linear way. The problem of these industries is more about building an efficient permanent data ingestion system for the search engine(s).

For the Elixir eco-system, still need much more fine tuning. Check my little RAG pipeline test on the Phoenix Liveview documentation to see through an example how a RAG pipeline improves the LLM's inference.

[ndrean]

Things improve quickly. Take a look at Google's Gemini 1.5; as explained in the paper below, it accepts 1M tokens, up to 10M as the window context, or 30.000 LOC up to 300.000. A significant improvement which might make the RAG even better.

In particular, they demo interacting with three.js. This works as said with Javascript, but probably not with small languages like Elixir and evolving framework.

The big limitation: the costs of using this. For example, the "small" Gemini Flash:

[ndrean]

If you are looking for something to read, the paper below explains the possible next gen models surpassing the transformer model (2017): it is called SSM (2021). It is supposed to be fast and supports long context (what the RAG does in some way).

An implementation: the Jamba 1.5 . You may need .... 2 80Gb GPU🤨 but - good news - a quantized version (8-bit precision) may only need 1 GPU. We are saved 🥳

https://huggingface.co/ai21labs/AI21-Jamba-1.5-Mini

The SSM explained:

[ndrean]

A new toy to test: Codeium (it's like Copilot)

Training: Codeium challenges "only" GPT-3.5, so it must not be so good....

FAQ: Elixir?