The term "free" seems pretty ambiguous in it's context these days..

I can use chatgpt for free in so many places and applications, but I'm not able to find a way to use this program for free at all. It's VERY deceptive in it's phrasing. Yea, like, the program is free, but it's fucking useless without paying a huge persistent usage fee.

Please tell me I'm wrong, cuz this looks insane, but it's insanely useless if I can't even access the core point of the thing without holding the pocket of yet another over-privileged corporation.. That's like saying Automatic1111 is 'free', but you gotta pay a fee for every image you create with stable diffusion (hypothetical).

imagine crying that someone gave you a free car but you still have to buy gas....ambiguous my rear end you're a brat

imagine crying that someone gave you a free car but you still have to buy gas....ambiguous my rear end you're a brat

Naw, your logic is way off, or non existent. It's more like being given free gas but having to pay for a car. Especially given what/how openAI is charging for their AI. I thought monthly fees for programs like Photoshop, and well, so many other things these days, were bad enough, but pay per use is the all new low..

Just sick and tired of corporations putting paywalls on everything. And, before you say it, I already considered making my own. I've got all the time in the world right now, so learning how to program would be easy enough. But programming my own model wouldn't even be the hard part (as much work as that would take to do solo).

The real road block is capital requirement. You've basically got to have millions/billions of dollars in hardware in order to reasonably train AIs like ChatGPT.

You can download and train some models that are 'open source', but they're still technically corporate owned and beholden to the terms and conditions that you must agree to in order to use them.

That's not a completely dead option for me, yet. As I have not yet looked into the full scope of implications of agreeing to such contracts. Though it may be logical to assume that they are geared in such a way that would be detrimental to anyone that isn't relatively wealthy already.

hey @Xephier102, I was disappointed to see that you must have an API key to use this service, but maybe you can try forking this repo, opening it in CursorAI, and then asking Cursor to help you adjust the project to use only FOSS LLMs or, if your hardware is capable, to use your own LocalLLM via something like Ollama, OpenLLM, or LM Studio.

More here: https://medium.com/thedeephub/50-open-source-options-for-running-llms-locally-db1ec6f5a54f

Also I would disagree that this program is useless! It's a free car whether you have gas or not, and you have the option to modify the base to suit your needs. It's awesome!

imagine crying that someone gave you a free car but you still have to buy gas....ambiguous my rear end you're a brat

Naw, your logic is way off, or non existent. It's more like being given free gas but having to pay for a car. Especially given what/how openAI is charging for their AI. I thought monthly fees for programs like Photoshop, and well, so many other things these days, were bad enough, but pay per use is the all new low..

Just sick and tired of corporations putting paywalls on everything. And, before you say it, I already considered making my own. I've got all the time in the world right now, so learning how to program would be easy enough. But programming my own model wouldn't even be the hard part (as much work as that would take to do solo).

The real road block is capital requirement. You've basically got to have millions/billions of dollars in hardware in order to reasonably train AIs like ChatGPT.

You can download and train some models that are 'open source', but they're still technically corporate owned and beholden to the terms and conditions that you must agree to in order to use them.

That's not a completely dead option for me, yet. As I have not yet looked into the full scope of implications of agreeing to such contracts. Though it may be logical to assume that they are geared in such a way that would be detrimental to anyone that isn't relatively wealthy already.

This tool would be he free car, and the api models are the gas. My logic is perfectly solid, youre just a young dumb spoon fed liberal with no job. The api usage is so insanely inexpensive most people don't worry about it. I'm sorry that would require you to get out of bed to beg your mom for money, but you could awalys just get a job.

"gpt-4o-mini

Pricing Pricing with Batch API* $0.150 / 1M input tokens $0.075 / 1M input tokens $0.075 / 1M cached** input tokens $0.600 / 1M output tokens $0.300 / 1M output tokens"

Get a job, it's insanely inexpensive

I'm guessing that you don't actually know much about business, or historical trends. So, let me paint a picture for you. Modern businesses commonly employ drug-dealer type strategies. So, where you see 'That's a great deal!!!' I see a gateway to an arbitrary dependency. It's a very common strategy for corps to offer a service/product for cheap, then over time it gets increasingly more costy, and/or they massively reduce the quality of said products/services.

spoon fed liberal

Not backing greedy corporations for progressively putting more and more behind paywalls in the form of monthly fees and fee per use, somehow = me being a 'spoon fed liberal' better than being a brainwashed fascist IMHO. 'spoon fed' you wanna know who the most spoiled/pampered people are in our society? Approx 30-60% of the people running it. Those people have never worked a hard day in their lives, or they got one summer job at McDonalds, and use that as their entire campaign for having 'come from nothing'. They get their ivy league education payed for by their rich parents, and their entire lives mapped out for them, right from conception.

And no, to cut off your predictable incoming brainwashed rhetoric, I'm not 'jealous'. Well, that's a partial lie, who wouldn't be jealous of making a million dollars in a year just for sitting on their duff as a glorified accountant in their top-story corner window office. But the point isn't the idea of 'oh boo hoo, they have more money than me' because obviously, that would just be stupid. The point is the ever increasing wealth gap, and the lack of a wealth cap, when corporations like Nvidia can make 2 trillion dollars in under 6 months, and have more money/power than many entire countries.

Many choose to be homeless rather than being forced to live like cattle, 3-6 to a tiny apartment, but that's what most the lower class is forced to live like. I personally chose to avoid the trap of being a homeless addict or stuck in a min wage job workin like a slave and living like cattle until rents inevitably take another big jump and push me onto the street. So I live in subsidized housing, go to school in the summer, and spend most my day training myself on coding and AI software through the winter. So, within the next five years, I should have a solid career. I've already spent a lifetime doing underpaid-overworked jobs where I was underappreciated for my efforts.

These days, we've got a society full of people like you that are anti-humanitarian, corporate cheerleaders. Not good people. The only thing you care about is yourself, your vanity, your pride, and maybe the handful of people close to you. Yet, you somehow think your mindset is better for society than mine. But that's because you've been trained to believe that slaving away for the corporations that are exponentially monopolizing the entire market economy, is 'helping your community' or 'being an asset to your society'.

Purdue Pharma, Johnson & Johnson, Boeing, Volkswagen, Monsanto, Takata Corporation, Wells Fargo, DuPont, ExxonMobil, Pfizer.

Look up any of those companies along with the keyword 'scandal'. It's not conspiracy theory, every one of those and many more have been taken to court and proven of serious scandals that scam people, hurt people, make people sick, and even kill people. All in the race to maximize profits. And people like you, think that helping corporations that care so little for human life that they'd run the bulk of us through a meat grinder if it could be proven to be best for overall profit gain, and wasn't illegal. But y'all gonna help them to gain enough capital to eventually overtake governments so that they make the laws; and you think people like ME are the bad guys? Maybe time to step back and look at the bigger picture.

4 most predictable responses to this.

Dumb parrot response. Throw insults, repeat some common rhetoric about free market economy, or fling more mindless insults about getting a job or w.e. Approx 1-2 sentences, commonly followed by blocking the other person so their obvious lack of wisdom can't be further showcased.
No response at all, IE: ghost. K'bye!
Smart parrot come back with well constructed common rhetoric. Mostly the same as dumb parrot, but without the arbitrary insults, and usually padded with more info, but still not actually researched or validated in a real world context, or done so but while ignoring the entire lower working class and a portion of the lower-middle class.
Fourth and least likely, smart response. These rare responses will occasionally come from smart humans in disguise as corporate cheerleaders. But, given that you've already attempted to discredit me at least twice in a single short response, I highly doubt you're the least bit interested in the facts if they don't back up your current pre-fabricated world views.

That said, let me reiterate, I admitted that those current prices aren't exactly off the charts, mind you, I have no doubt that those are in addition to the monthly fee for gpt4.0, which was approx 25/mo last I checked. But, the overarching point, as I described was in the logical future projections; especially if I am correct about the monthly fee requirement.

25 per month + usage fees seem great to start, but that's also how smartphone plans started, and now they're over double-triple that (CAD), and we've still got those to pay, plus Netflix, plus rent, at a bare minimum. Just for basic non-miserable survival, though granted that assumes you go nowhere and do nothing, ever in your free time.

Go ahead and bash me cuz I don't slave for a corporation. But I've got a 57 year old friend who's built like a 12 year old boy, got a bad back, and a hole in his heart, and he's working 48+ hrs a week, and still struggling to attain a stable place to live(for the past year now). Why should anyone bust their ass off for a system that doesn't even take care of it's elderly. And I live in Canada, we're supposed to be the liberal country. I can only imagine what a hell it's like in parts of America.. I have heard stories though, hard working people that get shit on for all their efforts.

Lets dismantle this self entitled garbage socialist rhetoric diatribe one ideocracy at a time:

"> I'm guessing that you don't actually know much about business, or historical trends. So, let me paint a picture for you. Modern businesses commonly employ drug-dealer type strategies. So, where you see 'That's a great deal!!!' I see a gateway to an arbitrary dependency. It's a very common strategy for corps to offer a service/product for cheap, then over time it gets increasingly more costy, and/or they massively reduce the quality of said products/services."

Im a 41 year old survivor of childhood molestation and abuse by the people who i trusted to protect me growing up. Once I broke free at 15 of the horrible abuse and left school, I went to work full time for my grandfathers painting business, for about 18 years. Im a 5th generation master painter. By the age of 18 i had 3 jobs and 2 cars of my own.
You thinking that people should just "spoon feed" you what cost them money to produce is just ridiculous, self entitled and the entire reason why communism, marxism and socialism HISTORICALLY destroy nations.

``>

spoon fed liberal

" Not backing greedy corporations for progressively putting more and more behind paywalls in the form of monthly fees and fee per use, somehow = me being a 'spoon fed liberal' better than being a brainwashed fascist IMHO. 'spoon fed' you wanna know who the most spoiled/pampered people are in our society? Approx 30-60% of the people running it. Those people have never worked a hard day in their lives, or they got one summer job at McDonalds, and use that as their entire campaign for having 'come from nothing'. They get their ivy league education payed for by their rich parents, and their entire lives mapped out for them, right from conception."``

Microsoft invested 13 BILLION into OpenAI development, anf they dont even break even from usage fees, but if should be free for you because youre too lazy to get job? NO. BAD BOY. GO TO YOUR DAMN ROOM AND DONT COME OUT UNTIL YOURE NOT A RETARD. ``

"And no, to cut off your predictable incoming brainwashed rhetoric, I'm not 'jealous'. Well, that's a partial lie, who wouldn't be jealous of making a million dollars in a year just for sitting on their duff as a glorified accountant in their top-story corner window office. But the point isn't the idea of 'oh boo hoo, they have more money than me' because obviously, that would just be stupid. The point is the ever increasing wealth gap, and the lack of a wealth cap, when corporations like Nvidia can make 2 trillion dollars in under 6 months, and have more money/power than many entire countries." Get off your ass and develop a marketable skill, you dont need millions to pay for OpenAI. Nobody gets anywhere crying for handouts, HISTORY again proves that effort gets reward, period end of story. "> Many choose to be homeless rather than being forced to live like cattle, 3-6 to a tiny apartment, but that's what most the lower class is forced to live like. I personally chose to avoid the trap of being a homeless addict or stuck in a min wage job workin like a slave and living like cattle until rents inevitably take another big jump and push me onto the street. So I live in subsidized housing, go to school in the summer, and spend most my day training myself on coding and AI software through the winter. So, within the next five years, I should have a solid career. I've already spent a lifetime doing underpaid-overworked jobs where I was underappreciated for my efforts." ``

Ive seen your posts on Github, you need a teacher, you're not getting anywhere teaching yourself because you lack basic critical thinking skills. If you actually stopped complaining and TRIED youd find several FREE ai inference options, like ollama, or even Groq and Mistral, but no youre a crybaby who wants gpt 4o mini api access for free.

> "These days, we've got a society full of people like you that are anti-humanitarian, corporate cheerleaders. Not good people. The only thing you care about is yourself, your vanity, your pride, and maybe the handful of people close to you. Yet, you somehow think your mindset is better for society than mine. But that's because you've been trained to believe that slaving away for the corporations that are exponentially monopolizing the entire market economy, is 'helping your community' or 'being an asset to your society'."

I buy struggling people on reddit pizza every week, what do you do for humanity? I dont slave for anyone, Im self made through hard work and specializes skillsets, and I own my own busineses, thats plural.

``> Purdue Pharma, Johnson & Johnson, Boeing, Volkswagen, Monsanto, Takata Corporation, Wells Fargo, DuPont, ExxonMobil, Pfizer.

"Look up any of those companies along with the keyword 'scandal'. It's not conspiracy theory, every one of those and many more have been taken to court and proven of serious scandals that scam people, hurt people, make people sick, and even kill people. All in the race to maximize profits. And people like you, think that helping corporations that care so little for human life that they'd run the bulk of us through a meat grinder if it could be proven to be best for overall profit gain, and wasn't illegal. But y'all gonna help them to gain enough capital to eventually overtake governments so that they make the laws; and you think people like ME are the bad guys? Maybe time to step back and look at the bigger picture." ``

Absolutely irrelevant and these companies are not to blame for your laziness and lack of success, you are. Take responsibility for your life.

`` 4 most predictable responses to this.

Dumb parrot response. Throw insults, repeat some common rhetoric about free market economy, or fling more mindless insults about getting a job or w.e. Approx 1-2 sentences, commonly followed by blocking the other person so their obvious lack of wisdom can't be further showcased.

No response at all, IE: ghost. K'bye!

Smart parrot come back with well constructed common rhetoric. Mostly the same as dumb parrot, but without the arbitrary insults, and usually padded with more info, but still not actually researched or validated in a real world context, or done so but while ignoring the entire lower working class and a portion of the lower-middle class.

Fourth and least likely, smart response. These rare responses will occasionally come from smart humans in disguise as corporate cheerleaders. But, given that you've already attempted to discredit me at least twice in a single short response, I highly doubt you're the least bit interested in the facts if they don't back up your current pre-fabricated world views.

"That said, let me reiterate, I admitted that those current prices aren't exactly off the charts, mind you, I have no doubt that those are in addition to the monthly fee for gpt4.0, which was approx 25/mo last I checked. But, the overarching point, as I described was in the logical future projections; especially if I am correct about the monthly fee requirement." Ceck again, API access to OpenAi starts at $60/mo + api fees. You are referring to chatGPT which it NOT what you are looking for, but you say you develop ai? yeah ok bud sure you do. lets talk langchain and gitops when youre ready.

" 25 per month + usage fees seem great to start, but that's also how smartphone plans started, and how they're over double-triple that (CAD), and we've still got those to pay, plus Netflix, plus rent, at a bare minimum. Just for basic non-miserable survival, though granted that assumes you go nowhere and do nothing, ever in your free time." ``

I could live in a cabin in the mountains without any of what you require to live "bare minimum" and be perfectly happy, youre not a real man.

" Go ahead and bash me cuz I don't slave for a corporation. But I've got a 57 year old friend who's built like a 12 year old boy, got a bad back, and a hole in his heart, and he's working 48+ hrs a week, and still struggling to attain a stable place to live(for the past year now). Why should anyone bust their ass off for a system that doesn't even take care of it's elderly. And I live in Canada, we're supposed to be the liberal country. I can only imagine what a hell it's like in parts of America.. I have heard stories though, hard working people that get shit on for all their efforts." you dont have a job, so you dont have to worry about being shit on for your hard work do you. Go ask mom to buy you OpenAI and be quiet, GitHub is not the place for your trash rhetoric

self entitled garbage socialist

'self entitled'? As opposed to people like you, that act entitled to dictate how others should live their lives? What kinda entitlement does that classify as? Authoritarian entitlement?

'socialist' Yes, I do care about other people, even if doing so does not yield personal gain for myself. It's called being a human being. Robots, like you, wouldn't understand.

I went to work full time for my grandfathers painting business, for about 18 years. Im a 5th generation master painter. By the age of 18 i had 3 jobs and 2 cars of my own.

So, you're a rescue pup that got brought to a nice home and taken care of, and 'worked' as a painter for 18 years, and you want to call Me 'entitled'..

I may not have been molested more than once as a child, well, briefly touched by a single person, dunno if that really counts, but anyways, neglected, condescended to, blamed for a life of failure, insulted, sworn at, physically abused, abandoned in group-homes on and off throughout my childhood as early as the age of five, and that's just the shit my mother did, or at least all I can remember off the top of my head. Then there's what other kids put me through.

I already had severe depression by the age of 6-7. Oh, and that's just my childhood. I worked roofing for a decade, a year of which would have been harder than your 18 years of painting. What's more, I worked for a cokehead boss, and made the terrible decision of moving in with the guy as a roommate. He was an all around shit person; manipulative, deceitful, hateful, vindictive.. And I was still naive and timid at the time; fearful and thought nothing of myself. So I just went along with it and did what I was told (the kind of person you would respect, apparently..).

"Microsoft invested 13 BILLION into OpenAI development, anf they dont even break even from usage fees"

Oh boo hoo ho hoooooo... Poor multi-trillion dollar corporation not getting short term gains in their long-term efforts to proprietize, monopolize, and control a powerful intelligence tool with infinite potential. For a person that spent so much of their life sitting on their ass with a paintbrush, it'd typically be logical to assume that you've put more thought into such things, but your knowledge doesn't seem to extend beyond what's reported by the corporate-owned news.

frankly, this is just like when M$ boo hoo'ed about how they lost money on production of the Xb1X. What they don't bother mentioning in the same headline, is how they make 5-10X (or more) the console cost in game sales alone, not including the DLC, expansions, deluxe edition, collector's edition, micro-transactions, remasters, etc. and all those relatively low-cost additions, or pieces of the game, often bring in a good bit more money than the games themselves. Well, cept the remasters, those aren't typically low cost. They polish old games, and sell them for just as much as a new game that was made from scratch.

Really, just like with your cheap assumptions of me, you've gotta read past the front page if you want the full story.

"HISTORY again proves that effort gets reward, period end of story."

History, is a good indicator, but again, not the full story. You can't look back at 20-50+ years ago for your facts, and expect they're gonna remain relevant to the current day.

Let's use the above point for context. M$ investing in OpenAI. This is no longer 20, or even 10 years ago. Large corporations have gained so much wealth since then, especially since they treated the recession like a fire sale, and bought a shit-ton of failing businesses on the cheap.

Because large corps like M$ and Nvidia control such a large chunk of the market, they're able to artificially inflate prices, hence why OpenAI had to become dependent on M$, and is now effectively holding M$'s pocket and waiting for M$ to buy them out.

So, with that said, if a company like OpenAI, that by all rights should have been the next M$, can't make it to the top without being assimilated, then what hope does a single individual have in this economy anymore? Not to say that I plan on rolling over and dying. Hence why I'm not out there working hard like ox to bring home meager hourly pay until I end up homeless from artificial inflation.

Ive seen your posts on Github, you need a teacher, you're not getting anywhere teaching yourself because you lack basic critical thinking skills. If you actually stopped complaining and TRIED youd find several FREE ai inference options, like ollama, or even Groq and Mistral, but no youre a crybaby who wants gpt 4o mini api access for free.

What posts? I don't remember mentioning gpt api within the past month or two.. PS: I spent the past 3 days attempting to design a program with Vram optimizations good enough to get minstrel running on my meager 4060ti 16gb card. Hit a brick wall. I think it may be that it's one of those models that has to be loaded entirely onto the GPU in order to function. Or at least requires more then 14.5gb available Vram to function.

Granted, I'm not exactly a coder yet, I've been taking it on as a side project on top of attempting to learn how to train models (not just load other people's settings, and hit 'train', I mean, learning what individual settings actually mean and the effect they have on training,so that I can adjust them dynamically to my situation). That said, I've been getting a lot of help from the free chatGPT on my phone. And yes, I know you're gonna jab at me for that, due to the irony, but let's be real, I haven't exactly been going on about gpt api. I've just been taking a little time to respond to your jabs on this post, and defend my general argument that corps are gaining too much power.

Think about it, Nvidia, OR M$, maybe even Apple, each on their own could flat-out pay off the entire canadian national debt, and still have over 160billion to spare. And you're gushing over M$ throwin around their chump change with '18b'.. I mean, I'm not an imbecile, obviously they couldn't literally walk into 24 Sussex drive (closest thing Canada has to a 'white house'), whip out their big financial cock, and pay it off on the spot. I'm just sayin, the fact that current corps are valued that high, has some scary implications for the future, when you consider the historical growth rate of corporate wealth, then use that to make projections on the future growth rate.

Money = power under capitalism. Name a single person you know that wouldn't do anything short of outright murder if someone handed them a million dollars. Many would murder for much less. Most would effectively sell themselves for that much, I mean, given that so many are willing to slave their entire life for that much drip fed over 40-60 years.

If you hate socialism, I can only imagine your take on communism, but there's an entire country, with a higher population than America, that have functioned just fine on communism for over 45 years. And what's more, unlike America, China (as 'evil' as people are trained to think it is) isn't running all over the rest of the planet trying to convert every other country like a coked up Jehovah's witness on steroids.

"I buy struggling people on reddit pizza every week, what do you do for humanity?" Oh, you socialist heathen, you!

I took a month long trip to the Philippines last year and gave money and food to (ACTUALLY)starving children (the occasional adult as well but, as being Canadian(VERY rare for children to be on the streets here, I've never seen it), I felt as though it was more important to take care of them, as their parents clearly can't). Not just once, I spent the bulk of my time there in the hotel room on my laptop, to minimize my spending, and only actually ended up going to one paid entertainment venue, the water-park near the ocean-side. But I did that so I'd have more funds to help those that needed it.

I'll give you credit though for giving money to actual people, vs those people that send money to corporate charities to inflate the CEO's yearly bonus. Really.. If UNICEF was actually effective, world hunger would be solved by now.

"UNTIL YOURE NOT A RETARD"

Intellectually superior, that is certainly not the vibe you're putting off with these random baseless insults. More like a child having a temper tantrum. We can either have a logical adult conversation, where we listen to one another's output and give relevant responses, or we could just not. Cuz I've been through too many of these crap flinging contests. I can easily say, at 44, I'm too old for that shit. I can fling crap with the best of them, and in a short bit of friction, it'll do in a pinch. But, when conversations expand to this point, it's nothing but arbitrary filler.

"Absolutely irrelevant and these companies are not to blame for your laziness and lack of success, you are."

So, corporations literally murdering people and getting away with it, in a conversation about corporate greed, is 'irrelevant' because you can't use that point to insult me? Really. Take a step back and reflect. The shit flinging may work well as an ice breaker to get people's walls down so that real conversations can take place, but if your walls are still up at this point, then you got issues that this conversation certainly won't fix/help in any way. And I don't think you're that mad just because a relative handful of people don't work harder (beyond all the work they've already put out in their lives). Forget critical thinking, you need a masterclass in mindfulness.

"I could live in a cabin in the mountains without any of what you require to live "bare minimum" and be perfectly happy"

Whatever you say there, Grizzly Adams.. Funny how people that make statements about how they could be happy living in the middle of nowhere with no other people around, are the ones that think they should dictate the rules for a co-dependent society with billions of people. More than a bit of Irony to that.

"youre not a real man."

Hmm, let me check.. Still got a penis.. Yep, I am a 'real' man, nothing fake about this.. Do I conform to play the emotionally oppressed societal role of being a 'man'? No, that that sense, I don't. I do prefer actual freedoms, over illusionary ones, like people that work themselves into the grave to perpetually rent property that they will never own, and won't be able to pass on to the children that they couldn't afford to have, but still think they enjoy 'FREEDOMES!' bestowed to them by their oh so generous government. Ironically, contrary to the capitalist propaganda, it is actually under capitalism that the majority will own nothing.

Right now, this primarily affects the lower class, but just wait until AI and automation further reduce and devalue available jobs in the middle-class sector.

Here's a good question. If you hate people so much and you want to move to a cabin in the middle of nowhere so you can talk to yourself about what a 'real man' you are, then why don't you? I mean, a 'real man' like you should have no difficulty building the capital to do that.

saying i can live in a cabin without the internet doesnt mean i hate people, do not project your trash values onto me. trying to run mistrel locally is an abso;ute garbage choice i thought you were researching this. I consistently run models on 6gb vram for simple jobs like web scraping and general language tasks. Until MS who you hate to much releases the new framework for 1 bit cpu inferencing you wont be running shit on your cpu so stop trying. You are a liar, nothing more. "look at me boohoo i cant pay pennies for ai inferencing but im programming advanced ai aplications"

what a walking contradiction you are. i wont even entertain you further, youre a tax on humanity, and this garbage you spew in the guise of intellectual conversation is the lamest attempt to sound intellectual ive ever seen.

btw, i actuality do develop AI applications, Im running several local models for free, on varrious consumer grade pcs, even a laptop. im not going to tell you how because feeding the animals at the zoo they lose the ability to fend for themselves.

hey @Xephier102, I was disappointed to see that you must have an API key to use this service, but maybe you can try forking this repo, opening it in CursorAI, and then asking Cursor to help you adjust the project to use only FOSS LLMs or, if your hardware is capable, to use your own LocalLLM via something like Ollama, OpenLLM, or LM Studio.

More here: https://medium.com/thedeephub/50-open-source-options-for-running-llms-locally-db1ec6f5a54f

Also I would disagree that this program is useless! It's a free car whether you have gas or not, and you have the option to modify the base to suit your needs. It's awesome!

In retrospect, you're pretty much right. To be fair, I made the original comment back in September, and I was still pretty fresh on the AI scene back then, and hadn't even considered the concept of re-coding stuff myself. Hell, I think I was just barely using Linux back then. Before that, I was a lifelong windows user (not including my few intermittent uses of Macintosh pcs and Dosshell).

Windows is definitely an OS that does not at all push people to code. It does everything for you (whether you want it to or not..), and there's next to zero customization. Also, either scripts aren't a thing in Windows, or they're something that I weirdly just never considered using in my 25+ years on the OS (in retrospect, I'm a bit puzzled by that point, tbh..).

Oh, as for hardware, I tried running a 12b parameter model on my computer, but after a few days of attempting to build a code for optimizing the Vram with ChatGPT free, I gave up on that for now. My current build is a Ryzen 5 7600, and 32gb of 7600mhz ram, and 4060ti 16gb. The main pinch is the Vram.. Costs so much to get more than 16 on one card. Even next year's cards aren't looking like an upgrade in the affordable range. Just the rumored 32gb 5090, but yea, 3k for one video card, wouldn't be affordable even if I had a job.

That said, I do plan on going back to the drawing board with that homecooked API, and also attempting smaller models. I gotta find one of a decent size for accuracy, and geared for coding, but like, ideally around 12gb; unless I can somehow get it to miraculously work with a larger model, like how I can run the big flux dev model in ComfyUI(which, honestly seems like magic to me, lol.. 22gb and an 8.8gb encoder on a 16gb card..).

i cant pay pennies for ai inferencing

Can you confirm that it's literally just the few pennys cost without the flat rate of 25 per month? Also, I didn't log into the site when going over that, the '25' is prob closer to 40 per month CAD, especially with taxes factored in, maybe closer to 45 depending on exchange rate. But, in any case, I don't really need it right now. I'm more busy trying to get AI running locally, which is a general preference either way.

As for minstrel, I didn't research it, a friend sent it to me, so I used that as an opportunity to attempt to build an API to get it running. It's not an 'advanced ai applications', lol. It's two scripts, one for loading the model, and enabling the user to choose a model from a list of available models in the 'model' folder, as well as some basic error handling, and the other for plugging the model into and handling optimizations. All done via the konsole window. So, nothing advanced about it, although in the long-term, I do plan to build a GUI interface over it.

Not sure how you're connecting me building a simple API, to having anything to do with paying for corporate AI use(at least, within the presented context).

"do not project your trash values onto me."

you're the one that said you'd be perfectly happy living like that, with no people around. Generally people that like other people, don't want to live around, not people.. I mean, it's not exactly rocket science logic there..

"I consistently run models on 6gb vram"

Good for you? The project/experiment I was attempting was to pull off what ComfyUI does with the flux dev model, and run it on the system ram, and Vram. How the hell are you working with a 6gb card, and you don't know about Vram offloading? You must be running like, a 1b parameter model, at best, lol...

"btw, i actuality do develop AI applications, Im running several local models for free, on varrious consumer grade pcs, even a laptop"

LOL... And you don't know about the process of model offloading, lowram, medvram, etc?

"i wont even entertain you further"

Aw, that's too bad, the angry little gremlin you are was actually causing me to break out in fits of laughter. I tried to be humane to you, but you're clearly not the humane type, besides tossing a few coins a week to beggars on reddit. Long as you never have to see the people that are actually suffering, right? And no, I'm not talking about me. I'll do just fine without your guidance on coding. I'd probably be better off with free chatgpt guiding me anyways, lol..

I've been dealing with capitalist apologist, know-nothing parrots/trolls like you for so long at this point, that literally nothing you say could phase me. I could have a civil conversation with you, or stoop down to your level and throw monkey feces back and forth. Makes no difference to me. 'tax on humanity' coming from the soft little privileged girl that was groomed by her grampy into painting for 18 years, and you talk like you're hard. That's cute. No one gets hard with a life that pampered, not even with bad childhood experiences.

That's why you don't want to continue the conversation.. XD.. Cuz you're too jaded to be civil, and at least smart enough to know that you're too dumb to come out on top in a battle of wits.

Ah well, if you're done with it, toodles! LOL!!

I challenge you to name 1 single succseful country built on socialism, you can't, so yes your values are trash, not only in concept but in application as well.

You say you're doing fine, but you can't afford more than free chatgpt, and you call using that to write code for you "learning to code". Big yikes. Just look at your github profile, it speaks volumes about your immaturity. You hate succs3ful people because you are a massive failure, that's what it all boils down to.

You've spent how long now trying to make whatever garbage you're trying to make when the solutions already exist right here on github and a simple YouTube search about ollama and huggingface would tell you all you need to know.

Hey I've world hard my entire life to get where I am so I don't have to be a bitter pathetic beggar virtue signaling self imposed miserable victim like you, I guess that means I owe you now huh, because you can't survive without my hard earned societal contributions lmfao.

By the way, if you were actually smart and not using ai to form your sentanc3s and arguments for you, you'd have a job and you wouldn't need to beg for handouts.

And here's your proof that you're crying about literal fractions if pennies https://openai.com/index/openai-o1-mini-advancing-cost-efficient-reasoning/

Wait my fault, that's almost 1 penny per batch, it's the PhD model.

Try this one it's more in line with your pathetic lack of income https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/

szczyglis-dev / py-gpt

The term "free" seems pretty ambiguous in it's context these days.. #71