Q3 2024: PostHog for LLMs

Lior539 commented 4 months ago

Goal

Two goals:

Get 50% of YC AI startups that using PostHog to use one of our LLM integrations
150 companies in total using any of LLM integraitons

Plan

[x] Launch Traceloop integration + related marketing
[ ] Keywords AI integration
[x] Reach out to AI Grant and offer sponsorships/credit/something
[ ] ~Bookface post when new batch starts to announce our all our LLM integrations (need to wait for Traceloop integration first too)~ Not done. Will maybe try for F24 batch
[x] James and Tim to plug in the LLM integrations when they go new YC batch event
[x] Onboarding email to LLM companies
[x] ~Add LLM integrations component to YC startup program page~ (won't do)
[x] ~Add LLM integrations component to Regular startup program page~ (won't do)
[ ] ~Run a Startup Spotlight featuring one the AI companies in the new batch~
[ ] Comarketing guide/tutorial/blog with Langfuse and LlamaIndex
[x] Add the ability to track how many companies are using the integrations in PostHog
[ ] Comarketing blogs with AWS Bedrock
[x] In-app onboarding: Add adding the integrations step there
[ ] ~Stretch: Convert langfuses chatbot blog for Hackernews post / beginner's guide to creating LLM app~
[x] Custom merch for LLM startups
[ ] ~Launch YC post~ Not done. Will maybe try during F24 batch

Related issues

Lior539 commented 4 months ago

tagging @charlescook-ph @andyvan-ph @joethreepwood @ivanagas @jtemperton for comment. Anything else I should add?

@joethreepwood especially interested in your feedback, since you've YC related stuff before + the work above touches some of your usual areas (onboarding, startup spotlight)

joethreepwood commented 4 months ago

YC runs on merch. One thing we could do is enhance our YC offer somehow to, say, give users an exclusive t-shirt, hoodie, badge or something that's only available to YC + AI teams.

Maybe we get super cheeky: @lottiecoxon designs two shirts. One says: "I'm in PostHog's Y Combinator program and I'm NOT making an AI product" and another one that says the opposite (or similar). Then, when people sign up, they can select one or the other.

Beyond that, the biggest impact is on-site stuff. If we can get YC to feature us in their materials, or give us a speaking slot, or let us do an on-site giveaway, that'll move the needle.

Spotlight-wise, I don;'t really do these anymore. If you want to write one, go for it though. It can definitely go in the emails.

raquelmsmith commented 4 months ago

We should add this info to in-app onboarding for product analytics, too, right? Can you come up with some suggestions for what that might look like? I'm not familiar enough with these yet to know what their onboarding should look like.

Lior539 commented 4 months ago

@joethreepwood

YC runs on merch. One thing we could do is enhance our YC offer somehow to, say, give users an exclusive t-shirt, hoodie, badge or something that's only available to YC + AI teams.

Thanks for this. I think its a good idea, but a bit high effort. Given that we most companies in the new batches use us already (because of all your great work!), I think we only need to let them know the integrations exist. It did give me an idea though, and I've added Onboarding emails: Add email about LLM integrations to onboarding flow if company in YC startup program to the list now. I think this will be significantly lower effort and should achieve similar results (combined with the other stuff on the list)

Lior539 commented 4 months ago

We should add this info to in-app onboarding for product analytics, too, right? Can you come up with some suggestions for what that might look like? I'm not familiar enough with these yet to know what their onboarding should look like.

@raquelmsmith I'm hesitant to add it to our in-app onboarding. The integrations require you sign up to our 3rd-party partners, so Im reluctant to send people to other apps/sign-up flows when they're in the middle of ours

joethreepwood commented 4 months ago

Defer to you on the merch point, but it'd actually be simple to do. We just need to get the merch made (that's the hard bit) and then we can have it as an optional add-in for YC and startup kits.

Onboarding emails: Add email about LLM integrations to onboarding flow if company in YC startup program to the list now. I think this will be significantly lower effort and should achieve similar results (combined with the other stuff on the list)

We can probably chat directly about how that'd work, but I think that may actually be trickier.

The reason for this is that we don't have any event or trigger to identify YC teams within PostHog. I have to do this manually and it's a non-trivial lift. We'd also, as an output, then just end up messaging every YC team about our LLM integrations regardless of whether they're an LLM team or not.

The better way to do this would be to add LLMs as an in-app onboarding step as @raquelmsmith mentioned. Then we can use that as a trigger to create a customized onboarding flow for every team building LLMs. I get your concern about the timing, but we can delay emails to compensate for that.

One thing to add may be reaching out to Helicone et al and getting them to put us in their onboarding.

andyvan-ph commented 4 months ago

This looks like a solid plan to me.

@KendalHall, do you know if James and/or Tim are attending any events at YC for the upcoming batch?

joethreepwood commented 4 months ago

@KendalHall, do you know if James and/or Tim are attending any events at YC for the upcoming batch?

They have something pencilled in for their trip in a few weeks. Kendall and I are already discussing.

ivanagas commented 4 months ago

I think https://aigrant.com/ is like the YC of AI, could we do something special for them like we do YC, or at the very least, get on their credit list? Likely has overlap with YC, but would be awesome to win all these companies.
Newsletter getting a bunch of AI founders we know to talk about analytics, become the go-to post for future founders to read

raquelmsmith commented 4 months ago

I'm hesitant to add it to our in-app onboarding. The integrations require you sign up to our 3rd-party partners, so Im reluctant to send people to other apps/sign-up flows when they're in the middle of ours

@Lior539 I think it would be the opposite - they're using an LLM provider and want to use PostHog to collect events. We used to do this with, eg Segment. If you use Segment, here's how to send stuff to PostHog, here's the API key you need, etc.

This went away at some point, prob because we don't want to encourage people to use Segment 😛 but we could do it for the LLM providers.

Lior539 commented 4 months ago

@Lior539 I think it would be the opposite - they're using an LLM provider and want to use PostHog to collect events. We used to do this with, eg Segment. If you use Segment, here's how to send stuff to PostHog, here's the API key you need, etc.

@raquelmsmith super interesting! You've convinced. I'll add this to the list and look into it

Lior539 commented 4 months ago

I think https://aigrant.com/ is like the YC of AI, could we do something special for them like we do YC, or at the very least, get on their credit list? Likely has overlap with YC, but would be awesome to win all these companies.

@ivanagas This is great! Thank you for this. Didnt know about AI grant

Lior539 commented 4 months ago

@KendalHall, do you know if James and/or Tim are attending any events at YC for the upcoming batch?

They have something pencilled in for their trip in a few weeks. Kendall and I are already discussing.

@joethreepwood can you let me know if/when you start planning what they'll do there? I'd want to make sure they mention the LLM stuff

Lior539 commented 1 month ago

Results here: https://posthog.slack.com/archives/C01FHN8DNN6/p1727081457239219 Sharing in slack as I dont want to show usage numbers privately (since each of our integration partners are competitors of each other)

PostHog / meta