What is the ideal success rate for Horizon 3

[sluongng]

In The Change Log podcast ep 395, you mentioned that you have been focusing on Horizon 3

But that changes dramatically during Horizon 3. We should be exploring things that are destined to fail at a very high percentage, so we should be right maybe 20% to 25% of the time, and only that many projects turn into a future-oriented product.

My question is: why 20-25%?

If the success rate is too high, aren't your 'tries' too conservative? If the success rate is too low, aren't you taking too many risk? Why 1/5 - 1/4 success rate strike a balance for Github?

Also: What do you think about Google's approach of giving employees "20 percent time" to work on what ever they want? Is that a good way to implement Horizon 3?

[jasoncwarner]

There's no particular reason I chose 20-25% to start the conversation except to set expectations. It's a good filter for the organization (in this case, GitHub), recruiting, and mindset.

In the absence of horizon 1 & 2, it is hard to describe, so let me go try to expand a bit more.

Horizon 1: current roadmap. Should be somewhere from a year to 18 months out in most start-up companies of any notable size. Your success rate here should be near 100%. You should have done the sufficient customer development, validation, feedback, pricing conversations etc etc. The brand new things have a plan to go alpha/beta/GA and build the fans before launch so it's not a flop etc.

Horizon 2: Next phase of roadmap. A bit further out, 18 months - 3 years. There are few experiments in there though mostly iterations on current ideas, products or roadmap. Think of this as current business line expansion. Success rate should be about 80%. A few speculative failures farther out are ok....meaning, that if you reviewed your original roadmap 3 years from the point in time, it's mostly right though a few things have changed.

If your current roadmap (horizon 1) is hugely different 18 months from your original one, you likely aren't doing either product development or customer development well enough or don't really understand your business. Common when a company is a high growth startup with inexperienced senior leadership though also common in established businesses with below average senior leadership. Know which one you are.

If your next phase roadmap (horizon 2) has some misses, that is ok. Best if they are farther out. If the misses are like 50%, that is likely a problem. If they are 20%, probably within bounds.

No hard and fast rules on any of this BTW, more based on experience and desire to have some established parameters.

Now, horizon 3: 3+ years out. Basically all speculative. Think "imagine if we could..." or "what if we were able to pull off..." type things. If your success rate is too high, you aren't being ambitious enough. If it's too low, you probably are thinking too ambitious (sharks with laser beams on their foreheads and colonizing mars and Iron-Man suits) for the timeframe (3-5 years).

Now, remember, I'm defining this for GitHub, not the industry specifically. I want a decently high success rate (20%) but I also want to set the tone that we are literally building the future and to expect misses. For us, this strikes a good balance.

[jasoncwarner]

Also: What do you think about Google's approach of giving employees "20 percent time" to work on what ever they want? Is that a good way to implement Horizon 3?

I've never personally worked in this mode so can't comment specifically.

Generally speaking, I feel we as tech build up too much myth and lore around stories we hear like "Google's 20% time". I imagine it worked for some period of time for some things and generally didn't work.

I honestly don't care to copy or whole sale lift and shift a program like this. I rather want to understand why it did or didn't work.

Given my (many) conversations with people about this specific program, I would say the following with the caveat that I didn't personally experience it.

I would not want to bank my future innovation of a company on any one approach.

I think all engineers should be free to experiment in a company. Ideas should come from anywhere.

I also think focused experimentation and incubation is absolutely needed.

Innovation in a company as it grows is literally the hardest thing in the world. If companies aren't actively building the future and looking for them, they will become incredibly aged and vulnerable very quickly.

Tale as old as time. True as it can be.

So, I think % time is a good approach to give individual engineers agency. I think incubation is a good approach to direct and focus.

Think of these programs as tools. Think of a company as a garden or farm. You as the CEO are the gardener or farmer. You don't use just one tool (a shovel) to do everything. That's not the goal. The goal is to have an amazing farm or garden so we need multiple things going on at once.

[sluongng]

Great answers!

Think of these programs as tools. Think of a company as a garden or farm. You as the CEO are the gardener or farmer. You don't use just one tool (a shovel) to do everything. That's not the goal. The goal is to have an amazing farm or garden so we need multiple things going on at once.

I really like this view point. 👍

Thanks!