solderneer / obsidian-ai-tools

Adding powerful semantic search, generative answers, and other AI tools to Obsidian, using Supabase + OpenAI.
https://obsidian.md
MIT License
234 stars 20 forks source link

[AI] Error #2

Open leonardsellem opened 1 year ago

leonardsellem commented 1 year ago

I set up the plugin strictly following the instructions (using supabase CLI and downloading the last release from github). The Table Editor shows the two tables document (7 records) and document_section (30 records). When I run the AI Tools: Refresh Index, after a few seconds showing the 🔼 [AI] Indexing ... indicator, it finally switches to 😔 [AI] Error. I have no idea what went wrong.

solderneer commented 1 year ago

Oh, that's strange. Also my bad, I should incorporate more verbose error feedback into the plugin. The errors are logged to the console, could you help me open Developer Tools, and run the index again? Let me know what the error says. You should see something like this.

Screen Shot 2023-08-04 at 1 26 10 PM
leonardsellem commented 1 year ago

Hi, Here is the log CleanShot 2023-08-04 at 08 48 43 I really love your project; that's exactly what I've been searching for months!

solderneer commented 1 year ago

Thank you!

It seems like it's a problem when the code calls the OpenAI API. Did you configure your OpenAI API Key in the plugin settings? Are the API keys still valid in OpenAI? And are you still under your rate limits?

You can check it here https://platform.openai.com/account/usage

leonardsellem commented 1 year ago

That's strange, I am far below my rate limits, and I am using my API key with no issue on many other services.

solderneer commented 1 year ago

So at the moment, the error handling is quite aggressive in the fact that it stops processing if any one document has an error. It could be that the error is because the document is being parsed incorrectly, has some invalid characters, goes over the embedding API token limit or something like that. You said some of the other documents parsed fine and were populated into the supabase, am I right?

If so, what happens if you add the Tweets directory to the excluded directories path (it's in the settings panel)? Does the error go away or change? It you don't mind, you can also email the problematic markdown file that is throwing the error in the log to me @ sudhar393@gmail.com and I can see if I can replicate it on my end.

leonardsellem commented 1 year ago

You were right! One single doc was crashing the indexing. I sent it to you by mail. Thank you so much!

solderneer commented 1 year ago

Wonderful glad it works, I'll look into the problematic file. In the meantime I've patched the code to not be so strict on errors.

Now, it continues processing the other files even if a file has an error in processing. And then it reports the problem via a UI notice for more feedback.

I'll try to find ways to provide more info to users during the indexing process too.

The patched release is here: https://github.com/solderneer/obsidian-ai-tools/releases/tag/1.1.4

solderneer commented 1 year ago

Ah, I've looked into the problematic file. It turns out, it's because the context length limit for the embedding API endpoint is 8000 ish tokens, and that file exceeds it. Normally, the documents are separated into paragraphs, which prevents them from exceeding the context limit but that file has no paragraphs at all.

The issue can be resolved by adding a new line in the file that separates the context into two chunks, which are sent to the API as 2 chunks. I'm reopening the issue so that I remember to add the code to automatically handle that, in the next patch. In the meantime, if you just add a newline in the middle of that file, everything should work without any errors.

beingskyler commented 1 year ago

Looking into what's causing the errors to throw across my vault now. Do you log anything in the console for documents that were stored properly? It might be easier for me to track things down if I can find some working examples.

solderneer commented 1 year ago

The errors are logged to the console. Valid documents used to be logged to the documents until 1.1.3, but not anymore to avoid polluting the console.

As an alternative, you can also just open up the supabase instance in Table View and see all the documents that were stored. The error in this issue is caused by files that are too long without paragraphing, so just need to break up the file into two or more.

beingskyler commented 1 year ago

I found the root cause of my issue which was—without surprise—not following the setup instructions closely enough. Everything's working great now. Great work.

solderneer commented 1 year ago

It's always the setup instruction's fault for not being simple/clear enough 😄 . Which step tripped you up, maybe there's a way to rewrite the instructions to be easier to follow?

beingskyler commented 1 year ago

Haha, the instructions were clear. The part that tripped me up was not reading them before I began trying to configure everything. Specifically, I didn't run the command to create the necessary tables in Supabase before I started the command to create the embeddings. 😅

leonardsellem commented 1 year ago

Just got a new similar error.

CleanShot 2023-08-12 at 22 06 35@2x

Below are the details of the request and response:

{"model":"text-embedding-ada-002","input":"**Australia had a couple of VC funds that emerged in 1999-2000 and invested in dStore and all these crazy businesses at absurd prices. All those funds completely blew up. They raised $100 million and returned $5 million—those kinds of numbers.****So, there was no Australian VC market at all. They were all on their last legs of their respective funds, all the partners had left, and someone was just filing the admin paperwork.****We were two twenty year-olds in Australia with no experience. We didn't even bother looking. We didn't pitch anyone for capital. It wasn't even a thing that you did. We knew what VC was, but there was nothing available in Australia. It wasn't even a choice to not get VC. It was just like, “That's not going to happen.”****And so, we had to figure out another way. We used to go around and if we needed to sound smart in Silicon Valley circles we’d say, “Oh, we didn't really go down the VC funding route. We were customer-funded.” People would reply, “Okay, oh, wow. That's an interesting choice.” And I'm like, “Okay, that worked.” “Revenue-funded” was another one that Scott pulled out at one point.**_**Or “We haven’t needed to raise capital.”**_**Exactly.****We ran the business cheaply for a long enough period of time that revenue was always higher than expenses on a cash basis. This seems crazy now given the VC boom, but it's the way a lot of people build their businesses, just not in tech.****We got lucky on a few things and I think that Tobi (CEO of Shopify) would say the same thing. He had no access to VC, so he didn't even bother looking, which meant that he didn't waste any time.****Secondly, if you have technical founders, as long as your own salaries are low enough, you can build something, can do well enough at marketing and other stuff, you can get a wheel going.****Thirdly, we all charged from the start—we had revenue coming in. This wasn't the “Hey, let's go get lots of eyeballs and work out how to charge later” advertising model of TikTok, Snapchat, et cetera. We actually had a revenue-producing product early on.****Lastly, we had patience. The thing about the nuclear winter—that 2002 to 2005-2006 period—if you actually look at it, amazing companies were founded in that period that had relatively little competition. What a startup has to do in year one in the crazy world we have today, we at Atlassian had three, four, five years to do. We could afford to be a little slower. Our exponential revenue curve didn’t look exponential at the start, it looked like a relatively flat line. Yea But, we were able to survive. We made $200,000 in year one, then like $800,000 in year two, then $3,000,000 in year three, then $10,000,000, $22,000,000, something like that. It became exponential over time.****At the start you can't see that; it looks like it's flat. We had the time to do that because of a lack of competitive spirit amidst a really bleak backdrop.****On Mike's work week**_**I want to switch gears just a bit. This next section is roughly “Teach me to do your job.” It's one of my favorite questions to dive into with interesting people. Let's start with a very simple one. Tell me how you spend your time. What does a calendar in the normal work week look like for you?**_ **I always try to be very intentional with my time. Time is the one thing you can't get back. By intentional, I mean that I choose as carefully as I can where to spend my time. Everything flows from that as it creates constraints.****For example, the first thing I put in is “kid time.” I have certain times of the week where I do school drop-offs, I do pickups, I do things like that. That's the first time that goes into the calendar and you cannot book a meeting over it. My EA has been trained that if someone tries, they get told “Eff off.” Unless it's crazy important—and I mean it has to be crazy important—that time is blocked out. That’s worked out really well as people learn to adapt around it. Being intentional is one thing.****Second, I have a lot of interests now. As such, I intentionally have 10% of my week dedicated to work outside of Atlassian on a calendar-time basis. Obviously sometimes I have a board meeting that takes six hours. But if a week's roughly fifty hours of work, there's five hours that can be done outside of work. My investment team and philanthropic team outside of work receive a certain portion of that and I have to decide how I want to use it over a month or a quarter. That 90/10 percentage split seems like it's worked well. Within the 90%—at Atlassian—that is made up of one on ones, as much on product and strategy as possible, and lastly, storytelling. I define storytelling as the broadest version of scaled communication.****One of the things that's really hard in a larger company is sharing and repeating stories. This is less about the company’s founding and such, but rather “What is our strategy? What are we trying to achieve? What are we doing?” Though people can look up our OKRs, it's not quite the same. If someone new starts tomorrow, they're going to get directed to the company OKRs throughout the course of onboarding. They'll read them and be like, “Okay.” It's different than if they hear repeatedly in different forums that we’re guiding the ship towards a few sets of goals. I think storytelling is incredibly important and that takes a lot of forms. Because of this, I try to divide my time evenly between product strategy and scaled storytelling—town halls, presentations, et cetera. Ironically, working from home has been amazing. Yesterday I recorded two videos—fifteen to twenty minutes each—that were going to different groups in different formats.****On Running a Distributed Company**_**I was going to ask this later but I think this is a good segue. One of the things that’s remarkable about Atlassian is that you as a company have always been so distributed. A lot of companies have had to adapt to being distributed during the pandemic, yet you have always done this. I'm curious, what has made it work so well for you?**_**Yes, our curse has become our blessing. We used to say that we were cursed by the Pacific Ocean.**_**That's a great line.**_**Ironically, to your prior question about Silicon Valley, one of the things I think we did well early was that we started a Silicon Valley office around six years into the business. It's grown up with Jeffrey, then with Jay [Simons], and other amazing Silicon Valley leaders.****Having a foot in both camps is the important part about that. We have one foot in the pragmatic business community of Australia—insurance companies, banks, customers that come and look at me and say, “Mike, I'm not Facebook, I just need to do it. I'm not doing it. I'm just a regular IT group. But I want to be good.” And we have one foot in Silicon Valley, which is froth bubbles, people saying “We're going to change the world!” And you're like, “How are you going to do that?” And they respond “I don't know yet!” It's actually good to have the mix of that and the Silicon Valley skills. We've had plenty of people now from the Facebooks and Googles and Amazons, but we also welcome the pragmatic folks with open arms.If you can get the mix right, the balance between the two in the company is really powerful.****The Pacific Ocean is in the middle of these two camps, as it always has been. I used to fly to the US ten times a year for the last fifteen years. Almost every month, across and back. So we are a distributed company and have been for a long time. That certainly helped us through TEAM Anywhere, the pandemic, working from home, and more.****The second thing that helped us is when we bought a company called Trello four years ago. Trello was a remote company by force. They were in Manhattan and they found it incredibly hard to hire—as NYC is incredibly expensive—and they kept losing staff, especially when employees began to have kids, new parents would move out of Manhattan as they couldn't afford it.****Because of this, Trello was forced to become a remote company and deal with all sorts of these problems. They now have staff in Hawaii, Brazil, and Spain and roughly 60% of Trello’s staff are outside Manhattan, not in the office. We absorbed and learned from that Trello culture when they became a part of Atlassian.****Before the pandemic, we had probably ten to twenty percent remote, highly-distributed workers. Though we were used to meetings on Zoom as a general rule, it was still a leap. We sent 5,500 people home at twenty-four hours notice in March last year and that caused a lot of challenges as there were a lot of non-remote processes that we didn't have.****For instance, we had to corral some Uber drivers in India to deliver laptops. We had staff starting and at the time you couldn't start remotely—you needed to be standing next to an IT person having him/her set up your UB keys, password, et cetera for you. That process alone took almost a month to figure out because we are hiring almost one hundred people a week!**_**With the pandemic everyone was forced into this in some ways. That said, before the pandemic one part of your remote culture that has always been very interesting to me is the fact that your senior leadership is actually on the “other side of the Pacific” too.**__**For me, coming from the Facebooks or the Googles of the world, that's very unusual. It's almost impossible to be in a senior leadership role where you have to work closely with the founder and CEO if you're not living within a one mile radius of that person. You've managed to make it work. How?**_**Before the pandemic, our executive team was split across three offices around the Pacific. We were located in Sydney, San Francisco, and Mountain View. Ironically, because it is incredibly painful to drive between San Francisco and Mountain View, our team wouldn't bother to do so even for executive meetings. Not to mention, our Sydney offices were already on Zoom. Of a team of nine, we had two or three people in Mountain View, two or three people in San Francisco, and then three or four people in Sydney on executive calls before the pandemic.**_**What do you think you got right and how did you make it work? At other companies, you learn very quickly that if you're not close to the founder and CEO physically, you're not going to get the promotion or you won’t last very long as an SVP. What do you think worked for you culturally there?**_**In all honesty, I'm not sure. We have done it for such a long time and we have just learned that it has to be made to work. People are now used to it.****Though we did do a lot of travel, we used our time together quite intentionally. We have been through lots of variations in the management team as we have structured our systems over time. At one point, we would get together every eight weeks. Every four months, the entire team would fly to Sydney. And eight weeks later, the Sydney team would fly to the States. The entire team was together for at least thirty-six hours every eight weeks; we did that for a fair while.****Our company is built to be distributed. We do not have a very strong centralized headquarters culture like Google or Facebook do. We have never had that and have managed to thrive without it. That has both strengths and weaknesses. It's a bit more redundant as decisions are made in the far-flung territories rather than in Rome.****On What a Great Meeting Looks Like**_**That’s a great line. Okay, I want to get back to how you’re spending your time.**__**What does a good product review, business review, or business update meeting with you look like? What does a bad one look like?**_**I’ll take two angles: the product and the team.****On the product—whatever it is that we're building—I always think about whether we've combined technology and creativity in a clever way. I think about whether the product is exciting; whether it has a distribution advantage or at least some way that we can see it work, where we’re playing the chess moves ahead.****You can’t help but get excited when you come in and go, “Oh, man, that thing's just going to crush it.” When the team has taken some new piece of technology and used some creativity to twist it or turn it in a unique way—that excites me. You can just see it working. If you do it long enough, you walk out of the meeting and you’re like, “That thing. That's a lock. That's a win. That one there—done. We don't even need to do an A/B test.” Ironically, we have a bunch of examples where we have literally gone, “I know, we're big on A/B testing. Just ship it. That thing's great.” From a product side, when you just walk out and you know—that’s a great meeting. It's very hard to explain why that is. I would say the closest thing is when you see the potent creativity and technology mix, when art and science have merged in the right way.****On a team basis, it’s a great meeting if the team seems to have its shit together. I know that sounds like a weird or a simple answer, however, you can pick up on things if you've watched or managed teams for long enough. You can say, “That's a strong team,” or “That's a weak team.” Sometimes even the product has a long way to go, but you can walk out confidently and say, “Man, that team knows its space. It knows what it's trying to do. It seems to have the right level of leadership.”****If it's just the product manager answering questions, that's a bit of an alarm. At Atlassian, we operate on a triad model—we have many different triads between the product manager, designer, and engineering lead. If the designer and engineer are answering each other's questions, that's usually a good sign that the team knows what they're trying to do. It’s crucial to home in on whether a team has the energy, consistency, and excitement to execute.**_**You can feel the energy in the room when the team is in sync, getting along, and excited. Similarly, you can also feel if they're not in alignment, if they’re trying to put on a brave face for the Emperor. You can just feel it in the room if you've been there enough times.**_**Exactly!****Two other things: Confidence is vital. I like to see if senior leaders disagree with founders. As I get older, I increasingly ask open questions and note if opinions change or not once I weigh in. If a leader disagrees and pushes back on something with evidence to back it up, that's a good sign.****It is vital to instill the team with the confidence to disagree with senior leaders. Not disagreeing offensively, but rather saying something like, “Hey, actually no, we did the customer research and I think left is a better direction to go on that particular issue.” or “No, the competitors are all charging that. Here's why I think that we should hit the bottom of the pricing curve.”****Disagreement with seniority in a constructive way usually shows that the team has their shit together. It means that they know more than you do and that's good. They inevitably should know more than I do.****One of Scott's favorite techniques—which I've started to adopt over the years— is to scratch one area quite deeply. We often use it in interviews. For instance, I have asked people to tell me about what happens when you click a button in a web browser in technical interviews. I try to go as deeply as I freaking can. I’ll continue: “How does that turn into HTTP?...Okay, cool. And that runs into TCP? How does that work?” If I get to the point that I've run out of questions, and the candidate can still go deeper, that means he/she is likely a really good engineer; he/she has thought about the seventeen layers of stuff that go together to make a web page work.****I like when engineers can pick any area—even those outside of their core speciality/expertise—and go deeply. They should always be aware of first principles. Similarly, you can ask me about Atlassian strategy and I can go six levels deep on anything straight off the top of my head. That's my job.**_**There's a classic Joel Spolsky blog post from back in the day where he talks about Bill Gates’ reviews and how Bill once grilled Joel on Excel formulas. Joe's got this tiny esoteric detail correct. And Bill would go, “Okay, I have confidence that you know what you're talking about. I trust that you figured out everything else.”**_**It’s important to do it in an area where you're not just being pedantic about details.****Say a product has five pillars. I will often say, “Tell me about that pillar. Why did you come to that? Was there customer research? What have the customers said?” Pick an area that's major enough that an engineer should know the details. That's usually a good sign that the team is going to be successful.****On His Decision Tree Mental Model**_**I’ve researched a few mental frameworks that you reference often. I’m going to throw a few phrases at you and I want to hear you talk about what each means to you. The first one is your tendency to probe for the “decision tree” when it comes to any decision. What does that mean?**_**The decision tree originally came from the book Fierce Conversations, but I don't know if author Susan Scott stole it from somewhere else. It's a learning device, an onboarding device, a scaling device, and a communication device.****I love communication devices. I often think in business it would help if we depersonalize communication. If I’m in a review with you, and you think I'm saying that you’re shit, you're instantly defensive. If instead I say that this product is shit or that we should talk about why the product is shit—even if you built it—it’s depersonalized. The decision tree really helps to depersonalize challenges in the course of one on one communication.****The basics of the decision tree are root, trunk, branch, and leaf. So you have four types of decisions and they vary on two axes in a generalized decision model. The two axes refer to the person making the decision and the information that is broadcast about the decision.****To break it down, a leaf decision is a decision where you're working for me, you make the decision, you don't tell me, and I don't need to know. People make thousands of leaf decisions every day.****A branch decision is one that you make and then you tell me, “FYI I'm going to let that person go, I'm going to do this, or I'm going to do that.” Okay, great. I don't need to weigh into that decision.****A root decision is mine to make. As such, I will tell you what the answer is.****The trunk sits in the middle of the prior three and is my decision to make. You need not tell me the outcome, however, you hopefully come to me with options like, “Hey, boss, we're going to decide what to do here. I think A and B are the big choices. Maybe I’m leaning towards B, but it’s your call. You have to make this decision.”****What goes wrong in business is that people get all of these decisions mixed up. Over time, you might think it’s your job to be building culture and making all the decisions. No! As you grow with a job, more of the decisions should move from root-trunk to trunk-branch or branch-leaf.****Jay [Simons] is a good example. After thirteen years here at Atlassian. Jay would bring me almost nothing. He worked for me for the entire thirteen years and our trust level was so high—along with his knowledge of the business—that he knew exactly what I was going to answer, so we just skipped that unnecessary step and things moved even more quickly.****Additionally, the decision tree is a really useful depersonalization device if things go wrong. It’s a very, very useful construct.****On His Obama-Inspired Prioritization System**_**Alright, here’s the next one. The name is a little unclear, but perhaps it's the Obama Priority System? In short, it allows you to assign tasks priority from P1 to P4. Though I did a lot of Googling, I could not find any evidence of President Obama actually using this tactic.**_**I inquired a long time ago about how Obama prioritized. I found that it was basically a simplified form of Kanban in a world where he did not control what was actually happening.****The way I remember reading about Obama's prioritization system was that every big issue going on for him as President was assigned Priority 1, 2, 3, or 4. He had a certain number of P1s and P2s that came directly to him, as he was going to make the call on those particular issues. With P3s, maybe he knew about them and advised from afar, but he did not delve into the weeds. P4s were off his radar completely. The latter two—P3s and P4s—were handled by his Chief of Staff, the Secretary of the Treasury, or whomever. He could only have two P1s and four P2s simultaneously to prevent decision fatigue and overwhelm.****Because he was President, there was a natural limitation to this Kanban strategy. Every week—sometimes every day—he would have to rejuggle this schedule because he lacked control over his inputs. For example, if Syria starts a war with Yemen, that becomes an immediate P1. He didn't get to choose that. It immediately went to the top of his pile.****That said, what Obama did so brilliantly was develop a system to rearrange his priorities. Without this, he would not be able to think clearly amidst the chaos of his office. He would move one of the two P1s to P2; one of the P2s to P3; and so on. Again, the basic idea was that he, as President, couldn't control his inputs. Though he had many issues that he wanted to tackle—Obamacare, for instance—he allowed himself two big issues and four small issues to work on simultaneously. Everything else was passed to other people down the chain.****I try to use this in my own thinking. Though, to be honest, the last year has been really bad for that, but that’s another story. I have P1s, P2s, P3s, and P4s that I move around. There's a whole chain underneath that reassigns tasks automatically. When I say that P2 is now a P3, my Chief of Staff and EA know exactly what that means and take action accordingly.****It's less Syria and Yemen, less big, hairy, global issues, so it happens less frequently than it did on Obama’s Kanan. That said, it's a useful mental model as I can only really focus on two big things and four small things.****This doesn’t just apply to work, oftentimes the four small things are personal things that are going on. I think it’s important and realistic that these things be on the list because they take up both time and mental space.****On Work-Life Balance and Family**_**I think that's a good model and a good segue to shifting into you and your personal life. One thing that has come up in many conversationnƓs how much you care about work, balance, and family. Could you talk to us a little bit about how you balance work, life, and family? And second—this is very connected—I heard a story about the IPO roadshow where you couldn't make it back and you wanted to prioritize family. Could you talk to us about that because I think they all go together.**_**I don’t want to be on my deathbed reflecting on the great software that I built. Hopefully I've achieved things and changed customers’ lives, but if that's all I have to talk about, that would be very sad. It’s all about balance. I always think that if a CEO doesn't model good work-life balance, how can he/she expect his/her staff to do so?****I finish work most days at 5:30PM pretty strictly and I have dinner at home. Assuming I’m in the same city, I try to have dinner with the kids. I sit down for dinner at 6PM and have done so since my kids were very young. Maybe I'll go back to work at nine o'clock and knock out some stuff. I find ways to fit work in, but my family is my priority.****On Founder Mental Health and “The Roadshow Story”****I think mental health is both really important and really challenging. In my mind, the story of entrepreneurs and mental health is far too seldom told. There are a number of really famous entrepreneurs that I know personally who have come to me and said, “I'm not coping.” Though we see them with a sword and a shield as these big leaders out there pioneering and creating the future, it's fucking tough.****All too often, we hear tragic stories where occasionally things don't work really well. I think that the reality is, there are very many mental health challenges that come with being an entrepreneur. I used to talk about the entrepreneurial pendulum with the exclamation mark on one end and the question mark at the other end. You want to be squarely in the middle, not at either side.****When you're at the exclamation mark you are crushing the world, you think, “This business is going to be a billion dollar business, and everything's going to be amazing!”****An hour later, you could be at the question mark, which is where you’re asking, “Are we going to get crushed? Why did we ever think we could do this? It's not going to work.”****It's not a healthy place to be and—like a real pendulum—it swings back and forth so, so fast.****The IPO story—it's been embarrassing actually, but it shouldn’t be—look, I was just struggling. Being from Australia, between the IPO roadshow itself and the lead up, I had flown to and from the States, three times in under a month. We had done ten days on the roadshow, beginning on the West Coast and then going to the East Coast. I think that I had crisscrossed America six times in three weeks and then hopped the Pacific another three times—I was insanely tired. Plus, the roadshow is an always-on, mentally challenging ten days. It's real sparring, a real mental challenge with super smart investors asking you detailed and difficult questions about your business. It was engaging and interesting, but just grueling.****The day after the IPO—the actual listing—we were supposed to get up at 5am for a flight. I woke up at 4:30am and I simply couldn't get out of bed. I was exhausted both mentally and physically. I couldn't get on the plane—I had seen too many planes in three weeks. So, I wrote a very raw and highly emotional note—Jerry Maguire style—to the entire company to explain my absence; I wanted my team to know that I was grateful, but I would not be present. It wasn’t very long because mentally and physically I was totally at zero.****I hit send, closed my laptop, and just forgot about work for a couple of days. Unintentionally, it has been held up as an example of dealing with burnout.**_**Obviously, when I talked to all of your team, this is the example which came up most often when I said, “What should I ask Mike about?” I can tell it left a deep impression on everybody around you.**_**It resembles winning the Super Bowl, disappearing into the locker room, and not celebrating on the field. You've won the game, but you are utterly drained from the effort.****Though the IPO was just a financing event, it was obviously a very climactic point in the journey. At the time, there was significant confusion about why I acted the way that I did. I think that made people stop, unpack the events, and consider that maybe the party's not as important as the journey itself.****I went to Disney World for three days, slept a lot, and had crappy hotdogs. It was way less glamorous than it seems. [laughs]****On What Makes a Cofounder Relationship Work**_**We have a lot to talk about and there are two things that I want to ensure I cover. One of the things is something that you touched on: you and Scott [Farquhar]. It's not common for two cofounders to last so long and not hate each other, sue each other, or much, much worse. But you two have been at this for almost twenty years now. How?**_**Twenty years next February! The best analogy is marriage—though I don't claim to be an expert at that either.**_**Ironically, Tobi [LĂŒtke] says the same thing about Harley [Finkelstein]. In our conversation, he noted that after his wife Fiona, Harley is one of the deepest relationships that he has in his life.**_**That’s right.****Interestingly, Tobi and Harley are very, very different people—much like Scott and I are—so it’s a similar situation.****I once read a saying that you'll get married three times in your life, it just depends on whether it's to the same person. It's very powerful. Your marriage, or your friendship, years zero to twenty and years twenty to forty are very different. You’re both different people and though you may not technically remarry, you have to almost reacclimate to one another. People change a lot, and so your marriage is a very different thing at different times. I think a cofounder relationship is no different.****I’ll add that Scott and I have never disagreed massively about direction or anything of tremendous importance. We have tried to be very, very clear inside the company about who's doing or running what. The danger of cofounders or co-CEOs is that you can develop what I call “Multiple Parents Syndrome.” That is, employees would go to “Mom” and ask for something. If they don’t get it, they run straight to “Dad.”****Due to our distributed nature, we need to be crystal clear which parts of the company report where. If Scott or I are going to play in each other's areas, we are sure to ask, “What for? What is that special project that requires my involvement?” Though we’ve switched things around a fair bit and taken some much needed time off, that sense of clarity is really important for the rest of the company. Ironically, when we both got married (at different times), we each took about six months off. The other person ran our whole company for six months, which was a great experience in and of itself.****On Moving Past the “Ramen Days” Mindset****It was a refresh point back in 2009-2010. We had reached a good point, stopped looking at our feet, and started looking at the horizon. The first seven or eight years of bootstrapping we solely focused on survival. Though the business had gotten well beyond the survival stage, mentally we had yet to arrive there.****Again, by the time Accel came in, we had a $50 million business, were generating $50 million ARR and $30 million a year in profit, and were growing at like 80% a year. Yet, we were of the mindset, “Fuck, we don't know if we're going to survive.”****We literally had $50 million revenue and $20-25 million in expenses; it was ludicrous, when I look back now. But mentally we were still in ramen days.**_**That's a very common founder attribute. I won't name names, but some amazing, well-known founders are still very much in that paranoid, “We’re going to die any day now” mindset.**_**Oh, we’re still paranoid. What’s gone away is the ramen thinking.****Now, that does not mean we're not very cost efficient. We have always been incredible stewards of capital and very, very capital efficient because of our DNA. In everything we do, we don't like spending a lot of money on stuff if we don't have to.****During that period—when we began to look up at the horizon—we started to think, “Shit, what do we do now? We could probably sell this business, move to Fiji, and never work again.” It was some realization. At this point, Scott and I had spent an important year conceptualizing whether we still wanted to keep doing this. We both decided that we did and decided that we could keep at it for atleast another eight or nine years. We were excited about the future and we thought there was more goodness ahead than behind. That's when we brought on Accel as a partner and sold secondary shares. We both took long sabbaticals as a refresh and set the IPO as our destination.****On Encouraging New Employees to Break Down Walls**_**Another common thing which came up was how you ask new employees, people who are earlier in their careers, to “break down walls.” That is such a common mantra for you, can you explain it?**_**The genesis of the idea started with an Australian incubator called Startmate that I've been involved in from its inception. It resembles Y Combinator; it’s three months, high intensity, startups are flown to Silicon Valley, shown around, and then we say, “Good luck.” I think the first class raised $200,000 or $300,000 in total and is now worth nearly $40 million. It's a very successful incubator program.****The founder of Startmate, who went on to found Australia's best venture capital firm (Blackbird VC), always gave what he called “the Godzilla speech” to people when they joined. His whole idea was that you get one chance to try to put a dent in a path. People join a job, an organization, and they take a linear path. Through the Godzilla speech, he would say, “Look, all these fucking giant lizards were once tiny, and you might think you're tiny, but you have to pretend to be giant.”****His goal was to dent them off the expected path. They had joined something with an expectation of what that path was and he wanted to make them stop and think in order to change the trajectory of their curve.****I loved it and embraced that mindset entirely. My exhortation was that they start knocking down the walls of the false constructs regarding a job, a place, an organization that had set up shop in their head. Their job consisted of picking up a sledge hammer and renovating the conceptual building of the company. That is how I hold new hires accountable. The recent graduates and those earlier in their careers who have really succeeded at Atlassian have just started knocking shit down. Generally they’re a bit afraid at first, but the really good ones say, “This wall’s in the wrong place.”****One of our core values is “Be the change you seek.” In other words, move the damn wall yourself! Metaphorically speaking, I don't care how fancy it looks. I don't care if we used expensive paint or whatnot. When you take this initiative and act boldly, you will receive one of three responses:****The first response is that no one will care. If you know deep down that you've improved the company, we’ll be a better business. You should try to leave this place better than you found it and you have to derive satisfaction from that.****The second is that someone will walk past and say, “Fucking A, man. I thought that was in the wrong place,” and then continue on their walk. That's probably the best response you’ll get. You'll be frustrated because that person has been here a decade and you’ll wonder why he/she never moved the wall, but it’s foolish to be angry at the people that came before you. They were doing as best as they knew how. They built the building as quickly as they could.****The third response is when someone says, “Whoa, what the fuck are you doing?!” Now, this is the response you are afraid of in your head. This will happen less than 1% of the time and only if the proverbial wall was really important, because all the important walls have somebody guarding them. Crucially, this will trigger a conversation about whether the wall should still be standing. I'll bet that most of the time, the desire to protect the wall is not backed by real facts. It's usually some variation of “We’ve always done it this way.” It’s vital that necessary, meaningful progress win out over an organization’s spurious desire to protect itself.**_**There are two similar themes from Facebook. One is that the campus is designed to be the opposite of the Apple campus. The Apple campus is perfect. Every single spot is machined to the nth level of perfection. It looks like it's so perfectly crafted that you don't want to touch it.**__**The Facebook campus is designed to look unbuilt; it’s full of exposed cables, exposed plumbing, et cetera. Though things are a little better now, it's always meant to look exposed and unbuilt because Mark felt that that made it feel like you had a license to mess and meddle with it. If you're at Facebook, you’re like, “Well, that's just some random junk on a wall.” So even the space, the decor is meant to convey that.**__**The second thing at Facebook is that on day one, in your first few hours at the company, you are taught about how you have autonomy to experiment and how early employees did so. Either they physically moved furniture around—to use your metaphor, literally broke down walls—or did something technical. I think that sets the tone and gives people license to experiment. Of course, they always make clear what the boundaries are.**_**Yes, exactly. You want all new hires not to be intimidated by what already exists.****There's a mythology that exists in people's heads when they join an organization that tells them it was built by smarter, better people in the past. To that, I say, “Are you kidding me? You're way fucking smarter than the first ten people we hired.” One of the first ten people we hired walked in off the street and we asked, “Hey, dude, can you write Java?” There was no coding test, there was no interview; it was like, “Dude, have a go. See if it works.”**_**How do you carry that on? I think every founder wants to do just what you said. The challenge is when that new person is deemed “too much of a rebel who doesn't understand the way we work.”**_**That's a really hard one.****I often share the story of the four-minute mile to get people in the right headspace. I'm not sure if you know the story, but there were all these physics studies that said it was impossible for human beings to run a mile faster than four minutes. Then, Roger Bannister broke it in 1954. Afterwards twelve people broke the four minute mile within like a year. They didn't train any better; it was a mental thing.****In the same way, I want my team to mentally feel like they can achieve great things. I’ll give a tangible example: I personally try to promote—this is going to sound weird—troublemakers. They can't be reckless, but they can be strongly opinionated. Once you start doing that, you set an example for other people to start making a ruckus. Those people are the ones that break a lot of glass in the right way. Generally with these “really good glass breakers,” if you personally would have broken the same glass, then you need to put them into a bigger job. We built so much of this business by putting people into jobs that were far too big for them. I've rarely been disappointed by this strategy.**_**It's very important. I think leaders underestimate the power of air cover. Saying, “I've got your back, go do this.”**_**On the Power of Believing in People****I agree, but it's not even “I've got your back.” It’s often, “I believe you can do this. I honestly believe in your abilities.” I have said to a few people, “I believe in your abilities more than you do.” That might scare members of the team, but I truly believe that they can do whatever they put their mind to.**_**That's a very powerful statement.**__**You must know Tyler Cowen of Marginal Revolution. He writes a great deal and often says that one of the most valuable things you can do for a young person is to tell them that you believe in them. In terms of step function changes you can have on a young person’s career, just telling them you believe in them is one of the lowest effort, highest ROIs.**_**Yes! In my mind, Atlassian business was built by a couple of twenty-five year-olds who had no experience. Because I inherently trust young, inexperienced people to do amazing things, I strive to ensure that the organization embraces that by putting them in big jobs.****On Promoting and Killing New Things****Another example, which I shamelessly stole from Aneel [Bhusri] at Workday, is the framework of a founder’s three roles:****1.** **Culture and Storytelling****2.** **People and Hiring****3.** **Promoting and Killing New Things****The last role really stuck with me and I love the analogy he used to describe it. He compared a new product or project—a small thing with a couple of resources—to a small sapling. As founder, he would stand right beside that tree and simply say, “Nobody touch this fucking thing.” Otherwise, the very bulk and size of the organization with all of its competing priorities takes its resources and squashes it. My number one job in the area of creation and innovation is to find new, exciting things and protect them so that they have a chance to succeed.****Your other option is to kill the little tree. Two things can happen: either it's obvious to everybody that this will grow into a formidable tree that I can just wander off or I make a mistake. If I made a mistake, if there's some reason where I as the CEO think that this thing I've protected is not going to work, then I am the only person who can kill it.****I thought that was really powerful, because a great many people are influenced by founder involvement in things. Once it's a protected species, people are going to say, “Alright, well, I guess Mike and Scott like that. So don't touch it.” My job is to avoid that corrosive thinking, to kill it and remove that “magical founder energy.” You do not want to build this misshapen forest of protected things that are never going to work.****On How Growing up around the World Shaped Atlassian** _**When I was looking you up, one thing that struck me is how you moved around a lot in your younger years. You were in the US and grew up in a few places. How do you think that shaped both your personal worldview and Atlassian as a company? This is a more introspective-type question, but I think there's something about Atlassian being such a distributed company, one that works all the way across the world, which has ties to your own personal story.**_**That is a deep question.****I derive a lot of things from growing up around the globe, for better and worse. My dad excelled in his career by making hard choices. He worked for Citibank, which is part of the reason we moved around early. He grew up in England, got into Cambridge, and worked super hard. He joined Citibank early and my understanding is that he always took the jobs that no one wanted. This explains why we moved to Taiwan at the start of 1980, when Taiwan was totally different than it is today. He always stood up, said “I'll do it,” and took the difficult tour of duty. I cherish and very much respect these attributes in him.****Because we moved around a lot, we learned a great deal about all different Asian cultures. I have embraced that with my own family. For example, my ten year-old has been to every country in Southeast Asia except Papua New Guinea. Every single one. It gives you a global view and definitely shapes you in a positive way.****The other formative event really does relate to Atlassian directly. My parents are English. So, for all of their sins, one of the good mistakes they made was that they sent me to boarding school in England. If you’re English and have a little bit of money, you send your kid to boarding school. It's a bit of a rite of passage. Now the problem was, I was living in Australia and I was seven years old. So at seven years old, in the second grade, I was flying by myself four times a year to England to attend school.**_**Wow.**_**My parents had moved around a lot. My sisters were older and they had done the same thing, so the idea was that I would have one school. Thus, if they moved every two years to a different country, I wouldn’t have to move schools. There was some logic to it, but it's pretty hard as a seven year-old.****In 1987, I was flying twenty-four hours. At that time, you had to stop twice to get from Australia to England. It seemed like a big, daunting world and I had to figure it out. Over time, it instead showed me how small the world was.****One of the beliefs that’s core to Atlassian is that you can get anywhere in twenty-four hours. It's absolutely integral to my life. Pandemics aside, this last year has been tough for me. I haven't been on an airplane in a year and I'm having a weird sort of cabin fever.**_**I love it. I think the idea of your childhood and being anywhere in twenty-four hours is such a powerful line that has strong ties to Atlassian’s worldview.**_**Yes, definitely. The ability to be anywhere in the world in twenty-four hours instills the belief that you can build a business in any part of the world. That was what I learned. Coming from Australia—scratch that, from the Asian Pacific as a whole—we have an attitude. We don't share that view that all tech starts in Silicon Valley. Instead, we ask, “Hey, why can’t we do it here? No one's done it yet. It doesn't mean we can't.”****On Being Called the Heart of Atlassian**_**That's a great story**__**When I was talking to your team, they said you’ll refuse to answer this, but I'm going to try to get an answer! People say “Mike is the heart of Atlassian as much as Scott is the head of it.” Why do people say that about you?**_**I struggle to agree with that characterization, but I guess I understand why people say that. I think that values and empathy are at the heart of our company. It’s very easy to make spreadsheet-based judgments in response to hard decisions. But, I don’t know. Maybe it’s our emotion, maybe it’s our spirit, but we have a different way of doing things.****I'll share another good story about me being accidentally impactful. We had a very large team and we made the choice to exit the chat business. We sold our chat tools to Slack, moved the team onto other things, and shut down the product. It was a very pragmatic decision; I think it was the right one, but it was one of those decisions that will always haunt you. We were adding five hundred people a year, we had four or five growing businesses, and we had to kind of choose our favorite children. Though we built an amazing thing and we had, I would argue, the best product at the time, we had to kill it.**_**I was using HipChat at the time. Twitter was an all-HipChat company back then, so I was very familiar with that. How did shutting HipChat down send a message to the team?**_**Whenever we’re shutting something down, there are often a series of meetings that Scott and I have to attend that are very, very hard. I think that how those meetings and delicate situations are handled is incredibly important. I often reflect on how to approach them with the right level of emotion and understanding.****I mean, people are terrified at that point. They’re wondering, “What's going to happen to my job, my livelihood?” All these things are racing through their heads. Though I'm not working in this office or on this product, I have to identify that they’re going through a really tough time. In this instance, we made it very clear that nobody was going to lose his/her job. But still, it’s very difficult for the people who worked on this product and battled in the trenches day in and day out.****To allude back to Aneel’s framework, I had to be the one to turn up and say, “Sorry, guys, we're shutting this whole thing down.” It's been a really vital business lesson, but one of the hardest that I have had to learn. When I shared the news with the broader team of a couple hundred people in Austin, I couldn't even make it through the whole spiel. I ended up standing in front of a room of two hundred people effectively bawling and trying to hold my shit together. Showing up and just being honest about it—that sucked, man. That was super, super hard for me and I wasn’t the one that was at a keyboard twelve hours a day for the last two years. The words didn't matter at that point. People understood that this was an emotional decision. It was one of those classics where most people agree with it, but they don’t like it.****I think the emotion that I showed resonated and reverberated around the company. Someone in the Bangalore office, for instance, likely heard that story and thought, “Okay, well the company did right by those people and I have some friends on that team. As a person, I feel really badly. I might disagree with it, but I feel like it was handled openly with empathy, with understanding, and with heart.” That's just the way we roll.****On COVID-19 and His Passion for Climate Action**_**I want to ask you one last thing. You hinted right in the beginning at this when you said that you have multiple interests. Talk to us about these multiple interests, especially around your focus on climate change.**_**I think it probably sounds trite, but it is the biggest existential threat to humanity. There are very many pieces to this.****It's an existential threat to humanity, not an existential threat to the planet. Don't confuse or conflate these two things. The planet will be just fine. Long after we're gone, the carbon cycle will run itself and the planet will be all good again. It’s just unfortunate that in a million years or whatnot, we won't be around, we will be extinct as a species. It’s very, very frustrating to me. When you understand the technology and you have a bit of money, people begin to listen to you. Because of this, I've gotten heavily involved.****From a planetary point of view, we have the science, the technology, and the understanding to solve this problem. People will tell you we don't. That's bullshit. We don't need any new technologies, we have them today. What we lack is the political will and the economic incentives to solve the problem. To be clear, this is not a science and technology problem, it's a political and economic problem. Ironically, if we rebuilt the world from scratch today, we could have a cheaper, better world system that was entirely renewable-powered and emission-free.****The existing world is the complicated part of this equation. There are jobs, economic interests, shareholders, and wealth inhibiting this progress. Many people have a vested interest in this disruption not occurring. In fact, it resembles the classic Silicon Valley problem: how do we disrupt the old to get to the new? As a half-finance, half-technology guy who enjoys science, it is incredibly frustrating as it will continue to grow exponentially if left unchecked.**_**Tech people are familiar with exponential growth.**_**Exactly, it resembles COVID in this way. If you shut down exponential growth when it's tiny, it's easy. Once it gets growing, tragic circumstances proceed from inaction. With climate change, there is a very clear exponential cost. I believe as a human species, we have to solve this problem.****We will solve it—I'm an optimist on climate— but the cost and difficulty of solving it ten years from now versus today is exponentially higher. We need to act sooner rather than later!**_**This was amazing. I just love your passion and energy, Mike. Thank you. I hope it was fun for you. It was super, super fun for me to meet your amazing team, learn about you, and see how much everyone loves you. This was so delightful.**_**I appreciate it, thanks for having me.**_**Thank you for reading the The Observer Effect. You can**_ [_**subscribe here**_](https://www.theobservereffect.org/mikecannonbrookes.html?utm_source=substack&utm_medium=email#) _**and get an email for every new interview (which should be about every few weeks). Send any thoughts/comments/questions by**_ email _**or via**_ [_**Twitter**_](https://twitter.com/ObserverOrg)_**.**_"}

Response:

{
  "error": {
    "message": "This model's maximum context length is 8191 tokens, however you requested 11559 tokens (11559 in your prompt; 0 for the completion). Please reduce your prompt; or completion length.",
    "type": "invalid_request_error",
    "param": null,
    "code": null
  }
}

Once again, it seems to be related to the length of the content to embed.