How we launch a user-facing feature every week

At Elicit, we try to launch a feature to users every week. This effectively means that our cycles / sprints are 1 week. We’ve been doing this since September 2021, when we launched the first version of literature review. 

Two years later, we’ve launched over 88 features across 125 weeks and we still launch a user-facing feature every week. That’s an average of 1.4 weeks per user-facing feature (note it doesn’t count all the features we don’t announce to users).

Today, we have 7 people directly executing on this cycle across a mix of engineering, design, product, and evaluation roles. This is a core part of how our team operates, what sets us apart from other teams, and what attracts teammates to Elicit. Here’s what we’ve learned from 2 years of executing at this pace. 

We first started doing this in September 2021, when we decided to support literature review workflows in Elicit. At that time, Elicit still lived within Ought. We knew we wanted to apply the earliest GPT models as research assistants and had iterated on a few different workflows with researchers. We were just starting to see signs of life with our “Find experts” workflow.

But we couldn’t ignore the fact that most people who magically ended up on our obscure landing page and signed up despite no information told us that their pain point was literature review. 28% of people signing up said this, vs. 13% for the next most popular category.

We personally knew literature search was a problem from our own research experiences, and had even considered building a lit review tool in February 2020. But the problem seemed too big and underdefined. What even is literature review? People couldn’t describe their processes and everyone seemed to want a different solution.

Eventually, the team felt like this was the right problem, even if we couldn’t quite circumscribe it or come up with interesting hypotheses for a solution. And in a stroke of magic, Andreas hacked a prototype over a weekend to summarize paper abstracts with GPT-3. Our minds were blown.

This was the first application of language models for research. 

I was still terrified about committing to this, especially when I could see such a clear use case for why Find Experts would be a valuable workflow. I also have a violently negative visceral reaction to complex solutions to underdefined problems. And I was paranoid that our researchy team might spend too much time on solutions that are interesting but not actually useful. I did not want to build in stealth for months only to launch with the completely wrong solution. 

So we decided that we could work on lit review but with the constraint that we would have to launch a user facing feature every week and double our user growth every month. If we could ship something every week and get feedback, we wouldn’t get lost in the whitespace. If we were going to fail, we were going to fail fast. 

It's going great. We are still launching a feature every week. This is now a core part of who we are. These launches set a rhythm for the entire business, not just the engineering and product teams. It’s why our investors invested in us and why great teammates have joined us. It’s how we grew to over 200,000 monthly users by word of mouth and $1 million in ARR 4 months after launching subscriptions. 

It’s actually not workaholism and heroism and burning out every day. All of us definitely work hard. But somehow we’ve managed to get into a groove where this level of execution is possible and sustainable. 

Why this is good

Startup culture already worships moving quickly and working hard so the main benefits of launching a feature every week is probably obvious to most people reading this. That being said, there are some less obvious benefits to launching a feature every week:  

• Keep debates small
  Because we’re launching so many features and can quickly course-correct, we don’t need to spend a long time getting consensus on any one feature. We reduce time spent on theoretical arguments and avoid making decisions based on opinions or authority. We just launch the feature and see what happens. This allows us to be truth-seeking without getting analysis paralysis. 
  
  We also reduce sunk cost - no one gets attached to any one particular feature or project. 
• Keep scope small
  If you have to launch something in a week, you’re forced to ruthlessly cut scope. This is really important for startups, which have unusually limited resources in an unusually large action space. It’s even more important for a startup like ours, where the user problems are unusually vast and underdefined. Scope is a vicious cycle - big scope leads to more big scope which slows down exponentially more than you may expect. 
• Attract other talented people who want to move quickly
  Our ability to launch a feature every week attracts talented candidates. One of our investors, Seth at Fifty Years, recently said “Great people leave companies because they’re not moving fast enough.” 
• Get more practice
  I strongly believe in the value of practice. Launching a feature every week allows you to get twice as many reps in over the course of a year than even a team that launches once every two weeks. I want us to build the greatest product execution machine ever. Again, no one feature matters. But success hinges on the machine running well. To do that, we have to practice a lot and get a little bit better with each rep. 
• Engage your users without spamming them
  Emailing your users every week telling them about how you fixed one of their problems or gave them something new to try is so much better than generically asking them to try your app again. The open rate for the email we send every week  is about 40%. Our unsub rate is ≤ 0.5%. We get a lot of positive qualitative feedback on our emails. 

What we’ve learned

So that you can learn from our mistakes:

1. Keep a backlog of features to announce
   Sometimes we knock out a bunch of features at once, or a few features happen to launch around the same time. It’s helpful to keep some of these in the backlog to smooth over the weeks where launches take longer. 
   
   If we want to work on a bigger project, we’ll also sometimes do a feature reminder email or a user education email.  
2. Keep the cycle focused & conservative. Protect cognitive overhead. 
   In the beginning, we tried to do too much each week. (This might not be specific to launching a feature every week.) We stuffed our cycles with everything that had any chance of getting done. There were too many dependencies based on the results of certain experiments, which was too hard to coordinate across 7 people and many time zones. 
   
   Now, we try to have 1 main focus or launch for the week. We only fit things in the cycle that we are highly confident can get done that week. To keep us honest, we commit to doing everything in the cycle no matter what. We pull things forward from following cycles if we can. This is one of the biggest things I’ve learned over the last year (from our Head of Engineering, James). 
   
   We’ve learned that we can’t squeeze in launches during retreat weeks and it’s worse to try. (It prevents people from being present during retreats.)
   
   We’ve gotten better at understanding the different stages of a project (roughly something like ideate, experiment, evaluate, design, build, and dogfood / evaluate again). We now know that we can’t go from idea or experiment to launch in the same week. Unlike traditional SaaS, we can’t always know if a certain approach or idea is even possible. 
   
   Relatedly, in the beginning we would try to make changes to the cycle late on a Friday, early Monday, or mid-week. This was chaotic and stressful for everyone. While we wanted to move fast and constantly update in light of new information, this level of change wasn’t effective. 
3. To avoid only launching incremental features, plan better
   In the early days of trying to launch a feature a week, it was easy to fall into the habit of launching small features that we could fit into each week. Avoiding that requires careful, advanced planning and people who can plan large-scale projects in detail, breaking them down into week-size bites. 
4. Practice, so that you don't burn people out
   Another failure mode is people burning out. Be careful not to launch month-long projects in a week, every week. The big projects must be broken up and sometimes you have to be fine with launching small things as you spend most of your resources preparing for a big thing. The key is everyday excellence, not spiky heroism,
   
   At the same time, when we spent too much time not launching weekly, the team asked to go back to it. Again, great people want to launch things and see that their work has an impact. 

What we want to do better 

Despite having iterated on this for a few years, there are still many things we can improve about operating like this. If you try to launch a feature a week, you may have to work through these challenges too: 

1. Plan projects, not just weeks 
   Planning a week really well is different from planning a project really well across multiple weeks. We’re now fairly good at organizing work on a week-by-week basis but we’re still learning how to organize larger multi-week or multi-month projects. We need to practice organizing projects and staggering them into our weekly launches, instead of reaching for features that can fit neatly into a week or two. If not, we’ll err too far on the side of launching small features, have to stop doing launches when we’re working on a big project, chaotically stuff large projects into weeks, have crazy fire drills, and / or ship features below our quality bar. 
2. Close the loop & monitor quality. Don’t launch & forget
   Focusing on launching a new feature every week can trade off with learning the most from each launch. Right now, our team leans heavily towards getting things out. We’ve underinvested in monitoring each feature’s use to make sure that the people discover and use them as we hoped. Going forward, we want to get really good at identifying the “theory of change” of each feature so that we’re not just taking a spray and pray approach, but developing rich models of what our users need and how they react to changes in Elicit. 
   
   Often, teams move less slowly in part because they want to release something higher quality. My hope is that we can grow our team and continue improving our execution so that our initial launches are higher quality without sacrificing speed. “Improve quality of X” should just be a followup launch that helps us maintain our rhythm.  
3. See past the next 4-6 weeks
   Focusing on launching a feature every week is hard enough that it can leave very little time for planning further out. Our team often says that they wish they could have more clarity on the product roadmap beyond the next few weeks of feature launches. I’m not sure this is specific to launching a feature every week - this might be a function of our stage, not having had a PM for the last year, and the fundamental uncertainty of our work. But I’m hoping that with our new Head of Product and the roadmap coming out of our 2024 retreat, everyone will know where we’re going over the next few months and help us get there faster. 
   
   When people can’t see far out into the future, they can’t proactively pull issues forward, or make progress when they have extra space in the cycle. This means the cycle planners (PM, Head of Eng) have to make sure everyone has something to do each week (which may overstuff the cycle), or people do random things that are cool but not on the critical path (and thus get blocked later or feel slightly demoralized from working on something that wasn’t immediately valuable). I hope that having a longer-term roadmap will give people agency as they move further ahead on the roadmap without compromising their impact, and help them feel more excellent as they start moving ahead of plan. It will also help people make better design & implementation decisions today if they know how certain infrastructure will need to be repurposed in the future. 

The ability to launch a user-facing feature every week feels like a distinctly Elician superpower. Our goal is to keep this going for the entire life of our company even as we get bigger and everything about our work gets more complex. Speed & rapid experimentation seem like the one thing that every company loses as it grows. Other things like impact or quality might improve with scale but speed never does. So it feels uniquely important to fight for starting from the beginning. 

While all startups want and should move quickly, it’s uniquely important & possible for generative AI-based startups. The technology and the landscape go through massive upheaval every few months.