Founder's Guide to Scaling Applications: When to Build, When to Buy and What Breaks

Algolia’s search-as-a-service, and full suite of APIs, allow teams to easily develop tailored, fast Search and Discovery experiences that delight and convert.

When it comes to seamlessly scaling your applications, a top-notch engineering team will be your foundation. Next comes the decisions to build or buy your infrastructure, DNS, monitoring, and analytics tools. Julien Lemoine, a Frenchman and Co-Founder/CTO of Algolia, shares his lessons-learned about how to stay focused and innovative as you scale while also avoiding the innovation-for-innovation’s-sake pitfalls.

The Positive and Negative Effects of a Great Engineering Team

About this topic, Mr. Lemoine said, “We all see the positive effects of having a great engineering team. Of course, good engineers can build a product way faster than others; it has a key impact for your company. But we don't discuss enough about the negative aspects of a very strong engineering team. First of all, I think any good engineer can easily give you 10 good reasons to build instead of buy, and they will always underestimate the long term costs and the long term impact of building vs. buying.”

Build vs. Buy: Decision #1 - Infrastructure

About this topic, Mr. Lemoine said, “It was in the very early days of the company; we were only two [people]. We had not even fund-raised, we had no salaries... so [the] very, very early days of the company. We decided to use bare-metal infrastructure and not use cloud infrastructure VMs, and a lot of people misunderstood us. The reason we took this decision was because of our bet on the market.

We were working in this field for more than 10 years before creating the company. We were convinced people wanted to have good performance. We wanted to have a [performance] factor of 10, but the hardware, even the network, play a big role in the performance. Search engines are very intensive in terms of CPU, SSD, memory. So, we had to select a very specific machine; very specific hardware. None of that was available on cloud infrastructure at that time, so that's why we took this crazy decision to use bare-metal and we had to build everything from scratch.

So, the first version of the software had zero availability, it was one server running on one provider and it was - on one side - kind of risky because one big hardware failure could cause us a lot of trouble, but it was the easiest way to get the feedback we needed.

We didn't [have] even 1% of the features we have today, but we had a big difference in terms of performance, compared to everything else on the market. And, the reaction we got was super positive; it confirmed our initial bets.

Long story short... today we have infrastructure on the cloud and it took us some time to have the good infrastructure, the good hardware, in Cloud to run our API. Today there is way more variety of hardware on cloud providers, then was the case in 2012.”

Decision #2 - GEO Aware DNS

About decision number two, Mr. Lemoine said, “The GEO distribution is done by the DNS and we were using a solution from Amazon AWS.To do that, we had one big issue first, which was a number of regions they were supporting; they were not the same like we had at the moment of the launch - 12 regions; Amazon was covering only 7 of them. So, we were not able to do the GEO routing for all our customers.

Second problem - which was a performance problem - when we were creating a region like a specific country - for one customer, it was taking a few minutes. Meaning, in the onboarding flow of our users, it was causing a huge drop in in the process. So we decided to look for other [DNS] solutions on the market. None of them was able to address our use case. It was a big disappointment.

As I mentioned, a good engineering team likes challenges. So, in no time, [our] engineers built a small prototype - based on open source - and we negotiated it with a network provider to build a network with our 12 regions to redirect the traffic to our custom-built DNS.

We decided to use a startup. When there is no solution in the market a good engineering team will always find this and they will come to you and give you a big list of things you could do way better than anyone else. Even if they have no expertise in this market.”

Decision #3 - Monitoring

About this topic, Mr. Lemoine said, “So this [phase] was way later. We were like 35 engineers focusing on performance. So, of course the monitoring of our API, monitoring our performance was key since the beginning. We were using a SaaS product to monitor our infrastructure but we hit a few limits in the solution.

The engineering team, one day, decided to build a prototype. So, they built a prototype internally - based on open source - and in one day they were able to have something better than our SaaS solution.

Even if it was better when we were looking at what we are missing in this solution, the list was huge. We had dozens, if not hundreds, of missing features for our needs. So, it means our team would have to develop all those features of our performance, and it means my road map would be putting some emphasis on those metrics which are not game changing for the business. It's not something which is exposed to customers.

Thinking short term; building something in a day which is great. It's a good engineering effort. But then, on the long-term, I prefer to pay for an external service. Again, good engineers, they have 10 different ways to explain to you that the best solution is to build and not buy. I never managed to convince them [to buy and not build], but one day the lead engineer discovered a startup, and we had exactly the same idea. Even if they were not able to execute, our vision of monitoring would have been way better than spending our time building prototypes.

The engineering team recognized that they would never have been able to develop the solutions they discovered. Internal teams will never be able to compete with a SaaS product. Never.”

Decision #4 - Analytics

About this topic, Mr. Lemoine said, “On every search [for our users] we monitor all the API code, and we give them some statistics. These analytics are also business-critical, because we used it for the billing. But, we knew the solution was not scaling. We looked at the solutions on the market. Nothing was perfectly good for our problem and especially because our scale was so big. We were already processing billions of API. But the price [for the solution] was too high for us. We would lose money by using a provider of analytics.

So, one of our best engineers, [for] close to a year, was spending a lot of time just on maintaining a solution... which is not good. We should have paid for the very expensive solution for a year. We would have lost money for sure, but I would have gained the time of one of my best engineers, that could have worked on something more critical.

How to Not Reinvent the Wheel

In closing, Mr. Lemoine said, “We all make mistakes and I think we have to deal with it. Accept [that] we can replace components easily over time and don't think about building something for the long term.The cost of maintaining a solution can be super high, so do not try to reinvent the wheel. I think one of the things that I am most proud of, is probably our engineers. They saw all those mistakes, and have become better engineering leaders.

I worked in a big group that in the 80s developed a specific software, two versions of a source code. The software was great. To this day, I still use it. They have a world team to maintain it. 30 years later they still maintain their internet choice and they are not able to migrate. So, the earlier you can move from something you build, to an external solution, the earlier you can keep your focus and the speed of iterations. The focus is a critical element in any engineering team. The speed of iteration is what will make the difference.”