The Future of Observability

How is observability changing in recent years, and what next?

In recent years, businesses have become increasingly reliant on observability to manage and maintain complex systems and infrastructure. As systems become even more complex, observability must evolve to keep pace with changing demands. The big question for 2023: what next for observability?

  • On-premise cloud era led to a few companies like Solarwinds, BMC, & CA Technology.
  • Cloud era (i.e. where AWS came in) led to a shaking market, with new companies like Datadog, New Relic, Sumologic, Dynatrace, Appdynamic, and more.
  • Cloud-native era (starting in 2019–20) has resulted in another market shakeup.

Why is observability changing in 2023 and beyond?

The main reason for the current shakeup is that businesses are building software using an entirely different technology from 2010. Rather than monolithic architectures, they use microservices, Kubernetes, and distributed architecture.

  • Better security 🔐
  • Easy scalability 📈
  • More efficiency for distributed teams 👬

Prohibitive costs

The first problem is relatively straightforward: cost. All legacy observability companies have become so expensive that most startups and medium businesses can’t afford them. As a result, they’re using old technology to host and process their data — technology that can’t respond to startups’ needs in 2023.

Evolving priorities in observability

Additionally, as the capabilities of observability have become more advanced, the KPIs and OKRs that dev and ops teams track have evolved.

  • Request latency
  • Saturation
  • Scalability
  • Traffic maps for where usage is happening
  • Optimizing and predicting future outcomes
  • How new code changes cloud usage

Changing expectations for observability

Finally, the rise of microservices architecture changes how IT teams observe application changes. One microservice can run across a hundred machines, and a hundred small services can run in one machine. There’s no “one-size-fits-all” approach. Dev and ops teams need deeper analysis to understand what is happening across their infrastructure.

What will the new generation of observability tools need in 2023?

These are the challenges. So how should the new generation of observability tools respond in 2023? From my perspective, here are seven things we will need to win the market.

1. Unified observability

All the legacy companies say they’re an unified observability platform. What this really means is that they have different tabs for metrics, logs, traces, etc. accessible from their platform.

Middleware Unified observability

2. Integrated observability & business data

As Bogomil from Sequoia mentioned in this blog, most businesses don’t correlate their observability and business data. This is a problem because there are powerful insights to be gained from analyzing the two side by side.

3. Vendor-agnostic (OTel)

Companies are looking for a solution that doesn’t lock in one vendor. That’s how most tech companies are contributing to open telemetry and making otel the go-to tool for data collector agents. OTel has many benefits like Interoperability, flexibility, and Improved performance monitoring.

4. Predictive observability

In the AI era, everything is moving to become a human-less experience. This can enable systems to do the things that humans simply cannot, like predicting errors before they even happen via machine learning.

5. Predictive security in observability

Observability and security work very closely. Most observability companies are moving to security because they control all the data collected from applications and infrastructure.

6. Cost optimization

Perhaps the biggest challenge in observability is cost. Although cloud storage is getting cheaper and cheaper, most observability companies aren’t lowering their prices to match. Customers get the short end of the stick, mainly because there are no alternatives.

7. Correlation to causation analysis

Most legacy observability platforms give basic information about what’s happening in the cloud or application. However, many times the inciting event takes place hours or even days before. As such, it’s important to monitor CI/CD pipelines to see when code gets pushed, as well as which regulation or request starts to create the problem.

8. AI-based alerts

Alert fatigue is a real challenge. When developers receive so many alerts that they mute email threads or Slack channels, this hides issues and slows down time to resolution.

Final thoughts on the state of observability

This is an exciting time to be in observability. As I mentioned earlier, the changes we’re seeing are opening the door to untold opportunities. The question remains: who will rise to the top in 2023?

--

--

Laduram is an entrepreneur, thought leader, and tech investor. Founder and CEO at Middleware.io, ex founder of Acquire.io and angel investor in 30+ companies.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Laduram Vishnoi

Laduram is an entrepreneur, thought leader, and tech investor. Founder and CEO at Middleware.io, ex founder of Acquire.io and angel investor in 30+ companies.