Containers are no longer a niche technology—more than 35 percent of Datadog customers with at least 500 hosts have adopted Docker, according to our latest research on Docker usage in the real world.
As containerization becomes more mainstream, we decided to examine the ecosystem of technologies for orchestrating container workloads. Platforms such as Kubernetes, Mesos, Amazon Elastic Container Service (ECS), Azure Container Service, and Google Kubernetes Engine provide a layer of abstraction and automation to help manage large numbers of ephemeral containers. The research that follows dives into how Datadog’s customers use orchestrators, especially ECS and Kubernetes, to manage millions of containers on any given day.
ECS prevails in AWS,
but Kubernetes leads elsewhere
The environment in which containers run appears to strongly influence the choice of orchestrator. Most of our customers running a container orchestrator in AWS choose ECS, although we also see substantial use of Kubernetes in AWS environments. In Google Cloud Platform, Azure, on-prem data centers, and other environments, the vast majority of companies choose Kubernetes for orchestration.
It is worth noting that Amazon launched two services in December 2017 that will likely reshape the orchestration landscape in AWS environments. Amazon’s new Elastic Container Service for Kubernetes (EKS) brings managed Kubernetes clusters to the AWS cloud, and AWS Fargate is a new service for deploying containers without managing underlying infrastructure. We will track how these services impact orchestration usage over the coming months.
Kubernetes’ share of Docker environments
gained 11 points in 9 months
As more companies adopt Docker, we have seen greater use of container orchestratorsas well—especially Kubernetes. (In this report, Kubernetes data includes companies running self-managed clusters, as well as users of Google Kubernetes Engine, Azure Container Service, and other hosted Kubernetes services.)
Kubernetes holds a 41-percent share of Docker environments as of October 2017, as compared to only 30 percent at the start of the year. Considering Docker’s rapid growth, that corresponds to an increase of about 50 percent in the share of all Datadog customers that are running Kubernetes.
ECS’s share of AWS Docker environments
gained 7 points in 9 months
As orchestration becomes more popular, companies in AWS are increasingly adopting Amazon ECS to manage their container fleets. As of October 2017, ECS holds a 34-percent share of Docker environments in AWS, up from 27 percent in January. Because Docker usage on AWS is increasing overall, that corresponds to an increase of about 40 percent in the share of all Datadog customers that are running ECS.
Orchestration shortens Docker
host lifetimes by 40%
The reach of orchestration goes beyond the container fleet—it also impacts the underlying infrastructure. In clusters under orchestration by ECS or Kubernetes, hosts have an average lifespan of approximately 10 days, as opposed to 17 days for Docker hosts without orchestration. Non-Docker hosts, for comparison, live an average of 23 days. We conclude that orchestration enables organizations to focus on services and workloads, treating hosts as disposable infrastructure units that can be automatically provisioned or destroyed based on demand.
Kubernetes containers churn
8x faster than ECS containers
Although the choice of orchestrator depends greatly on the infrastructure environment, ECS and Kubernetes tend to exhibit significant operational differences once they are deployed. In particular, Kubernetes workloads churn through containers extremely quickly: in Kubernetes environments, containers have an average lifespan of just 1.5 days, compared to 12 days for containers managed by ECS. This discrepancy may be due in part to the availability of Kubernetes objects such as jobs, which automatically terminate containers once they run to completion.
ECS runs fewer off-the-shelf images
In Kubernetes clusters, we see widespread deployment of container images for common infrastructure technologies like NGINX, Elasticsearch, and Postgres. These same images tend to appear in unorchestrated Docker environments as well, albeit in smaller numbers. But in ECS clusters, our research shows very little adoption of common, publicly available container images: only NGINX appears in more than 10 percent of ECS environments. We conclude that many ECS users have likely adopted hosted AWS services such as Amazon Relational Database Service (RDS), Amazon Elasticsearch Service, and Amazon Simple Queue Service (SQS), rather than running containerized services for those infrastructure components.
Kubernetes has 3x the container density of ECS
At the median Kubernetes organization, each host runs about nine containers over a five-minute sampling window, versus just three containers in the median ECS organization. Combined with the vastly different churn rates for containers found in Fact 5, it appears that Kubernetes users tend to run large numbers of short-lived, lightweight containers, whereas ECS is home to longer-running containers that are closer to standalone applications.
Most organizations mix “latest” containers
with specific versions
When pulling a container image from Docker Hub or another container registry, the user can retrieve a specific version of the image by providing a tag. About 16 percent of organizations use the :latest tag exclusively, which may be an explicit or implicit choice: :latest is the default for docker pull commands if no tag is provided. About 10 percent of organizations, on the other hand, always provide a specific version tag when pulling a container image. The remaining 74 percent use a mix of tags—pulling the :latest image in some cases and specifying a particular version in others.
