A well-known design decision of Elasticsearch is that a fixed number of shards has to be specified when creating an index.It is not possible to start out with just one or only a few shards and add more shards later as the data increases.Now what to do if we find ourselves in a situation where the capacity of an index is exhausted and we really need to extend that index?

singles dating  index1-43

Probably, just following that recommendation will satisfactorily handle many scenarios arising in practice.

Also, when designing a system, we should spend some time thinking about the expected amounts of data anyway, even without any sharding involved.

On the other hand, there is no guarantee that our expectations or measurements are correct – unexpected success of a startup is a typical example.

First of all, let’s try to understand why Elasticsearch doesn’t provide anything like dynamic shard splitting.

Quite a few other database and search engine products offer shard splitting so that extending a database or an index can be done conveniently via a single API call.

The most obvious advantage of the approach taken by Elasticsearch is that a particular shard key will always point to the same shard.

There is no need to implement a complicated shard splitting algorithm, with heavyweight behind-the-scenes data migrations, high disk space requirements, and all kinds of unexpected error situations.

Further reasoning, shedding some light on why shard splitting is considered a bad idea, is given here.

The recommendation for Elasticsearch users goes like this: First, estimate the capacity of a single shard by performing measurements with realistic amounts of data.