Elastic now provides a bit more cautious rationale :. A kagillion shards is bad. It is difficult to define what constitutes too many shards, as it depends on their size and how they are being used. A hundred shards that are seldom used may be fine, while two shards experiencing very heavy usage could be too many. Shard overallocation should be especially avoided when we are dealing with small static datasets that are not expected to grow and for which new indexes are created regularly according to Index Management Lifecycle IML.
In fact, there are several considerations to keep in mind when you select the shard count for your indexes. Even though there is no fixed limit on shards imposed by Elasticsearch, the shard count should be proportional to the amount of JVM heap available. No matter what actual JVM heap size you have, the upper bound on the maximum shard count should be 20 shards per 1 GB of heap configured on the server.
So, for example, for a node with the heap size of 32 GB close to the maximum we should have a maximum of shards. This is the upper bound on the shard number per node and should not be considered to be the recommended value. By all means, try to keep the number of shards per node as reasonable as possible, especially in the case of small static indexes.
In general, try to keep the shard size between 1 and 5 GB for such indexes. Also, a very important practice that can help you determine the optimal shard size is benchmarking using realistic queries and data. Benchmarking should always be done with queries and index loads similar to what you expect in production.
This will help you determine the optimal shard size. Production clusters should always have at least 2 replicas for failover. Our customers expect their businesses to grow and their datasets to expand accordingly.
There is therefore always a need for contingency planning. In addition, we all want to minimize downtime and avoid resharding. We strongly encourage you to rely on overallocation for large datasets but only modestly. If you know you will have a very small amount of data but many indexes, start with 1 shard, and split the index if necessary. If you estimate you will have tens of gigabytes of data, start with 5 shards per index in order to avoid splitting the index for a long time.
For GB index, that would be 25 shards. If you estimate you will have terabytes of data, increase shard size a bit. For example, for 1TB index 50 shards could be a relevant suggestion. These suggestions are only indicative - optimal values depend heavily on your usage pattern and forecasted growth of data in Elasticsearch.
You can change number of shards without losing your data, but this process will require a brief downtime when index is rewritten. Having a large number of indexes or shards affect performance you get out from Elasticsearch. Some rough numbers from three-node Aiven Elasticsearch business-8 cluster:. Because of this, a machine with more IO headroom think SSDs and a multi-core processor can definitely benefit from sharding.
Creating indices with only one shard will negate that feature, limiting your future growth without a total reindexing of your data. When you were running these tests… were your queries limited to a specific shard or did you query all shards in the index each time? One of the biggest benefits of ES is that you can route your search to a specific shard to avoid that extra query cost in situations where you know how your data is distributed.
I am curious if the results would be much different if you limited your queries to only the shard where you knew the data existed. Thanks for sharing the information. Can you please indicate how much was your average document size? Based on the above are you getting documents per second as your Write TPS, right?
What is the best approach to follow with Elasticsearch to handle big set of data check question details? The natural way to do this seems 2 — create a shard per month.
I would try this for a single month first and measure performance. Next, see how performance is influenced by scaling to two and then four months. AFAIK, the impact on performance should be….
To avoid this issue, make sure that every index in your cluster is initialized with fewer replicas per primary shard than the number of nodes in your cluster by following the formula below:. Where N is the number of nodes in your cluster, and R is the largest shard replication factor across all indices in your cluster.
In the screenshot below, the many-shards index is stored on four primary shards and each primary has four replicas. To resolve this issue, you can either add more data nodes to the cluster or reduce the number of replicas.
In our example, we either need to add at least two more nodes in the cluster or reduce the replication factor to two, like so:. After reducing the number of replicas, take a peek at Kopf to see if all shards have been assigned. Shard allocation is enabled by default on all nodes , but you may have disabled shard allocation at some point for example, in order to perform a rolling restart , and forgotten to re-enable it.
If this solved the problem, your Kopf or Datadog dashboard should show the number of unassigned shards decreasing as they are successfully assigned to nodes. It looks like this solved the issue for all of our unassigned shards, with one exception: shard 0 of the constant-updates index.
Pinpoint and resolve unassigned shards and other Elasticsearch issues with Datadog. In this case, primary shard 0 of the constant-updates index is unassigned. It may have been created on a node without any replicas a technique used to speed up the initial indexing process , and the node left the cluster before the data could be replicated.
Another possibility is that a node may have encountered an issue while rebooting. When this process fails for some reason e. In this scenario, you have to decide how to proceed: try to get the original node to recover and rejoin the cluster and do not force allocate the primary shard , or force allocate the shard using the Cluster Reroute API and reindex the missing data using the original data source, or from a backup.
Before proceeding with this action, you may want to retry allocation instead, which would allow you to preserve the data stored on that shard.
0コメント