Elastic Stack 6.x Upgrade Guide

Your browser lacks required capabilities. Please upgrade it or switch to another to continue.

Loading…

!!Planning to upgrade your Elastic Stack to 6.x? You've come to the right place! Answer a few questions, and we'll build a list of the steps you need to take and point you at the relevant docs. Ready to get started? [[Let's Go!->Stack]] \ <<set $version = "unknown">>\ <<set $cloud = "unknown">>\ <<set $upgrade_56 = "unknown">> <<set $upgrade_60 = "unknown">> <<set $xpack = false>>\ <<set $xpack_security = false>>\ <<set $enable_tls = false>>\ <<set $kibana = false>>\ <<set $beats = false>>\ <<set $logstash = false>>\ <<set $v2indices = false>>\ <<set $v1indices = false>>\ <<set $upgrade_assistant = false>>\ <<set $shield = false>>\ <<set $watcher = false>>\ <<set $marvel = false>>\ <<set $reporting = false>>\ <<set $debug = false>>\ <<set $oss_only = false>>\ <<set $logstash_template = false>>\

!!X-Pack 5.6 Upgrade Assistant \ The Upgrade Assistant in X-Pack 5.6 identifies issues you need to address before upgrading and simplifies migrating indices that need to be reindexed before you upgrade. The Upgrade Assistant is free to use with a Trial or Basic license. You can install X-Pack solely for the purpose of upgrading, there's no obligation to continue using it once you've upgraded. If installing X-Pack isn't an option (because X-Pack isn't open source or you'd need to get approval, for example), we'll show you the manual steps you need to take before and after upgrading. <<link "Yes">>\ <<set $upgrade_assistant = true>>\ <<set $upgrade_60 = "rolling">> <<goto "Upgrade list">> <</link>> I'll use the X-Pack Upgrade Assistant <<link "No">>\ <<set $upgrade_assistant = false>>\ <<set $upgrade_60 = "rolling">>\ <<goto "Upgrade list">> <</link>> I'll upgrade without installing X-Pack

!!Upgrading from 5.x to 6.x \ <<if $cloud == "yes">>\ You must upgrade to 5.6 before upgrading to 6.x. \ [[Next->Upgrade list]]\ <<else>>\ You have a couple options:\ * Avoid down-time by upgrading in two steps. Perform a rolling upgrade to 5.6, and then perform a rolling upgrade to 6.x. * Shut down the cluster and upgrade directly to 6.x.\ If your stack includes Kibana, Logstash, Beats, or X-Pack, we recommend upgrading to 5.6 and using the X-Pack Upgrade Assistant to prepare for the upgrade to 6.x. <<link "Upgrade to 5.6 first">> <<set $upgrade_56 = "rolling">>\ <<set $upgrade_60 = "rolling">>\ <<if $xpack == false>>\ <<goto "Upgrade assistant">>\ <<else>>\ <<goto "Upgrade list">>\ <<endif>>\ <</link>>\ <<link "Jump straight to 6.x">> <<set $upgrade_60 = "fcr">>\ <<goto "Upgrade list">>\ <</link>>\ <<endif>>

<<if $cloud == "yes">>\ <<include "cloud-upgrade">>\ <<else>>\ !!Here you go! These are the steps you need to take to upgrade to 6.x. <<if $upgrade_56 == "fcr">>\ <<include "fcr-upgrade-56">> <<endif>>\ <<if $upgrade_56 == "rolling">>\ <<include "rolling-upgrade-56">> <<endif>>\ <<if $upgrade_60 == "fcr">>\ <<include "prepare-60">>\ <<include "fcr-upgrade-60">> <<elseif $upgrade_60 == "rolling">>\ <<include "prepare-60">>\ <<include "rolling-upgrade-60">>\ <<elseif $upgrade_63 == "rolling">>\ <<include "prepare-63">>\ <<include "rolling-upgrade-60">>\ <<endif>>\ <<endif>>\ Want to save this information? Copy and paste to your favorite editor.

!!First off, we need some information about your stack. \ What version of Elasticsearch are you currently running? <<radiobutton "$version" "2.x">> 2.x <<radiobutton "$version" "5.0-5.5">> 5.0-5.5 <<radiobutton "$version" "5.6">> 5.6 <<radiobutton "$version" "6.0-6.2">> 6.n \ Are you running in Elastic Cloud? <<radiobutton "$cloud" "yes">> Yes <<radiobutton "$cloud" "no">> No \ Which products are you using with Elasticsearch? <<checkbox "$kibana" false true>> Kibana <<checkbox "$logstash" false true>> Logstash <<checkbox "$beats" false true>> Beats <<checkbox "$hadoop" false true>> Elasticsearch for Hadoop <<link "Next">> <<if $version == "unknown" && $cloud == "unknown">> <<replace "#no_version">>You must select the version you are upgrading from. <</replace>> <<replace "#no_cloud">>You must select one of the options. <</replace>> <<elseif $cloud == "unknown" && $version !== "unknown">> <<replace "#no_cloud">>You must select one of the options. <</replace>> <<replace "#no_version">><</replace>> <<elseif $cloud !== "unknown" && $version == "unknown">> <<replace "#no_cloud">><</replace>> <<replace "#no_version">>You must select the version you are upgrading from.<</replace>> <<elseif $cloud == "yes">> <<if $version == "2.x">><<goto "v1-indices-cloud">>\ <<elseif $version != "6.0-6.2">><<goto "v2-indices-cloud">>\ <<else>><<goto "Upgrade list">> <<endif>>\ <<elseif $version == "2.x">><<set $upgrade_56 = "fcr"; $upgrade_60 = "rolling"; $v2indices = true>><<goto "SWM">>\ <<elseif $version == "5.0-5.5">><<goto "X-Pack?">> <<elseif $version == "5.6">><<set $upgrade_60 = "rolling">><<goto "X-Pack?">> <<elseif $version == "6.0-6.2">><<set $upgrade_63 = "rolling">><<goto "X-Pack?">> <<endif>> <</link>>

!!Which X-Pack features do you use? \ <<checkbox "$xpack_security" false true>> Security <<checkbox "$xpack_ml" false true>> Machine Learning <<checkbox "$xpack_watcher" false true>> Watcher/Alerting <<checkbox "$xpack_monitoring" false true>> Monitoring <<checkbox "$xpack_reporting" false true>> Reporting <<if $version != "6.0-6.2">>\ <<set $upgrade_assistant == true>>\ <<endif>>\ <<link "Next">> <<if $version != "6.0-6.2">>\ <<if $xpack_security>><<goto "Security">>\ <<else>><<goto "v2-indices">>\ <<endif>>\ <<else>>\ <<goto "Upgrade list">>\ <<endif>>\ <</link>>

!!Transport Layer Security (TLS) is required as of 6.0 \ TLS prevents unidentified nodes from joining a cluster and encrypts inter-node communications. Do you have TLS enabled? [[Yes|v2-indices][$enable_tls = "no"]]\ [[No|v2-indices][$enable_tls = "yes"; $upgrade_60 = "fcr"]]

!!!Upgrade to 5.6 [[Back up your data.|https://www.elastic.co/guide/en/elasticsearch/reference/5.6/modules-snapshots.html]] <<if $v1indices>>\ [[Reindex your Elasticsearch 1.x indices.|https://www.elastic.co/guide/en/elasticsearch/reference/5.6/reindex-upgrade.html]] You will not be able to read and reindex 1.x indices once you upgrade to 5.6.\ <<endif>>\ [[Use the Elasticsearch Migration plugin to check for upgrade issues.|https://github.com/elastic/elasticsearch-migration]] <<include "breaking-56">>\ <<if $hadoop>>\ [[Upgrade Elasticsearch for Hadoop to 5.6.|https://www.elastic.co/guide/en/elasticsearch/hadoop/5.6/install.html]] Elasticsearch Hadoop 5.6 is compatible with Elasticsearch 2.x and Elasticsearch 5.x.\ <<endif>>\ [[Upgrade Elasticsearch to 5.6.|https://www.elastic.co/guide/en/elasticsearch/reference/5.6/restart-upgrade.html]] You must perform a full cluster restart to upgrade to Elasticsearch 5.6:\ \ [[Install Elasticsearch 5.6.|https://www.elastic.co/guide/en/elasticsearch/reference/5.6/_installation.html]] <<if $xpack || (!$xpack && $upgrade_assistant)>>\ [[Install X-Pack.|https://www.elastic.co/guide/en/elasticsearch/reference/5.6/installing-xpack-es.html]] <<endif>>\ <<include enable-tls>>\ Restart your Elasticsearch cluster. \ <<include "components-56">>\

!!!Upgrade to 5.6 [[Back up your data.|https://www.elastic.co/guide/en/elasticsearch/reference/5.6/modules-snapshots.html]] [[Use the Elasticsearch Migration plugin to check for upgrade issues.|https://github.com/elastic/elasticsearch-migration]] <<include "breaking-56">>\ <<if $xpack_ml>>\ [[Stop any running Machine Learning jobs.|https://www.elastic.co/guide/en/x-pack/current/stopping-ml.html]] <<endif>>\ [[Upgrade one node at a time to Elasticsearch 5.6.|https://www.elastic.co/guide/en/elasticsearch/reference/5.6/rolling-upgrades.html]] \ Install Elasticsearch 5.6. <<if $xpack || $upgrade_assistant>>\ Install X-Pack. <<endif>>\ Restart the node. \ <<include components-56>>\

!!!Perform a Full Cluster Restart Upgrade to 6.x <<if $xpack_ml>>\ [[Stop any running Machine Learning jobs.|https://www.elastic.co/guide/en/x-pack/current/stopping-ml.html]] <<endif>>\ Stop sending data to your cluster. [[Shut down your cluster and install Elasticsearch 6.x on all nodes.|https://www.elastic.co/guide/en/elasticsearch/reference/6.7/restart-upgrade.html]] X-Pack is installed automatically with the default distribution from 6.3 onward. <<include enable-tls>>\ Restart your Elasticsearch cluster. <<include components-60>>

<<if $kibana>>\ [[Upgrade Kibana to 5.6.|https://www.elastic.co/guide/en/kibana/5.6/upgrade.html]] <<if $xpack || $upgrade_assistant>>\ [[Install X-Pack into Kibana.|https://www.elastic.co/guide/en/kibana/5.6/installing-xpack-kb.html]] <<endif>>\ Restart Kibana. <<endif>>\ <<if $logstash>>\ [[Upgrade Logstash to 5.6.|https://www.elastic.co/guide/en/logstash/5.6/upgrading-logstash.html]] <<if $xpack>>\ [[Install X-Pack into Logstash.|https://www.elastic.co/guide/en/logstash/5.6/installing-xpack-log.html]] <<endif>>\ Restart Logstash. <<endif>>\ <<if $beats>>\ [[Upgrade each beat to 5.6 and restart them.|https://www.elastic.co/guide/en/beats/libbeat/6.7/upgrading-5-to-6.html#upgrading-to-5.6]] <<endif>>\ <<if $hadoop && $upgrade_56 == "rolling">>\ [[Upgrade Elasticsearch for Hadoop to 5.6.|https://www.elastic.co/guide/en/elasticsearch/hadoop/5.6/install.html]]\ <<endif>>\

!!!Upgrade to 6.x <<if $xpack_ml>>\ [[Stop any running Machine Learning jobs.|https://www.elastic.co/guide/en/elastic-stack-overview/6.7/stopping-ml.html]] <<endif>>\ <<if $hadoop>>\ Disable ingestion from Hadoop while you upgrade the stack. [[Upgrade Elasticsearch for Hadoop to 6.x.|https://www.elastic.co/guide/en/elasticsearch/hadoop/6.7/install.html]] <<endif>>\ [[Upgrade Elasticsearch to 6.x.|https://www.elastic.co/guide/en/elasticsearch/reference/6.7/rolling-upgrades.html]] You can perform a rolling upgrade to Elasticsearch 6.x.\ <<if $upgrade_63 == "rolling">>\ <<include "components-63">>\ <<else>>\ <<include components-60>>\ <<endif>>

!!!Prepare to upgrade to 6.x [[Back up your data.|https://www.elastic.co/guide/en/elasticsearch/reference/6.7/modules-snapshots.html]] <<include breaking-60>>\ <<if $xpack_security>>\ [[If you're using the default changeme password, change your password.|https://www.elastic.co/guide/en/x-pack/5.6/setting-up-authentication.html#reset-built-in-user-passwords]] <<endif>>\ <<if $upgrade_assistant>>\ <<if !$xpack>>\ [[Install X-Pack into Elasticsearch and Kibana.|https://www.elastic.co/guide/en/x-pack/5.6/installing-xpack.html]] To use X-Pack just to upgrade, disable Security by setting {{{xpack.security.enabled}}} to false.\ Restart Elasticsearch and Kibana. <<endif>>\ [[Use the Upgrade Assistant to:|https://www.elastic.co/guide/en/kibana/5.6/xpack-upgrade-assistant.html]] \ Identify issues you need to address before upgrading to 6.x. <<if $kibana >>\ Upgrade the internal Kibana index. <<endif>>\ <<if $xpack>>\ Upgrade the internal X-Pack indices. <<endif>>\ <<if $v2indices>>\ Upgrade your 2.x indices. <<endif>>\ \ <<else>>\ [[Check the Elasticsearch deprecation log.|https://www.elastic.co/guide/en/elasticsearch/reference/5.6/settings.html#deprecation-logging]] <<if $v2indices>>\ [[Reindex your 2.x indices.|https://www.elastic.co/guide/en/elasticsearch/reference/6.7/reindex-upgrade.html]] You will not be able to read indices created on 2.x after you upgrade to 6.x.\ <<endif>>\ <<endif>>\

<<if $xpack_security && $upgrade_60 == "fcr">>\ [[Create a temporary super user and upgrade the internal .security index.|https://www.elastic.co/guide/en/elastic-stack/6.7/upgrading-elastic-stack.html#upgrade-internal-indices]] [[Delete the temporary super user.|https://www.elastic.co/guide/en/elasticsearch/reference/6.7/users-command.html]] <<endif>>\ <<if $kibana && $upgrade_60 == "fcr">>\ [[Upgrade the internal .kibana index.|https://www.elastic.co/guide/en/elastic-stack/6.7/upgrading-elastic-stack.html#upgrade-internal-indices]] <<endif>>\ <<if $kibana>>\ [[Upgrade Kibana to 6.x.|https://www.elastic.co/guide/en/kibana/6.7/upgrade.html]] <<if $xpack && $upgrade_60 == "fcr">>\ [[Use the Kibana Upgrade assistant to upgrade the internal X-Pack indices.|https://www.elastic.co/guide/en/kibana/6.7/xpack-upgrade-assistant.html#reindex-helper]] <<endif>>\ <<endif>>\ <<if $logstash>>\ [[Upgrade Logstash to 6.x.|https://www.elastic.co/guide/en/logstash/5.6/upgrading-logstash.html]] You can run Logstash 5.6 with Elasticsearch 6.x until it's convenient to upgrade.\ <<if $xpack>>\ [[Install X-Pack into Logstash.|https://www.elastic.co/guide/en/logstash/5.6/installing-xpack-log.html]] <<endif>>\ <<endif>>\ <<if $beats>>\ [[Upgrade each beat to 6.x.|https://www.elastic.co/guide/en/beats/libbeat/6.7/upgrading-5-to-6.html#upgrading-to-5.6]] You can run Beats 5.6 with Elasticsearch 6.x until it's convenient to upgrade. <<endif>>\ <<if $hadoop && $upgrade_60 == "fcr">>\ [[Upgrade Elasticsearch for Hadoop to 6.x.|https://www.elastic.co/guide/en/elasticsearch/hadoop/6.7/install.html]] <<endif>>

!!Here you go! <<if $version != "6.0-6.2">> These are the steps you need to take to upgrade to 6.x. On Elastic Cloud, all major configuration changes to upgrade Elasticsearch, Kibana, and X-Pack are handled for you. <<if $version != "5.6">>\ !!!Upgrade to 5.6 <<if $v1indices>>\ [[Reindex any 1.x indices that you need to carry forward.|https://www.elastic.co/guide/en/elasticsearch/reference/5.6/reindex-upgrade.html]] <<endif>>\ <<include breaking-56>>\ <<if $version == "2.x">>\ [[Upgrade your cluster to 5.6.|https://www.elastic.co/guide/en/cloud/current/ec-upgrading-v5.html]] You must perform a full-cluster restart to upgrade from 2.x to 5.6. <<elseif $version == "5.0-5.5">>\ [[Upgrade your cluster to 5.6.|https://www.elastic.co/guide/en/cloud/current/ec-upgrading-v5.html]] You can perform a rolling upgrade to 5.6. <<endif>>\ <<endif>>\ !!!Prepare to upgrade to 6.x [[Reindex any 2.x indices that you need to carry forward.|https://www.elastic.co/guide/en/elasticsearch/reference/6.7/reindex-upgrade.html]] <<include breaking-60>> !!!Upgrade to 6.x [[Perform a full cluster restart upgrade to 6.x.|https://www.elastic.co/guide/en/cloud/saas-release/ec-upgrading-v6.html]] This is required to enable Transport Layer Security (TLS). TLS prevents unidentified nodes from joining a cluster and encrypts inter-node communications.\ <<else>>\ [[Perform a one-click rolling upgrade to 6.x|https://www.elastic.co/guide/en/cloud/current/ec-upgrading.html]] On Elastic Cloud, all major configuration changes to upgrade Elasticsearch, Kibana, and X-Pack features are handled for you.\ <<endif>>\ <<if $logstash>>\ [[Upgrade Logstash to 6.x.|https://www.elastic.co/guide/en/logstash/6.7/upgrading-logstash.html]] You can run Logstash 5.6 with Elasticsearch 6.x until it's convenient to upgrade.\ <<endif>>\ <<if $beats>>\ [[Upgrade each beat to 6.x.|https://www.elastic.co/guide/en/beats/libbeat/6.7/upgrading-minor-versions.html]] You can run Beats 5.6 with Elasticsearch 6.x until it's convenient to upgrade.\ <<endif>>\

!!Do you use X-Pack? \ X-Pack provides security, alerting, monitoring, reporting, graph, and machine learning capabilities for the Elastic Stack. [[Yes|X-Pack features][$xpack = true]]\ [[No|v2-indices][$xpack = false]]\

!!Do you use any of Elastic's licensed products? \ Select the products you use: <<checkbox "$shield" false true>>Shield <<checkbox "$watcher" false true>>Watcher <<checkbox "$marvel" false true>>Marvel <<checkbox "$reporting" false true>>Reporting <<link Next>>\ <<if $shield || $watcher || $marvel || $reporting>>\ <<set $xpack = true;$upgrade_assistant = true>>\ <<endif>>\ <<if $shield>><<goto security-2x>>\ <<else>><<goto v1-indices>>\ <<endif>>\ <</link>\

!!Do you have any indices that were created in 2.x? \ To see the version of each index, use the GET settings API: {{{curl -XGET 'http://localhost:9200/_settings/index.version.created'}}} \ If an index's version starts with 20, you must reindex it to carry it forward as you upgrade. <<link "Yes">> <<set $v2indices = true>> <<if $version == "5.0-5.5">> <<goto "5.x upgrade">> <<elseif $version == "5.6" && $xpack == false>> <<goto "Upgrade assistant">> <<else>> <<goto "Upgrade list">> <<endif>> <</link>> I have 2.x indices that I need to carry forward <<link "No">> <<set $v2indices = false>> <<if $version == "5.0-5.5">> <<goto "5.x upgrade">> <<elseif $version == "5.6" && $xpack == false>> <<goto "Upgrade assistant">> <<else>> <<goto "Upgrade list">> <<endif>> <</link>> I don't have 2.x indices to carry forward

!!Do you have any indices that were created in 1.x? \ To see the version of each index, use the GET settings API: {{{curl -XGET 'http://localhost:9200/_settings/index.version.created'}}} \ If an index's version starts with 10, you must reindex it to carry it forward as you upgrade. <<link "Yes">> <<set $v1indices = true>> <<if $xpack == false>> <<goto "Upgrade assistant">> <<else>> <<set $upgrade_assistant == true>> <<goto "Upgrade list">> <<endif>> <</link>> I have 1.x indices that I need to carry forward <<link "No">> <<set $v1indices = false>> <<if $xpack == false>> <<goto "Upgrade assistant">> <<else>> <<goto "Upgrade list">> <<endif>> <</link>> I don't have 1.x indices to carry forward

!!Do you have any indices that were created in 1.x? \ To see the version of each index, use the GET settings API: {{{curl -XGET 'http://localhost:9200/_settings/index.version.created'}}} \ If an index's version starts with 10, you must reindex it to carry it forward as you upgrade. <<link "Yes">> <<set $v1indices = true; $v2indices = true>> <<goto "Upgrade list">> <</link>> I have 1.x indices that I need to carry forward <<link "No">> <<set $v1indices = false>> <<goto "Upgrade list">> <</link>> I don't have 1.x indices to carry forward

Address any 5.x breaking changes that affect your applications: \ [[Elasticsearch breaking changes|https://www.elastic.co/guide/en/elasticsearch/reference/5.6/breaking-changes.html]] <<if $beats>>\ [[Beats breaking changes|https://www.elastic.co/guide/en/beats/libbeat/5.6/breaking-changes.html]] <<endif>>\ <<if $logstash>>\ [[Logstash breaking changes|https://www.elastic.co/guide/en/logstash/5.6/breaking-changes.html]] <<endif>>\ <<if $kibana>>\ [[Kibana breaking changes|https://www.elastic.co/guide/en/kibana/5.6/breaking-changes.html]]\ <<endif>>\

Address any 6.0 breaking changes that affect your applications: \ [[Elasticsearch breaking changes|https://www.elastic.co/guide/en/elasticsearch/reference/6.0/breaking-changes.html]] <<if $beats>>\ [[Beats breaking changes|https://www.elastic.co/guide/en/beats/libbeat/6.0/breaking-changes-6.0.html]] <<endif>>\ <<if $logstash>>\ [[Logstash breaking changes|https://www.elastic.co/guide/en/logstash/6.0/breaking-changes.html]] <<endif>>\ <<if $kibana>>\ [[Kibana breaking changes|https://www.elastic.co/guide/en/kibana/6.0/breaking-changes-6.0.html]]\ <<endif>>\

!!Transport Layer Security (TLS) is required as of 6.0 \ TLS prevents unidentified nodes from joining a cluster and encrypts inter-node communications. Do you have TLS enabled? [[Yes|v1-indices][$enable_tls = "no"]]\ [[No|v1-indices][$enable_tls = "yes"]]

<<if $enable_tls == "yes">>\ [[Enable Transport Layer Security (TLS) to secure cluster communications.|https://www.elastic.co/guide/en/x-pack/current/ssl-tls.html]] TLS is required as of 6.0 to prevent unidentified nodes from joining a cluster and encrypt inter-node communications.\ <<endif>>\

<<if $kibana>>\ [[Upgrade Kibana to 6.x.|https://www.elastic.co/guide/en/kibana/6.7/upgrade.html]] <<endif>>\ <<if $logstash>>\ [[Upgrade Logstash to 6.x.|https://www.elastic.co/guide/en/logstash/6.7/upgrading-logstash.html]] You can run Logstash with Elasticsearch 6.x until it's convenient to upgrade.\ <<endif>>\ <<if $beats>>\ [[Upgrade each beat to 6.x.|https://www.elastic.co/guide/en/beats/libbeat/6.7/upgrading-minor-versions.html]] You can run Beats with Elasticsearch 6.x until it's convenient to upgrade.\ <<endif>>\

!!!Prepare to upgrade to 6.x [[Back up your data.|https://www.elastic.co/guide/en/elasticsearch/reference/6.7/modules-snapshots.html]] [[Check the Elasticsearch deprecation log.|https://www.elastic.co/guide/en/elasticsearch/reference/6.7/settings.html#deprecation-logging]]