在此选用elasticsearch-5.6.16.tar.gz方式安装,Elasticsearch安装在5.0版本以后必须依赖jdk1.8之上的java版本
安装JDK1.8
下载链接:
上传到机器[root@CentOS7_node1 src]# tar xf jdk-8u161-linux-x64.tar.gz -C /usr/local/[root@CentOS7_node1 src]# mv /usr/local/jdk1.8.0_161 /usr/local/java[root@CentOS7_node1 src]# tail -5 /etc/profile#################JAVA#################export JAVA_HOME=/usr/local/javaexport JRE_HOME=/usr/local/java/jreexport CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/libexport PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH[root@CentOS7_node1 src]# source /etc/profile[root@CentOS7_node1 src]# java -versionjava version "1.8.0_161"Java(TM) SE Runtime Environment (build 1.8.0_161-b12)Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)
安装Elasticsearch
下载链接:
上传到机器[root@CentOS7_node1 src]# tar xf elasticsearch-5.6.16.tar.gz -C /usr/local/[root@CentOS7_node1 src]# ls /usr/local/elasticsearch-5.6.16/bin config data lib LICENSE.txt logs modules NOTICE.txt plugins README.textile
- bin目录:二进制脚本启动es程序文件以及启动es插件目录
- config目录:elasticsearch配置文件
- data目录:默认Elasticsearch生成的索引/切片数据文件存放目录,可以指定多个位置来存储数据
- lib目录:一些开发的jar包
- logs目录:elasticsearch日志目录
- modules目录:模块目录,详情看官网链接:
- plugins目录:elasticsearch插件目录,此版本tar包装后默认无插件
bin目录文件介绍
[root@CentOS7_node1 elasticsearch-5.6.16]# tree bin/bin/├── elasticsearch #Elasticsearch启动脚本├── elasticsearch.bat #Win启动脚本,tar包也是可以在Win运行的├── elasticsearch.in.bat├── elasticsearch.in.sh├── elasticsearch-keystore├── elasticsearch-keystore.bat├── elasticsearch-plugin #集成插件启动脚本 ├── elasticsearch-plugin.bat├── elasticsearch-service.bat├── elasticsearch-service-mgr.exe├── elasticsearch-service-x64.exe├── elasticsearch-service-x86.exe├── elasticsearch-systemd-pre-exec├── elasticsearch-translog└── elasticsearch-translog.bat0 directories, 15 files
config目录文件介绍
[root@CentOS7_node1 elasticsearch-5.6.16]# tree config/config/├── elasticsearch.yml #ES节点集群等相关配置├── jvm.options #JVM堆内存及GC相关配置├── log4j2.properties #log4j2框架日志输出相关配置└── scripts #脚本存放目录,默认无脚本1 directory, 3 files
logs目录文件介绍
log目录默认是没有的,需要你正常启动elasticsearch后,会在你的es目录下自动生成一个log目录[root@CentOS7_node1 elasticsearch-5.6.16]# tree logs/logs/├── elasticsearch_deprecation.log #ES描述日志├── elasticsearch_index_indexing_slowlog.log #索引日志├── elasticsearch_index_search_slowlog.log #搜索日志└── elasticsearch.log #集群名称会生成一个以集群名称命名的日志 0 directories, 4 files
创建elasticsearch所用到的用户及组
elasticsearch默认是不能以root身份去运行的,否则启动会报错误信息为“can not run elasticsearch as root Elasticsearch”,所以我们要创建一个普通用户来管理elasticsearch
创建elasticsearch用户及组[root@CentOS7_node1 /]# groupadd elasticsearch [root@CentOS7_node1 /]# useradd elasticsearch -g elasticsearch[root@CentOS7_node1 /]# echo '3edc#EDC' | passwd --stdin elasticsearchChanging password for user elasticsearch.passwd: all authentication tokens updated successfully.
更改Elasticsearch目录属性信息
[root@CentOS7_node1 /]# chown -Rf elasticsearch:elasticsearch /usr/local/elasticsearch-5.6.16[root@CentOS7_node1 /]# ll /usr/local/elasticsearch-5.6.16total 240drwxr-xr-x. 2 elasticsearch elasticsearch 4096 May 5 14:57 bindrwxr-xr-x. 2 elasticsearch elasticsearch 4096 May 5 15:11 configdrwxr-xr-x. 2 elasticsearch elasticsearch 4096 Mar 13 23:34 lib-rw-r--r--. 1 elasticsearch elasticsearch 11358 Mar 13 23:31 LICENSE.txtdrwxr-xr-x. 2 elasticsearch elasticsearch 4096 May 5 14:02 logsdrwxr-xr-x. 13 elasticsearch elasticsearch 4096 Mar 13 23:35 modules-rw-r--r--. 1 elasticsearch elasticsearch 194187 Mar 13 23:34 NOTICE.txtdrwxr-xr-x. 2 elasticsearch elasticsearch 4096 Mar 13 23:34 plugins-rw-r--r--. 1 elasticsearch elasticsearch 9491 Mar 13 23:31 README.textile
启动elasticsearch
切换到elasticsearch用户来启动ES[root@CentOS7_node1 ~]# su elasticsearch[elasticsearch@CentOS7_node1 root]$ /usr/local/elasticsearch-5.6.16/bin/elasticsearch -h默认情况下,Elasticsearch在前台运行,将其日志打印到标准输出(stdout),然后按下Ctrl-C即可停止starts elasticsearchOption Description ------ ----------- -EConfigure a setting #ES启动时可以指定配置信息,是以Key Value的形式来配置 #-E选项实例:./bin/elasticsearch -d -Ecluster.name=my_cluster -Enode.name=node_1#-d 表示后台启动ES,-Ecluster.name指定ES的集群名称,node.name指定节点名称,这两参数在ES的config/elasticsearch.yml中都有设置-V, --version Prints elasticsearch version information and exits #显示ES的版本信息-d, --daemonize Starts Elasticsearch in the background #-d选项改为以后台守护进程方式启动-h, --help show help -p, --pidfile Creates a pid file in the specified path on start #指定ES的PID文件路径 -q, --quiet Turns off standard ouput/error streams logging in console #关闭控制台中的标准输出/错误流日志记录-s, --silent show minimal output #精简显示启动信息 -v, --verbose show verbose output #详细显示启动信息
[elasticsearch@CentOS7_node1 /]$ /usr/local/elasticsearch-5.6.16/bin/elasticsearch [2019-05-07T23:39:22,140][INFO ][o.e.n.Node ] [] initializing ...[2019-05-07T23:39:22,229][INFO ][o.e.e.NodeEnvironment ] [lPEqF6-] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [18gb], net total_space [24.5gb], spins? [unknown], types [rootfs][2019-05-07T23:39:22,229][INFO ][o.e.e.NodeEnvironment ] [lPEqF6-] heap size [1.9gb], compressed ordinary object pointers [true][2019-05-07T23:39:22,230][INFO ][o.e.n.Node ] node name [lPEqF6-] derived from node ID [lPEqF6-7TaClsuN_NVm9NA]; set [node.name] to override[2019-05-07T23:39:22,230][INFO ][o.e.n.Node ] version[5.6.16], pid[17264], build[3a740d1/2019-03-13T15:33:36.565Z], OS[Linux/3.10.0-957.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_161/25.161-b12][2019-05-07T23:39:22,231][INFO ][o.e.n.Node ] JVM arguments [-Xms2g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djdk.io.permissionsUseCanonicalPath=true, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j.skipJansi=true, -XX:+HeapDumpOnOutOfMemoryError, -Des.path.home=/usr/local/elasticsearch-5.6.16][2019-05-07T23:39:22,900][INFO ][o.e.p.PluginsService ] [lPEqF6-] loaded module [aggs-matrix-stats][2019-05-07T23:39:22,900][INFO ][o.e.p.PluginsService ] [lPEqF6-] loaded module [ingest-common][2019-05-07T23:39:22,901][INFO ][o.e.p.PluginsService ] [lPEqF6-] loaded module [lang-expression][2019-05-07T23:39:22,901][INFO ][o.e.p.PluginsService ] [lPEqF6-] loaded module [lang-groovy][2019-05-07T23:39:22,901][INFO ][o.e.p.PluginsService ] [lPEqF6-] loaded module [lang-mustache][2019-05-07T23:39:22,901][INFO ][o.e.p.PluginsService ] [lPEqF6-] loaded module [lang-painless][2019-05-07T23:39:22,901][INFO ][o.e.p.PluginsService ] [lPEqF6-] loaded module [parent-join][2019-05-07T23:39:22,901][INFO ][o.e.p.PluginsService ] [lPEqF6-] loaded module [percolator][2019-05-07T23:39:22,901][INFO ][o.e.p.PluginsService ] [lPEqF6-] loaded module [reindex][2019-05-07T23:39:22,901][INFO ][o.e.p.PluginsService ] [lPEqF6-] loaded module [transport-netty3][2019-05-07T23:39:22,901][INFO ][o.e.p.PluginsService ] [lPEqF6-] loaded module [transport-netty4][2019-05-07T23:39:22,902][INFO ][o.e.p.PluginsService ] [lPEqF6-] no plugins loaded[2019-05-07T23:39:24,457][INFO ][o.e.d.DiscoveryModule ] [lPEqF6-] using discovery type [zen][2019-05-07T23:39:24,935][INFO ][o.e.n.Node ] initialized[2019-05-07T23:39:24,936][INFO ][o.e.n.Node ] [lPEqF6-] starting ...[2019-05-07T23:39:27,518][INFO ][o.e.t.TransportService ] [lPEqF6-] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}[2019-05-07T23:39:27,529][WARN ][o.e.b.BootstrapChecks ] [lPEqF6-] max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536][2019-05-07T23:39:27,529][WARN ][o.e.b.BootstrapChecks ] [lPEqF6-] max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144][2019-05-07T23:39:30,589][INFO ][o.e.c.s.ClusterService ] [lPEqF6-] new_master {lPEqF6-}{lPEqF6-7TaClsuN_NVm9NA}{O8BQXY-3Q1O9nrC34rDU5w}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)[2019-05-07T23:39:30,622][INFO ][o.e.h.n.Netty4HttpServerTransport] [lPEqF6-] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}[2019-05-07T23:39:30,622][INFO ][o.e.n.Node ] [lPEqF6-] started[2019-05-07T23:39:30,635][INFO ][o.e.g.GatewayService ] [lPEqF6-] recovered [0] indices into cluster_state
以上告警有WARN警告信息,我们一定要注意下,并解决此问题,这涉及到了ES的两种模式,分别为:开发者模式和生产模式
开发者模式可生产模式
默认情况下,Elasticsearch假定您正在开发模式下工作。如果未正确配置上述任何设置,则会向日志文件写入警告,但您将能够启动并运行Elasticsearch节点。
一旦配置了网络设置network.host(此参数在elasticsearch.yml文件中配置),Elasticsearch就会假定您正在转向生产并将上述警告升级为异常。这些异常将阻止您的Elasticsearch节点启动。这是一项重要的安全措施,可确保您不会因服务器配置错误而丢失数据。 为了更让我们的测试环境贴合生产模式,我们坚决不允许它出半点问题启动失败问题
1.文件描述符过低
[2019-05-07T23:39:27,529][WARN ][o.e.b.BootstrapChecks ] [lPEqF6-] max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]解决:
修改文件描述符数量echo '* soft nofile 65536' >> /etc/security/limits.confecho '* hard nofile 65536' >> /etc/security/limits.conf然后重新登录终端[root@CentOS7_node1 ~]# ulimit -n65536
2.max_map_count值为65530]太低,至少增加到262144
[2019-05-07T23:39:27,529][WARN ][o.e.b.BootstrapChecks ] [lPEqF6-] max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144] 在linux系统上,elasticsearch默认使用hybrid mmapfs / niofs来存储索引文件,因此操作系统主要会通过mmap来限制存储的空间,因此如果存储空间满了,那么会抛出异常,我们可以使用如下命令来更改设置。解决:
修改max_map_count值临时设置:[root@CentOS7_node1 ~]# sysctl -w vm.max_map_count=655360vm.max_map_count = 655360永久设置:[root@CentOS7_node1 ~]# echo 'vm.max_map_count=655360' >> /etc/sysctl.conf [root@CentOS7_node1 ~]# sysctl -p vm.max_map_count = 655360
3.这个问题我们现在虽然是没遇见,但是建议先把坑填上,不然还是会出现的
在ES启动的时候警告信息为:“Unable to lock JVM Memory: error=12, reason=Cannot allocate memory”,无法分配内存的意思解决:
[root@CentOS7_node1 ~]# echo '* soft memlock unlimited' >> /etc/security/limits.conf[root@CentOS7_node1 ~]# echo '* hard memlock unlimited' >> /etc/security/limits.conf[root@CentOS7_node1 ~]# tail -2 /etc/security/limits.conf* soft memlock unlimited* hard memlock unlimited
再次启动
首先要退出终端,然后重新登录,让elasticsearch用户能够重新加载我们刚修改的配置[root@CentOS7_node1 ~]# su elasticsearch[elasticsearch@CentOS7_node1 root]$ ulimit -n65536[elasticsearch@CentOS7_node1 root]$ /usr/local/elasticsearch-5.6.16/bin/elasticsearch [2019-05-08T00:00:43,188][INFO ][o.e.n.Node ] [] initializing ...[2019-05-08T00:00:43,265][INFO ][o.e.e.NodeEnvironment ] [lPEqF6-] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [18.1gb], net total_space [24.5gb], spins? [unknown], types [rootfs][2019-05-08T00:00:43,265][INFO ][o.e.e.NodeEnvironment ] [lPEqF6-] heap size [1.9gb], compressed ordinary object pointers [true][2019-05-08T00:00:43,266][INFO ][o.e.n.Node ] node name [lPEqF6-] derived from node ID [lPEqF6-7TaClsuN_NVm9NA]; set [node.name] to override[2019-05-08T00:00:43,266][INFO ][o.e.n.Node ] version[5.6.16], pid[21842], build[3a740d1/2019-03-13T15:33:36.565Z], OS[Linux/3.10.0-957.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_161/25.161-b12][2019-05-08T00:00:43,266][INFO ][o.e.n.Node ] JVM arguments [-Xms2g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djdk.io.permissionsUseCanonicalPath=true, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j.skipJansi=true, -XX:+HeapDumpOnOutOfMemoryError, -Des.path.home=/usr/local/elasticsearch-5.6.16][2019-05-08T00:00:44,000][INFO ][o.e.p.PluginsService ] [lPEqF6-] loaded module [aggs-matrix-stats][2019-05-08T00:00:44,000][INFO ][o.e.p.PluginsService ] [lPEqF6-] loaded module [ingest-common][2019-05-08T00:00:44,000][INFO ][o.e.p.PluginsService ] [lPEqF6-] loaded module [lang-expression][2019-05-08T00:00:44,000][INFO ][o.e.p.PluginsService ] [lPEqF6-] loaded module [lang-groovy][2019-05-08T00:00:44,000][INFO ][o.e.p.PluginsService ] [lPEqF6-] loaded module [lang-mustache][2019-05-08T00:00:44,001][INFO ][o.e.p.PluginsService ] [lPEqF6-] loaded module [lang-painless][2019-05-08T00:00:44,001][INFO ][o.e.p.PluginsService ] [lPEqF6-] loaded module [parent-join][2019-05-08T00:00:44,001][INFO ][o.e.p.PluginsService ] [lPEqF6-] loaded module [percolator][2019-05-08T00:00:44,001][INFO ][o.e.p.PluginsService ] [lPEqF6-] loaded module [reindex][2019-05-08T00:00:44,001][INFO ][o.e.p.PluginsService ] [lPEqF6-] loaded module [transport-netty3][2019-05-08T00:00:44,001][INFO ][o.e.p.PluginsService ] [lPEqF6-] loaded module [transport-netty4][2019-05-08T00:00:44,001][INFO ][o.e.p.PluginsService ] [lPEqF6-] no plugins loaded[2019-05-08T00:00:45,824][INFO ][o.e.d.DiscoveryModule ] [lPEqF6-] using discovery type [zen][2019-05-08T00:00:46,288][INFO ][o.e.n.Node ] initialized[2019-05-08T00:00:46,288][INFO ][o.e.n.Node ] [lPEqF6-] starting ...[2019-05-08T00:00:48,555][INFO ][o.e.t.TransportService ] [lPEqF6-] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}[2019-05-08T00:00:51,637][INFO ][o.e.c.s.ClusterService ] [lPEqF6-] new_master {lPEqF6-}{lPEqF6-7TaClsuN_NVm9NA}{QRumAuUXQpW14l6cEgBUGA}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)[2019-05-08T00:00:51,672][INFO ][o.e.h.n.Netty4HttpServerTransport] [lPEqF6-] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}[2019-05-08T00:00:51,672][INFO ][o.e.n.Node ] [lPEqF6-] started[2019-05-08T00:00:51,692][INFO ][o.e.g.GatewayService ] [lPEqF6-] recovered [0] indices into cluster_state
ES端口详解
再次打开一个终端
[root@CentOS7_node1 ~]# ps -ef|grep javaelastic+ 21842 21777 14 00:00 pts/1 00:00:09 /usr/local/java/bin/java -Xms2g -Xmx2g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -server -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -Djdk.io.permissionsUseCanonicalPath=true -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Dlog4j.skipJansi=true -XX:+HeapDumpOnOutOfMemoryError -Des.path.home=/usr/local/elasticsearch-5.6.16 -cp /usr/local/elasticsearch-5.6.16/lib/* org.elasticsearch.bootstrap.Elasticsearchroot 22132 20125 0 00:01 pts/2 00:00:00 grep --color=auto java[root@CentOS7_node1 ~]# ss -anplt | grep javaLISTEN 0 128 ::ffff:127.0.0.1:9200 :::* users:(("java",pid=21842,fd=135))LISTEN 0 128 ::1:9200 :::* users:(("java",pid=21842,fd=134))LISTEN 0 128 ::ffff:127.0.0.1:9300 :::* users:(("java",pid=21842,fd=121))LISTEN 0 128 ::1:9300 :::* users:(("java",pid=21842,fd=119))
查看java进程可以看到我们的elasticsearch是以elasticsearch用户所启动
查看端口看到ES默认启用两个端口,分别是TCP端口9200及TCP端口9300,但是只监听在了127.0.0.1的地址上,如需修改监听地址,我们可以通过config/elasticsearch.yml文件来修改。 9200端口是为搜索服务所提供的端口。 9300是ES集群中通信端口,根据9300端口来判定集群各节点的存活性,此端口采用单播的形式进行集群通信。测试ES是否启动成功
打开一个终端,访问本地9200端口,查看返回信息
[elasticsearch@CentOS7_node1 root]$ curl http://127.0.0.1:9200 { "name" : "mDAWBTb", #本节点名称 "cluster_name" : "elasticsearch", #集群名称 "cluster_uuid" : "DEXGYpVzRhOnXcmykCtJTQ",#集群UID "version" : { #以下为版本信息 "number" : "5.6.16", #ES版本 "build_hash" : "3a740d1", #ES默认会为各个节点分片一个UID标识符,具有唯一性 "build_date" : "2019-03-13T15:33:36.565Z",#es数据构建时间 "build_snapshot" : false, #是否构建快照 "lucene_version" : "6.6.1" #Apache Lucene检索工具库版本 }, "tagline" : "You Know, for Search"}