Kafka Eagle
Search…
2.Install

1.Download Kafka Eagle

2.Extract Kafka Eagle

Here we extract to /data/soft/new directory and extract, as shown below:
1
tar -zxvf kafka-eagle-${version}-bin.tar.gz
Copied!
If you have installed the version before, delete the modified version, and rename the current version, as shown below:
1
rm -rf kafka-eagle
2
mv kafka-eagle-${version} kafka-eagle
Copied!

3.Configure Kafka Eagle profile

1
vi /etc/profile
2
3
export KE_HOME=/data/soft/new/kafka-eagle
4
export PATH=$PATH:$KE_HOME/bin
Copied!
Then, we use the . /etc/profile to enable the configuration to take effect immediately.

4.Configure Kafka Eagle system-config file

1
cd ${KE_HOME}/conf
2
vi system-config.properties
3
4
# Multi zookeeper&kafka cluster list -- The client connection address of the Zookeeper cluster is set here
5
kafka.eagle.zk.cluster.alias=cluster1,cluster2
6
cluster1.zk.list=tdn1:2181,tdn2:2181,tdn3:2181
7
cluster2.zk.list=xdn1:2181,xdn2:2181,xdn3:2181
8
9
# Kafka broker nodes online list
10
cluster1.kafka.eagle.broker.size=10
11
cluster2.kafka.eagle.broker.size=20
12
13
# Zkcli limit -- Zookeeper cluster allows the number of clients to connect to
14
kafka.zk.limit.size=25
15
16
# Kafka Eagle webui port -- WebConsole port access address
17
kafka.eagle.webui.port=8048
18
19
# Kafka offset storage -- Offset stored in a Kafka cluster, if stored in the zookeeper, you can not use this option
20
cluster1.kafka.eagle.offset.storage=kafka
21
cluster2.kafka.eagle.offset.storage=kafka
22
23
# Whether the Kafka performance monitoring diagram is enabled
24
kafka.eagle.metrics.charts=false
25
26
# Kafka Eagle keeps data for 30 days by default
27
kafka.eagle.metrics.retain=30
28
29
# If offset is out of range occurs, enable this property -- Only suitable for kafka sql
30
kafka.eagle.sql.fix.error=false
31
kafka.eagle.sql.topic.records.max=5000
32
33
# Delete kafka topic token -- Set to delete the topic token, so that administrators can have the right to delete
34
kafka.eagle.topic.token=keadmin
35
36
# Kafka sasl authenticate
37
cluster1.kafka.eagle.sasl.enable=false
38
cluster1.kafka.eagle.sasl.protocol=SASL_PLAINTEXT
39
cluster1.kafka.eagle.sasl.mechanism=SCRAM-SHA-256
40
cluster1.kafka.eagle.sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="admin" password="admin-secret";
41
# If not set, the value can be empty
42
cluster1.kafka.eagle.sasl.client.id=
43
# Add kafka cluster cgroups
44
cluster1.kafka.eagle.sasl.cgroup.enable=false
45
cluster1.kafka.eagle.sasl.cgroup.topics=kafka_ads01,kafka_ads02
46
47
cluster2.kafka.eagle.sasl.enable=true
48
cluster2.kafka.eagle.sasl.protocol=SASL_PLAINTEXT
49
cluster2.kafka.eagle.sasl.mechanism=PLAIN
50
cluster2.kafka.eagle.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="admin-secret";
51
cluster2.kafka.eagle.sasl.client.id=
52
cluster2.kafka.eagle.sasl.cgroup.enable=false
53
cluster2.kafka.eagle.sasl.cgroup.topics=kafka_ads03,kafka_ads04
54
55
# Default use sqlite to store data
56
kafka.eagle.driver=org.sqlite.JDBC
57
# It is important to note that the '/hadoop/kafka-eagle/db' path must exist.
58
kafka.eagle.url=jdbc:sqlite:/hadoop/kafka-eagle/db/ke.db
59
kafka.eagle.username=root
60
kafka.eagle.password=smartloli
61
62
# <Optional> set mysql address
63
#kafka.eagle.driver=com.mysql.jdbc.Driver
64
#kafka.eagle.url=jdbc:mysql://127.0.0.1:3306/ke?useUnicode=true&characterEncoding=UTF-8&zeroDateTimeBehavior=convertToNull
65
#kafka.eagle.username=root
66
#kafka.eagle.password=smartloli
Copied!

5.Start Kafka Eagle

1
cd ${KE_HOME}/bin
2
chmod +x ke.sh
3
./ke.sh start
Copied!
As shown in the following figure:
ke_bash
Last modified 1yr ago