1. 하둡의 설정파일 위치( Configuration file path of Hadoop )
$HADOOP_HOME/etc/hadoop
2. 핵심 설정 파일( Core configuration files )
– /etc/hosts
– core-site.xml
– hdfs-site.xml
– mapred-site.xml
– yarn-site.xml
– slaves
3. /etc/hosts
1 2 3 4 5 6 7 8 | xxx.xxx.xxx.xxx namenodexxx.xxx.xxx.xxx secondarynamenodexxx.xxx.xxx.xxx datanode01xxx.xxx.xxx.xxx datanode02xxx.xxx.xxx.xxx datanode03............xxx.xxx.xxx.xxx datanode0N |
4. core-site.xml
1 2 3 4 5 6 7 8 9 10 | <configuration><property> <name>fs.defaultFS</name> <value>hdfs://namenode:9000</value></property><property> <name>hadoop.tmp.dir</name> <value>/data/hadoop/tmp</value></property></configuration> |
5. hdfs-site.xml
$HADOOP_HOME 변수를 치환해야 합니다.(You have to replace the $HADOOP_HOME value.)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 | <configuration> <property> <name>dfs.replication</name> <value>3</value> </property> <property> <name>dfs.webhdfs.enabled</name> <value>true</value> </property> <property> <name>dfs.secondary.http.address</name> <value>secondarynamenode:50090</value> </property> <property> <name>dfs.hosts.exclude</name> <value>$HADOOP_HOME/etc/hadoop/exclude</value> </property> <property> <name>dfs.hosts</name> <value>$HADOOP_HOME/etc/hadoop/include</value> </property> <property> <name>dfs.namenode.name.dir</name> <value>/data/hadoop/dfs/name</value> </property> <property> <name>dfs.datanode.data.dir</name> </property> <property> <name>dfs.namenode.checkpoint.dir</name> <value>/data/hadoop/dfs/namesecondary</value> </property> <property> <name>dfs.datanode.hdfs-blocks-metadata.enabled</name> <value>true</value> </property></configuration> |
6. mapred-site.xml
$HADOOP_HOME 변수를 치환해야 합니다.(You have to replace the $HADOOP_HOME value.)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | <configuration> <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> <property> <name>mapreduce.jobtracker.hosts.exclude.filename</name> <value>$HADOOP_HOME/etc/hadoop/exclude</value> </property> <property> <name>mapreduce.jobtracker.hosts.filename</name> <value>$HADOOP_HOME/etc/hadoop/include</value> </property></configuration> |
7. yarn-site.xml
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | <configuration> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> <property> <name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name> <value>org.apache.hadoop.mapred.ShuffleHandler</value> </property> <property> <name>yarn.resourcemanager.hostname</name> <value>namenode</value> </property></configuration> |