本文共 20183 字，大约阅读时间需要 67 分钟。
大家需要在每个节点上提前装好"expect"工具
expect的使用请看我的另一篇文章：
spawn命令激活一个Unix程序来进行交互式的运行。
send命令向进程发送字符串。
expect命令等待进程的某些字符串
set timeout 1 设置超时时间 timeout -1 为永不超时
expect eof
只有spawn执行的命令结果才会被expect捕捉到，因为spawn会启动一个进程，只有这个进程的相关信息才会被捕捉到，主要包括：标准输入的提示信息，eof和timeout。
这里，eof是必须去匹配的，在spawn进程结束后会向expect发送eof；如果不去匹配，有时也能运行，比如sleep多少秒后再去spawn下一个命令，但是不要依赖这种行为，很有可能今天还可以，明天就不能用了。expect \"#\" 期待返回shell提示符(是#或者$)
interact 命令
执行完成后保持交互状态，把控制权交给控制台，这个时候就可以手工操作了。如果没有这一句登录完成后会退出，而不是留在远程终端上。如果你只是登录过去执行一段命令就退出，可改为 expect eof
id_dsa/ id_dsa.pub：你用openssh工具生成的私钥公钥对
authorized_keys ：你使用ssh连接的linux服务器需要认证你的身份，所以你需要在连接的linux服务器上安装自已的公钥，authorized_keys这里面就是存放你自己的id_dsa.pub的内容scp是有Security的文件copy，基于ssh登录。操作起来比较方便，比如要把当前一个文件copy到远程另外一台主机上，可以如下命令。
scp /home/daisy/full.tar.gz
大体思路
1、首先在一个文本文件中保存1000台机器的hadoop用户名和密码
2、用shell遍历这个文件 写一个循环用namenode的去循环登陆其他的999个节点，执行生成密钥的工作，然后把生成的公钥写回namenode3、在namenode上生成密钥 写入这个文件4、把第三部生成的文件拷贝到剩下的机器上5、用循环遍历验证免密的效果本解决方法主要包括两个脚本： sshpass.sh和ssh4slaves
1. sshpass.sh
#!/bin/bash# Name : sshpass.sh# Time : 17/09/2012# Author : simplestone@dbinterest.com# Purpose : For fast and easy setup of the SSH Passwordless access among all the nodes# in a cluster. # User : Any user you are performing the test! Better to settup a separate user from your# working env to avoid troubles!!! "root" is used in this example, and you can change it# via the export virable "USER=root"# Attention: The test env is assuming that each $USER on each $HOST is usring the same password!# And this likely makes sense as no body want to put more trouble on this.# Usage : 1st, make sure the script has the execute permisison "chmod +x ssh_pass.sh"# ./ssh_pass.sh password# : 2nd, ensure the "ssh4slaves.sh" script is with ssh_pass.sh for all nodes setup!!!# : 3rd, "expect" has to be installed on all the nodes for the SSH configexport FILELOC="/root"export SLAVESFILE="$FILELOC/sshslaves"export HOSTS=`cat $FILELOC/sshhosts`export SLAVES=`cat $FILELOC/sshslaves`export SSH4SLAVESCRIPT="$FILELOC/ssh4slaves.sh"export MASTER=hdp01export USER=rootexport PASSWD=$1export SSHLOC="$FILELOC/.ssh/"export RSAFILE="$FILELOC/.ssh/id_rsa"export RSAPUBFILE="$FILELOC/.ssh/id_rsa.pub"export AUTHFILE="$FILELOC/.ssh/authorized_keys"export EXPECTCHK=`rpm -qa expect | wc -l`#if [ $EXPECTCHK != 1 ] then echo '' echo "########################################################################################" echo "Please install the \"expect\" package first on all nodes to allow the script to run!!!" echo "yum -y install expect" echo "########################################################################################"else if [ -e $RSAFILE ] then echo "########################################################################################" echo "Attention: This is for TEST ONLY, please fully test it before applying it to PROD" echo "environment!!! OR you might get in trouble!!!" echo '' echo "BETTER TO HAVE A NEW USER FOR THE TEST TO AVOID DESTROYING YOUR ENVIRONMENT!" echo '' echo "Please manually delete the ssh related file on each host before executing the script!!!" echo '' for host in $HOSTS do echo "Please run command on $host: rm -rf $SSHLOC" done echo "########################################################################################" else # Just generate for host in $HOSTS do if [ $host = "$MASTER" ] then echo '' echo "###########################################################" echo "Generating RSA keys for MASTER host $MASTER" echo "###########################################################" echo '' expect -c " set timeout 1 spawn ssh $USER@$host expect \"yes/no\" send -- \"yes\r\" expect \"password:\" send -- \"$PASSWD\r\" expect \"#\" send \"ssh-keygen -t rsa -P '' -f $RSAFILE\r\" expect \"#\" send \"ssh-copy-id -i $RSAPUBFILE $MASTER\r\" expect \"password:\" send -- \"$PASSWD\r\" expect eof " else echo '' echo "###########################################################" echo "Generating RSA keys for all OTHER hosts..." echo "hostname is $host" echo "###########################################################" echo '' expect -c " set timeout 1 spawn ssh $USER@$host expect \"yes/no\" send -- \"yes\r\" expect \"password:\" send -- \"$PASSWD\r\" expect \"#\" send \"ssh-keygen -t rsa -P '' -f $RSAFILE\r\" expect \"#\" send \"ssh-copy-id -i $RSAPUBFILE $MASTER\r\" expect \"yes/no\" send -- \"yes\r\" expect \"password:\" send -- \"$PASSWD\r\" expect eof " fi done ### for host in $SLAVES do echo '' echo "############################################################################" echo "Copying authorized_keys to host $host from the MASTER host $MASTER..." echo "############################################################################" echo '' expect -c " set timeout 1 spawn scp $AUTHFILE "$USER@$host:$SSHLOC" expect \"password:\" send -- $PASSWD\r expect eof " done # for host in $SLAVES do echo '' echo "############################################################################" echo "Distributing the $SLAVESFILE file to slave host $host..." echo "############################################################################" echo '' scp $SLAVESFILE "$host:$FILELOC" echo '' echo "############################################################################" echo "Distributing the $SSH4SLAVESCRIPT script to slave host $host..." echo "############################################################################" echo '' scp $SSH4SLAVESCRIPT "$host:$FILELOC" done for host in $SLAVES do echo '' echo "############################################################################" echo "Working on the slaves node $host to ensure no prompt for the "yes/no" question..." echo "############################################################################" echo '' ssh -q $USER@$host $SSH4SLAVESCRIPT done ### Check whether the Passwordless ssh works ### for host in $HOSTS do echo '' echo "############################################################################" echo "Check whether the Passwordless SSH works for $host..." echo "############################################################################" echo '' ssh $host uname -a && date done fifi#### rm -rf /root/.ssh# mv /root/.ssh /root/sshlogin#{ eval "$GET_ID" ; } | ssh $1 "umask 077; test -d .ssh || mkdir .ssh ; cat >> .ssh/authorized_keys; test -x /sbin/restorecon && /sbin/restorecon .ssh .ssh/authorized_keys" || exit 1#cat /root/.ssh/id_rsa.pub | ssh hdp01 "umask 077; test -d .ssh || mkdir .ssh ; cat >> .ssh/authorized_keys; test -x /sbin/restorecon && /sbin/restorecon .ssh .ssh/authorized_keys" || exit 1#/root/.ssh/id_rsa.pub ./ssh_pass.sh password （password替换程序里的$1参数），本例中密码为stonetest
2. ssh4slaves
#!/bin/bash# Name : ssh4slaves.sh# Time : 17/09/2012# Author : simplestone@dbinterest.com# Purpose : For fast and easy setup of the SSH Passwordless access among all the slave nodes# in a cluster. Mainly to ensure no prompt for "yes/no" again!!!# User : Any user you are performing the test! Better to settup a separate user from your# working env to avoid troubles!!! "root" is used in this example, and you can change it# via the export virable "USER=root"# Attention: The test env is assuming that each $USER on each $HOST is usring the same password!# And this likely makes sense as no body want to put more trouble on this.# Usage : This script is called by the main script "ssh_pass.sh"# 1st, make sure the script has the execute permisison "chmod +x ssh4slaves.sh" before# distributing it to other slaves node.# 2nd, Remember to change variable "PASSWORD" before start the main script "sshpass.sh"export FILELOC="/root"export SLAVES=`cat $FILELOC/sshslaves`export USER=rootexport PASSWD=stonetestfor host in $SLAVESdo echo '' echo "Ensure ssh passwordless works among all slave nodes..." echo '' expect -c " set timeout 1 spawn ssh $USER@$host expect \"yes/no\" send -- \"yes\r\" expect eof " done
3. 其他配置
[root@hdp01 ~]# pwd/root[root@hdp01 ~]# cat sshhostshdp01hdp02hdp03[root@hdp01 ~]# cat sshslaveshdp02hdp03[root@hdp01 ~]# ls -lrth | tail -2-rwxr-xr-x 1 root root 1.3K Sep 18 02:08 ssh4slaves.sh-rwxr-xr-x 1 root root 6.5K Sep 18 02:11 ssh_pass.sh4. 测试输出
[root@hdp01 ~]# ./ssh_pass.sh stonetest###########################################################Generating RSA keys for MASTER host hdp01###########################################################spawn ssh root@hdp01The authenticity of host 'hdp01 (192.168.1.121)' can't be established.RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added 'hdp01,192.168.1.121' (RSA) to the list of known hosts.root@hdp01's password: Last login: Tue Sep 18 02:09:29 2012 from hdp02.dbinterest.local[root@hdp01 ~]# ssh-keygen -t rsa -P '' -f /root/.ssh/id_rsaGenerating public/private rsa key pair.Your identification has been saved in /root/.ssh/id_rsa.Your public key has been saved in /root/.ssh/id_rsa.pub.The key fingerprint is:3a:c3:98:b3:e4:39:fa:fe:87:c6:22:90:16:57:4e:47 root@hdp01.dbinterest.localThe key's randomart image is:+--[ RSA 2048]----+| .E || o . || + . || . . . || .o S ||o. + . ||.. =.=. || .oo++o. || .=*=.. |+-----------------+[root@hdp01 ~]# ssh-copy-id -i /root/.ssh/id_rsa.pub hdp01root@hdp01's password: Now try logging into the machine, with "ssh 'hdp01'", and check in: .ssh/authorized_keysto make sure we haven't added extra keys that you weren't expecting.[root@hdp01 ~]# ###########################################################Generating RSA keys for all OTHER hosts...hostname is hdp02###########################################################spawn ssh root@hdp02The authenticity of host 'hdp02 (192.168.1.122)' can't be established.RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added 'hdp02,192.168.1.122' (RSA) to the list of known hosts.root@hdp02's password: Last login: Tue Sep 18 02:09:23 2012 from hdp02.dbinterest.local[root@hdp02 ~]# ssh-keygen -t rsa -P '' -f /root/.ssh/id_rsaGenerating public/private rsa key pair.Created directory '/root/.ssh'.Your identification has been saved in /root/.ssh/id_rsa.Your public key has been saved in /root/.ssh/id_rsa.pub.The key fingerprint is:a9:89:fe:40:8a:8e:21:55:da:3b:6b:68:4f:3e:8f:fc root@hdp02.dbinterest.localThe key's randomart image is:+--[ RSA 2048]----+| || || . || + . || o o S || o o o o ||+ ..* o ||+.o=o= ||.o oB=E |+-----------------+[root@hdp02 ~]# ssh-copy-id -i /root/.ssh/id_rsa.pub hdp01The authenticity of host 'hdp01 (192.168.1.121)' can't be established.RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added 'hdp01,192.168.1.121' (RSA) to the list of known hosts.root@hdp01's password: Now try logging into the machine, with "ssh 'hdp01'", and check in: .ssh/authorized_keysto make sure we haven't added extra keys that you weren't expecting.###########################################################Generating RSA keys for all OTHER hosts...hostname is hdp03###########################################################spawn ssh root@hdp03The authenticity of host 'hdp03 (192.168.1.123)' can't be established.RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added 'hdp03,192.168.1.123' (RSA) to the list of known hosts.root@hdp03's password: Last login: Tue Sep 18 02:09:19 2012 from hdp02.dbinterest.local[root@hdp03 ~]# ssh-keygen -t rsa -P '' -f /root/.ssh/id_rsaGenerating public/private rsa key pair.Created directory '/root/.ssh'.Your identification has been saved in /root/.ssh/id_rsa.Your public key has been saved in /root/.ssh/id_rsa.pub.The key fingerprint is:a4:3d:dd:54:42:c0:45:ec:ed:ae:d6:bd:14:a0:9b:16 root@hdp03.dbinterest.localThe key's randomart image is:+--[ RSA 2048]----+| ..*= . || . .o || . ..o || + . oo o || . S .E.. . || . + . .|| + o o || . . + .|| ... ..|+-----------------+[root@hdp03 ~]# ssh-copy-id -i /root/.ssh/id_rsa.pub hdp01The authenticity of host 'hdp01 (192.168.1.121)' can't be established.RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added 'hdp01,192.168.1.121' (RSA) to the list of known hosts.root@hdp01's password: Now try logging into the machine, with "ssh 'hdp01'", and check in: .ssh/authorized_keysto make sure we haven't added extra keys that you weren't expecting.[root@hdp03 ~]# ############################################################################Copying authorized_keys to host hdp02 from the MASTER host hdp01...############################################################################spawn scp /root/.ssh/authorized_keys root@hdp02:/root/.ssh/root@hdp02's password: authorized_keys 100% 1227 1.2KB/s 00:00 ############################################################################Copying authorized_keys to host hdp03 from the MASTER host hdp01...############################################################################spawn scp /root/.ssh/authorized_keys root@hdp03:/root/.ssh/root@hdp03's password: authorized_keys 100% 1227 1.2KB/s 00:00 ############################################################################Distributing the /root/sshslaves file to slave host hdp02...############################################################################sshslaves 100% 12 0.0KB/s 00:00 ############################################################################Distributing the /root/ssh4slaves.sh script to slave host hdp02...############################################################################ssh4slaves.sh 100% 1277 1.3KB/s 00:00 ############################################################################Distributing the /root/sshslaves file to slave host hdp03...############################################################################sshslaves 100% 12 0.0KB/s 00:00 ############################################################################Distributing the /root/ssh4slaves.sh script to slave host hdp03...############################################################################ssh4slaves.sh 100% 1277 1.3KB/s 00:00 ############################################################################Working on the slaves node hdp02 to ensure no prompt for the yes/no question...############################################################################Ensure ssh passwordless works among all slave nodes...spawn ssh root@hdp02The authenticity of host 'hdp02 (192.168.1.122)' can't be established.RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added 'hdp02,192.168.1.122' (RSA) to the list of known hosts.Last login: Tue Sep 18 02:11:54 2012 from hdp01.dbinterest.local[root@hdp02 ~]# Ensure ssh passwordless works among all slave nodes...spawn ssh root@hdp03The authenticity of host 'hdp03 (192.168.1.123)' can't be established.RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added 'hdp03,192.168.1.123' (RSA) to the list of known hosts.Last login: Tue Sep 18 02:11:55 2012 from hdp01.dbinterest.local[root@hdp03 ~]# ############################################################################Working on the slaves node hdp03 to ensure no prompt for the yes/no question...############################################################################Ensure ssh passwordless works among all slave nodes...spawn ssh root@hdp02The authenticity of host 'hdp02 (192.168.1.122)' can't be established.RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added 'hdp02,192.168.1.122' (RSA) to the list of known hosts.Last login: Tue Sep 18 02:11:58 2012 from hdp02.dbinterest.local[root@hdp02 ~]# Ensure ssh passwordless works among all slave nodes...spawn ssh root@hdp03The authenticity of host 'hdp03 (192.168.1.123)' can't be established.RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added 'hdp03,192.168.1.123' (RSA) to the list of known hosts.Last login: Tue Sep 18 02:11:59 2012 from hdp02.dbinterest.local############################################################################Check whether the Passwordless SSH works for hdp01...############################################################################Linux hdp01.dbinterest.local 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 x86_64 x86_64 x86_64 GNU/LinuxTue Sep 18 02:12:05 PDT 2012############################################################################Check whether the Passwordless SSH works for hdp02...############################################################################Linux hdp02.dbinterest.local 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 x86_64 x86_64 x86_64 GNU/LinuxTue Sep 18 02:12:05 PDT 2012############################################################################Check whether the Passwordless SSH works for hdp03...############################################################################Linux hdp03.dbinterest.local 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 x86_64 x86_64 x86_64 GNU/LinuxTue Sep 18 02:12:06 PDT 2012
5. 其他节点测试
[root@hdp02 ~]# [root@hdp02 ~]# ssh hdp02Last login: Tue Sep 18 02:12:00 2012 from hdp03.dbinterest.local[root@hdp02 ~]# exitlogoutConnection to hdp02 closed.[root@hdp02 ~]# ssh hdp03Last login: Tue Sep 18 02:12:02 2012 from hdp03.dbinterest.local[root@hdp03 ~]# exitlogoutConnection to hdp03 closed.[root@hdp02 ~]#----------[root@hdp03 ~]# [root@hdp03 ~]# ssh hdp01Last login: Tue Sep 18 02:12:22 2012 from hdp02.dbinterest.local[root@hdp01 ~]# exitlogoutConnection to hdp01 closed.[root@hdp03 ~]# ssh hdp02Last login: Tue Sep 18 02:12:25 2012 from hdp02.dbinterest.local[root@hdp02 ~]# exitlogoutConnection to hdp02 closed.[root@hdp03 ~]# ssh hdp03Last login: Tue Sep 18 02:12:30 2012 from hdp02.dbinterest.local[root@hdp03 ~]# exitlogoutConnection to hdp03 closed.[root@hdp03 ~]#
代码下载见附件
参考文章：
本文出自 “” 博客，请务必保留此出处
转载地址：http://sjhax.baihongyu.com/