- 浏览: 83475 次
- 性别:
- 来自: 广州
文章分类
- 全部博客 (110)
- myeclipse JVM 虚拟机内存设置 (1)
- 查询含有clob字段表的sql语句 (1)
- 项目个人价值体现 (1)
- Java多线程并发编程 (1)
- spring (4)
- 启悟 (1)
- hadoop (27)
- mysql数据库乱码问题 (1)
- linux (6)
- 架构与设计 (1)
- java (6)
- mysql (2)
- 分页编程 (1)
- 励志 (2)
- 技术要求 (0)
- guava (1)
- 分布式开发(SOA) (4)
- 微服务架构 + API 网关 (5)
- 消息中间件 (4)
- Dubbo (8)
- 面谈 (0)
- 高并发架构 (1)
- maven (1)
- MongoDB (1)
- hbase (2)
最新评论
问题的描述; yarn 启动时,nodemanager 成功,但是resourcemanager失败.
查看日记的出错信息:
2015-10-11 21:50:20,201 INFO org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService failed in state STARTED; cause: org.apache
.hadoop.yarn.exceptions.YarnRuntimeException: java.net.BindException: Problem binding to [h3:8031] java.net.BindException: Cannot assign requested address; For more details see: http://wiki.
apache.org/hadoop/BindException
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.net.BindException: Problem binding to [h3:8031] java.net.BindException: Cannot assign requested address; For more details see: ht
tp://wiki.apache.org/hadoop/BindException
at org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.getServer(RpcServerFactoryPBImpl.java:139)
at org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getServer(HadoopYarnProtoRPC.java:65)
at org.apache.hadoop.yarn.ipc.YarnRPC.getServer(YarnRPC.java:54)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService.serviceStart(ResourceTrackerService.java:159)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:503)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:898)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:938)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:935)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:935)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:979)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1104)
Caused by: java.net.BindException: Problem binding to [h3:8031] java.net.BindException: Cannot assign requested address; For more details see: http://wiki.apache.org/hadoop/BindException
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:719)
at org.apache.hadoop.ipc.Server.bind(Server.java:419)
at org.apache.hadoop.ipc.Server$Listener.<init>(Server.java:561)
at org.apache.hadoop.ipc.Server.<init>(Server.java:2166)
at org.apache.hadoop.ipc.RPC$Server.<init>(RPC.java:897)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server.<init>(ProtobufRpcEngine.java:505)
at org.apache.hadoop.ipc.ProtobufRpcEngine.getServer(ProtobufRpcEngine.java:480)
at org.apache.hadoop.ipc.RPC$Builder.build(RPC.java:742)
at org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.createServer(RpcServerFactoryPBImpl.java:169)
分析了下,地址绑定失败,大概的意思是指8301的端口给占用了。
google下,说是Namenode和ResourceManger如果不是同一台机器,不能在NameNode上启动 yarn,应该在ResouceManager所在的机器上启动yarn。现在问题解决了。
查看日记的出错信息:
2015-10-11 21:50:20,201 INFO org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService failed in state STARTED; cause: org.apache
.hadoop.yarn.exceptions.YarnRuntimeException: java.net.BindException: Problem binding to [h3:8031] java.net.BindException: Cannot assign requested address; For more details see: http://wiki.
apache.org/hadoop/BindException
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.net.BindException: Problem binding to [h3:8031] java.net.BindException: Cannot assign requested address; For more details see: ht
tp://wiki.apache.org/hadoop/BindException
at org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.getServer(RpcServerFactoryPBImpl.java:139)
at org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getServer(HadoopYarnProtoRPC.java:65)
at org.apache.hadoop.yarn.ipc.YarnRPC.getServer(YarnRPC.java:54)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService.serviceStart(ResourceTrackerService.java:159)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:503)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:898)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:938)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:935)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:935)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:979)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1104)
Caused by: java.net.BindException: Problem binding to [h3:8031] java.net.BindException: Cannot assign requested address; For more details see: http://wiki.apache.org/hadoop/BindException
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:719)
at org.apache.hadoop.ipc.Server.bind(Server.java:419)
at org.apache.hadoop.ipc.Server$Listener.<init>(Server.java:561)
at org.apache.hadoop.ipc.Server.<init>(Server.java:2166)
at org.apache.hadoop.ipc.RPC$Server.<init>(RPC.java:897)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server.<init>(ProtobufRpcEngine.java:505)
at org.apache.hadoop.ipc.ProtobufRpcEngine.getServer(ProtobufRpcEngine.java:480)
at org.apache.hadoop.ipc.RPC$Builder.build(RPC.java:742)
at org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.createServer(RpcServerFactoryPBImpl.java:169)
分析了下,地址绑定失败,大概的意思是指8301的端口给占用了。
google下,说是Namenode和ResourceManger如果不是同一台机器,不能在NameNode上启动 yarn,应该在ResouceManager所在的机器上启动yarn。现在问题解决了。
发表评论
-
MapReduce程序开发(good)
2016-09-06 18:25 242http://blog.csdn.net/qq10108856 ... -
ZooKeeper典型应用场景一览
2016-07-08 16:17 383ZooKeeper典型应用场景一览 摘自:http://www ... -
Apache HBase 2015年发展回顾与未来展望
2016-02-05 10:00 392摘自: http://mp.weixin.qq.com/s? ... -
搜索技术博客-淘宝
2015-12-10 18:33 449搜索技术博客-淘宝: http://www.searcht ... -
大数据性能调优之HBase的RowKey设计
2015-11-12 13:47 350摘自;http://www.open-open.com ... -
HBase处理中文字符串
2015-11-12 13:44 692摘自:http://blog.csdn.net/piratel ... -
详细的讲解yarn和mapreduce的内存配置
2015-11-03 17:00 354摘自: http://docs.hortonworks.co ... -
Hadoop YARN常见问题以及解决方案 任务合理调度分配
2015-11-03 14:46 1130摘自: http://dongxicheng.org/map ... -
Windows下使用Hadoop2.6.0-eclipse-plugin插件
2015-10-29 15:43 341Windows下使用Hadoop2.6.0-eclipse-p ... -
HBase优化案例分析:Facebook Messages系统问题与解决方案
2015-10-21 18:58 417摘自:http://www.infoq.com/cn/arti ... -
HBase总结(十二)Java API 与HBase交互实例
2015-10-21 18:57 394import java.io.IOException; im ... -
NoSQL选型及HBase案例详解(hbase结合情景案例分析)
2015-10-21 18:55 497摘自:http://www.thebigdata.cn/Jie ... -
Hbase课程大纲学习路线
2015-10-21 18:49 970Hbase课程大纲 学习内容: Hbase系统搭建与部署,Hb ... -
基于HBase的海量微博数据高效存储解析
2015-10-20 17:15 761通过分析HBase的特点, ... -
Hadoop 2.0+YARN启动脚本分析与实战经验
2015-10-16 23:38 570start-all.sh脚本现在已经废弃,推荐使用start- ... -
windows7+eclipse+hadoop2.5.2环境配置
2015-10-11 23:45 330http://www.dataguru.cn/thread-4 ... -
YARN ResourceManager调度器的分析
2015-10-11 23:39 318摘自:http://blogread.cn/it/articl ... -
Hadoop 新 MapReduce 框架 Yarn 详解
2015-10-11 23:08 494摘自:http://www.open-open.c ... -
hbase常识及habse适合什么场景
2015-09-18 14:15 339http://blog.csdn.net/lifuxiangc ... -
HBase案例设计分析
2015-09-18 14:03 403http://www.thebigdata.cn/JieJue ...
相关推荐
Hadoop ResourceManager API
YARN配置、启动与验证 YARN配置、启动与验证 序号 任务名称 任务一 YARN组件参数配置 任务二 MapReduce组件参数配置 任务三 配置SSH无密钥登录(slave1为主节点) 任务四 分发YARN与MapReduce配置文件 任务五 启动...
NodeManager ResourceManager NodeManager NodeManager 3. 配置集群 (1)核心配置文件 配置core-site.xml (2)HDFS配置文件 配置 hadoop-env.sh 配置 hadoop-site.xml (3)YARN配置文件 配置 yarn-env.sh 配置 ...
NodeManager ResourceManager NodeManager NodeManager 3. 配置集群 (1)核心配置文件 配置core-site.xml (2)HDFS配置文件 配置 hadoop-env.sh 配置 hadoop-site.xml (3)YARN配置文件 配置 yarn-env.sh 配置 ...
NodeManager ResourceManager NodeManager NodeManager 3. 配置集群 (1)核心配置文件 配置core-site.xml (2)HDFS配置文件 配置 hadoop-env.sh 配置 hadoop-site.xml (3)YARN配置文件 配置 yarn-env.sh 配置 ...
NodeManager ResourceManager NodeManager NodeManager 3. 配置集群 (1)核心配置文件 配置core-site.xml (2)HDFS配置文件 配置 hadoop-env.sh 配置 hadoop-site.xml (3)YARN配置文件 配置 yarn-env.sh 配置 ...
通过测试,hdfs与yarn组件启动后有以下进程: node1 jps:4个进程 NameNode、JournalNode、DFSZKFailoverController、Jps node2 jps:7个进程 DFSZKFailoverController、NodeManager、JournalNode、QuorumPeerMain、...
NodeManager Restart DockerContainerExecutor Using CGroups Secure Containers Registry YARN REST APIs Introduction Resource Manager Node Manager Timeline Server Hadoop Compatible File Systems ...
Hadoop集群具体来说包含两个集群:HDFS集群和YARN集群,两者逻辑上...(2)YARN集群:负责海量数据运算时的资源调度,集群中的角色主要有 ResourceManager /NodeManager (3)MapReduce:它其实是一个应用程序开发包。
全书共四部分13 章:第一部分(第1~2 章...第二部分(第3~7 章)结合源代码详细剖析和讲解了YARN 的第三方开源库、底层通信库、服务库、事件库的基本使用和实现细节,详细讲解了YARN 的应用程序设计方法...
yarn.nodemanager.aux-services mapreduce_shuffle yarn.resourcemanager.address hadoop0001:8032 yarn.resourcemanager.scheduler.address hadoop0001:8030 yarn.resourcemanager.resource-tracker.
YARN ResourceManager + JobHistoryServer + ProxyServer + Spark从属node3:HDFS数据节点+ YARN NodeManager + Spark从站node4:HDFS数据节点+ YARN NodeManager + Spark从属node5:HDFS数据节点+ YARN NodeManager...
Hadoop集群安装 集群规划 服务器hadoop101 服务器hadoop102 ... Yarn ... NodeManager ... Resourcemanager NodeManager NodeManager 1. 虚拟机
mapReduce存在问题:JobTracker单点故障JobTracker承受的访问压力大,影响系统扩展不支持...它由下面几大构成组件:ResourceManager:负责为集群中的所有应用程序分配资源每个节点代理的NodeManager每个应用对应一
node2 : YARN ResourceManager + JobHistoryServer + ProxyServer node3 : HDFS DataNode + YARN NodeManager + Spark Slave node4 : HDFS DataNode + YARN NodeManager + Spark Slave 入门 。 运行vagrant box...
node2:YARN ResourceManager + JobHistoryServer + ProxyServer node3:HDFS数据节点+ YARN NodeManager + Spark从站 node4:HDFS数据节点+ YARN NodeManager + Spark从站 入门 。 运行vagrant box add centos...
1 、Hadoop是什么?它能做什么? 2 、Hadoop和MapReduce的关系是什么? 3 、Hadoop中的HDFS是什么?它有哪些优点? 4 、HDFS的组成结构是...18、YARN中的ResourceManager和NodeManager的作用分别是什么? 。。。。。。
hadoop-2.7.2安装包,解压至/opt/module文件夹 启动HDFS只需要敲入命令行:sbin/hadoop-daemon.sh start|stop namenode|datanode|secondarynamenode ...sbin/yarn-daemon.sh start|stop resourcemanager|nodemanager
3.5验证是否启动成功 使用jps命令验证 27408 NameNode 28218 Jps 27643 SecondaryNameNode 28066 NodeManager 27803 ResourceManager 27512 DataNode http://192.168.1.101:50070 (HDFS管理界面) ...
Yarn作为一个资源调度平台,有一个全局的管理者叫做ResourceManager,ResourceManager负责对集群的整体计算及资源做统筹规划,有各个节点的管理者叫做NodeManager