大数据疑难解决

· 2024-04-17 · # linux # hadoop

记录一些实战遇到的问题和解决办法

NoNode for /hbase/hbaseid

运行hbase shell出现类似报错，
解决办法
修改hbase-env.sh，为该文件增加
export HBASE_DISABLE_HADOOP_CLASSPATH_LOOKUP="true"

运行出现 entry in command string: null chmod 0700 异常解决方案

去 https://github.com/cdarlint/winutils
随便找个近似版本,下载里面的 hadoop.dll 和 winutils.exe
添加 hadoop.dll 和 winutils.exe 到 windows 的环境变量比如放到 hadoop/bin 里

no hbase master found

hbase 启动失败，idea 里面操作无报错无返回
查看日志

2024-04-24T09:40:45,557 ERROR [main] regionserver.HRegionServer: Failed construction RegionServer
java.io.IOException: Incomplete HDFS URI, no host: hdfs://master:8020:hbase

2024-04-24T09:40:45,562 ERROR [main] master.HMasterCommandLine: Master exiting
java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMaster.
    ...
Caused by: java.io.IOException: Incomplete HDFS URI, no host: hdfs://master:8020/hbase
    ...

但 hdfs://master/8020 可达

谁知 hbase 竟然需要 hadoop 的版本支持
可见网址
https://hbase.apache.org/book.html#hadoop

我的 hbase 版本是 hadoop 所不支持的，自然不能运行

        <hadoop.version>2.7.5</hadoop.version>
        <hbase.version>2.5.8</hbase.version>

官网可不提供旧版的下载，不过还好提供了 archive
https://archive.apache.org/dist/hbase/

Hadoop-2.7.1+ 对应得是 HBase-1.4.x，直接选择最新版
https://archive.apache.org/dist/hbase/1.4.14/hbase-1.4.14-bin.tar.gz

Master is initializing

HTTP ERROR 500
Problem accessing /master-status. Reason:

    Master is initializing

/opt/zookeeper/bin/zkCli.sh

ls /

http://master:50070/

2024-04-24 10:57:11,397 INFO [master/master/192.168.29.20:16000-SendThread(master:2181)] zookeeper.ClientCnxn: Opening socket connection to server master/192.168.29.20:2181. Will not attempt to authenticate using SASL (unknown error)

2024-04-24 10:57:25,981 WARN [933646237@qtp-1397530667-0] mortbay.log: /master-status: org.apache.hadoop.hbase.PleaseHoldException: Master is initializing

2024-04-24 10:54:19,310 [myid:] - WARN [QuorumConnectionThread-[myid=2]-2:o.a.z.s.q.QuorumCnxManager@401] - Cannot open channel to 3 at election address slave2/192.168.29.22:3888
java.net.ConnectException: 拒绝连接 (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:476)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:218)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:200)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:394)
at java.net.Socket.connect(Socket.java:606)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:384)
at org.apache.zookeeper.server.quorum.QuorumCnxManager $QuorumConnectionReqThread.run(QuorumCnxManager.java:458) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$ Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)