新浦京81707con > 注册购买 > ubuntu下hbase的伪分布式安装与配置

原标题:ubuntu下hbase的伪分布式安装与配置

浏览次数:60 时间:2019-11-07

葡京线路检测中心,在推行hbase或mapreduce的先后时,有时会遇上ClassNotFoundException。借使jar包里的MANIFEST.MF未有一点名Main-Class,就能够超过这一问题。但若是钦点正确,依旧会遭遇这么的标题。

Hbase Mapreduce 例子

http://Hadoop.apache.org/hbase/docs/current/api/org/apache/hadoop/hbase/mapreduce/package-summary.html#package_description

http://wiki.apache.org/hadoop/Hbase/MapReduce  (Deprecated)

 

内需重启Hadoop的法子

怀有机器都有改变配置

1:修改$HADOOP_HOME/conf/hadoop-env.sh ,增添HBase类库援用

export HBASE_HOME=/home/iic/hbase-0.20.3

export HADOOP_CLASSPATH=$HBASE_HOME/hbase-0.20.3.jar:$HBASE_HOME/hbase-0.20.3-test.jar:$HBASE_HOME/conf:${HBASE_HOME}/lib/zookeeper-3.3.0.jar

 

无需重启Hadoop的艺术(把信任类库打包进jar/lib目录下,同不时间代码中调用job.setJarByClass(XXX.class);卡塔尔

Another possibility, if for example you do not have access to hadoop-env.sh or are unable to restart the hadoop cluster, is bundling the hbase jars into a mapreduce job jar adding it and its dependencies under the job jar lib/ directory and the hbase conf into the job jars top-level directory.

 

测量检验,出现分外:java.lang.OutOfMemoryError: Java heap space

bin/hadoop org.apache.hadoop.hbase.PerformanceEvaluation sequentialWrite 4 

1.hadoop版本:hadoop-1.1.1,HBase版本:hbase-0.94.3

有人的建议是将持有应用的jar包都打进要试行的jar包里。那是生机勃勃种缓慢解决措施。但就算将jar包全打进了新的jar文件,依然会遇上该难题。如本身在实行hbase的IndexBuilder这些事例时,碰着如下难题:

HBase map reduce 2

此例子,把表mrtest中的列contents的值,反转后,保存到列text里。

bin/hbase shell

create 'mrtest', 'contents', 'text'

put 'mrtest', '1', 'contents:', 'content'

put 'mrtest', '1', 'text:', 'text'

get 'mrtest', '1'

 

类com.test.hadoop.hbase.HBaseTest 生成100W的测量试验数据。

 

/home/iic/hadoop-0.20.2/bin/hadoop jar  examples/examples_1.jar  examples.TestTableMapReduce

 

2.到官网下载好hbase-0.94.3.tar.gz,解压hbase安装包到多个可用目录如(/home/hadoop/hbase-0.94.3/)
root@ubuntu:/home/hadoop#   sudo tar zxvf  hbase-0.94.3.tar.gz

12/06/11 18:23:32 INFO mapred.JobClient: Task Id : attempt_201206111811_0001_m_000002_1, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.Hadoop.hbase.mapreduce.MultiTableOutputFormat
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:867)
        at org.apache.hadoop.mapreduce.JobContext.getOutputFormatClass(JobContext.java:235)
        at org.apache.hadoop.mapred.Task.initialize(Task.java:513)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:353)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
        at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.mapreduce.MultiTableOutputFormat
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:264)
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:820)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:865)
        ... 8 more

HBase 自带例子

hbase-0.20.3srctest

安装hbase-0.94.3所在顾客为hadoop,所在组为hadoop(root顾客操作)
root@ubuntu:/home/hadoop#  chown -R hadoop hbase-0.94.3
root@ubuntu:/home/hadoop#  chgrp -R hadoop hbase-0.94.3
或者
root@ubuntu:/home/hadoop#  chown -R hadoop:hadoop hbase-0.94.3

org.apache.hadoop.hbase.mapreduce.MultiTableOutputFormat是带有在hbase-0.94.0.jar里的。笔者打客车jar包里明确有该jar文件,依旧找不到。 和 的HBase, MapReduce and the CLASSPATH

总结表的总集团数(org.apache.hadoop.hbase.mapreduce.RowCounter卡塔尔

bin/hadoop jar /home/iic/hbase-0.20.3/hbase-0.20.3.jar rowcounter scores grade

结果

10/04/12 17:08:05 INFO mapred.JobClient:     ROWS=2

 

3.安装景况变量:在/etc/profile和/home/hadoop/.bashrc中配置丰盛(铁锈色字体的)
#set java environment
HBASE_HOME=/home/hadoop/hbase-0.94.3
HIVE_HOME=/home/hadoop/hive-0.9.0
HADOOP_HOME=/home/hadoop/hadoop-1.1.1
JAVA_HOME=/home/hadoop/jdk1.7.0
PATH=$JAVA_HOME/bin:$HBASE_HOME/bin:$HIVE_HOME/bin:$HADOOP_HOME/bin:$PATH
CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$HBASE_HOME/lib:$HIVE_HOME/lib:$JAVA_HOME/lib/tools.jar
export HBASE_HOME
export HADOOP_HOME
export JAVA_HOME
export HIVE_HOME
export PATH
export CLASSPATH

本文由新浦京81707con发布于注册购买,转载请注明出处:ubuntu下hbase的伪分布式安装与配置

关键词: 新浦京81707con Hbase hbase安装配置 hbase部署 HBase伪分

上一篇:如何实现High,备份方案

下一篇:没有了