需要开启快照功能,在hbase-site.xml
文件中添加如下配置项:
|
|
进入hbase shell执行:
|
|
需要开启快照功能,在hbase-site.xml
文件中添加如下配置项:
|
|
进入hbase shell执行:
|
|
由于在使用2.3.1版本elasticsearch过程中发现经常出现异常RemoteTransportException - AlreadyClosedException[this IndexReader is closed]
,发现是在2.3.0中引入的一个bug,在2.3.3中进行了修复,所以对其进行了升级。在小版本之间升级可以进行滚动升级(Rolling upgrades),但大版本之间就只能集群重启升级(Full cluster restart upgrade)。
使用sshpass实现的一个自动登录脚本。
发现一个有趣的canvas,可以改改给女朋友一个惊喜。
在做postgres的备份时,需要定时执行pg_dump命令,这就需要在人不介入的情况传入密码,有两种方式可以实现。
一是使用环境变量PGPASSWORD,二是使用.pgpass文件。
这几天遇到一个问题,hive在执行join操作时报错,return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
,又没有特别有用的信息,经过一番分析和搜索找到解决方法。在这里记录一下。
elasticsearch restful api和java api搜索示例。
在yarn的resource manager页面中有一项是Memory Reserved,解释如下:
An implementation detail of this change that prevents applications from starving under this new flexibility is the notion of reserved containers. Imagine two jobs are running that each have enough tasks to saturate more than the entire cluster. One job wants each of its mappers to get 1GB, and another job wants its mappers to get 2GB. Suppose the first job starts and fills up the entire cluster. Whenever one of its task finishes, it will leave open a 1GB slot. Even though the second job deserves the space, a naive policy will give it to the first one because it’s the only job with tasks that fit. This could cause the second job to be starved indefinitely. To prevent this unfortunate situation, when space on a node is offered to an application, if the application cannot immediately use it, it reserves it, and no other application can be allocated a container on that node until the reservation is fulfilled. Each node may have only one reserved container. The total reserved memory amount is reported in the ResourceManager UI. A high number means that it may take longer for new jobs to get space.
在linux下有时会遇到一种情况,某个文件删除了,但是磁盘空间并没有释放。这是因为仍然有进程在使用此文件,其文件句柄没有被释放。
下面介绍两种方式来解决此问题。
在夜间模式时间段可以自定义的情况下,判断传入的时间在不在夜间模式时间段内。
使用JodaTime进行时间操作。