存档

作者存档

fastdfs体验

2013年1月21日 评论已被关闭

download source code

wget http://fastdfs.googlecode.com/files/FastDFS_v4.04.tar.gz

yum install libevent libevent-devel

Setup#step 1. download FastDFS source package and unpack ittar xzf FastDFS_v4.04.tar.gz#for example:tar xzf FastDFS_v4.04.tar.gz

阅读全文…

分类: cloud, dfs 标签:

centos5.4升级网卡驱动

2013年1月4日 评论已被关闭

机器型号:Dell R710,操作系统Centos5.4,症状:流量大时网卡不通,日志无明显迹象。

解决办法,升级网卡驱动:

升级前:

[root@2hei.net]# ethtool -i eth0
driver: bnx2
version: 1.9.3
firmware-version: 5.2.3 NCSI 2.0.11
bus-info: 0000:01:00.0

步骤如下:

wget http://ftp.us.dell.com/network/Bcom_LAN_16.4.0_Linux_Source_A00.tar.gz

tar -zxvf Bcom_LAN_16.4.0_Linux_Source_A00.tar.gz
cd Bcom_LAN_16.4.0_Linux_Source_A00
cd NetXtremeII/
rpm -ivh netxtreme2-6.4.44-1.src.rpm

cd /usr/src/redhat/
rpmbuild -bb SPECS/netxtreme2.spec
rpm -ivh RPMS/x86_64/netxtreme2-6.4.44-1.x86_64.rpm

screen -R update_driver

vim upgrade.sh

#!/bin/bash

rmmod bnx2
depmod
modprobe bnx2

sh upgrade.sh &

升级后:

[root@2hei.net]#ethtool -i eth0
driver: bnx2
version: 2.1.6b
firmware-version: bc 3.5.12 ipms 1.6.0
bus-info: 0000:03:00.0

[root@2hei.net]#modinfo bnx2
filename: /lib/modules/2.6.18-164.el5/updates/bnx2.ko
version: 2.1.6b
license: GPL
description: Broadcom NetXtreme II BCM5706/5708/5709/5716 Driver

分类: linux 标签: ,

站点被封

2013年1月4日 评论已被关闭

blog久未更新,今天突然收到邮件,说我的站点影响了服务器的性能(reason: site causing performance problems),经检查,是多年前装着玩的一个discuz站点,mysql表占用了2.8GB空间,吼吼,从未维护过的bbs上面居然注册了很多用户,从log上看,真真不少流量,大伙玩的很happy,各个自娱自乐,宛然成了灌水乐园。。。

于是¥%……&*(%¥%,对不起鸟,数据库直接给drop掉,发布目录也给清空了,这下大家满意了吧!

分类: others 标签:

alexa排名facebook超过了google

2012年9月18日 评论已被关闭

几天没见, Facebook目前Alexa排名全球第一,美国本土排名第二,Google正好相反。这让我想起了几年前yahoo被google赶超的情形,真是三两年河东,三两年河西啊,难免感慨唏嘘。

从另一侧面也反映了目前社交网络的火爆程度带来的巨大流量,而智能手机及各种应用,api的调用更是功不可没。

Alexa地址: http://www.alexa.com/topsites/global;0

分类: IT 标签: , ,

run or spawn in pexpect

2012年8月10日 评论已被关闭

Pexpect is a Python module for spawning child applications and controlling
them automatically. Pexpect can be used for automating interactive applications
such as ssh, ftp, passwd, telnet, etc. It can be used to a automate setup
scripts for duplicating software package installations on different servers. It
can be used for automated software testing. Pexpect is in the spirit of Don
Libes’ Expect, but Pexpect is pure Python. Other Expect-like modules for Python
require TCL and Expect or require C extensions to be compiled. Pexpect does not
use C, Expect, or TCL extensions. It should work on any platform that supports
the standard Python pty module. The Pexpect interface focuses on ease of use so
that simple tasks are easy.

There are two main interfaces to Pexpect — the function, run() and the class,
spawn. You can call the run() function to execute a command and return the
output. This is a handy replacement for os.system().
I was failed when using spawn() to get output of remote ssh command,

#!/usr/bin/env python
# -*- coding: utf-8 -*-
# filename: pexpect_2hei_spawn.py

import pexpect
if __name__ == ‘__main__’:

child = pexpect.spawn(‘ssh -oStrictHostKeyChecking=no myname@host.example.com’)
child.expect (‘Password:’)
child.sendline (mypassword)
child.expect (‘$’)
child.sendline (‘hostname’)
print child.before
print child.after

Function run() worked for me,that’s great!

#!/usr/bin/env python
# -*- coding: utf-8 -*-
# filename: pexpect_2hei.py

from pexpect import *

if __name__ == ‘__main__’:

User = ‘myuser’
Host = ‘www.2hei.net’
Pwd = ‘mypwd’
print run (“ssh -oStrictHostKeyChecking=no ” +User+”@”+Host+” ‘hostname;uptime'”, events={‘(?i)password’:Pwd+’\n’})

分类: python 标签:

中国移动之”大云” — CHINA MOBILE’S BIGCLOUD

2012年7月30日 评论已被关闭

中国移动的“大云”弹性计算系统(BC-EC)包括三部分:

1)web portal组件为用户提供自助服务的通道;
2)Front-end管理服务组件提供系统服务,应用,系统操作,用户管理,账单服务及数据库等;
3)Front-end组件提供和Qzone类似的功能来管理OpenNebula的back-ends服务。

中国移动近期准备发布基于BC-EC方案的public的云服务:包含1000+台severs,其中的700台提供服务器的虚拟环境,另外300台服务器为大云提供云存储空间。
相关概念:

  • 移动“大云”:

大云是中国移动研究院集成并扩展了多个云计算相关的开源软件和自主研发的部分功能组件的产品,在大云中主要采用了 Hadoop 分布式计算软件和 OpenNebula 弹性计算管理系统,虚拟机使用的是 Xen。在大云的开发过程中,移动对 Hadoop 进行了一些改进,并集成了一些附加功能,目前这些修改大都在逐步开放源代码。 阅读全文…

分类: cloud, IaaS 标签:

EC2推出高I/O Instance

2012年7月27日 评论已被关闭

随着google和微软的加入,IaaS领域竞争日趋激烈,最近发现EC2新增加了一种适用于数据库应用的需要高I/O性能的instance,应用场景包括像cassandra和MongoDB等NoSQL数据库。该instance使用了固态硬盘(SSD),同时还提供了高性能的CPU,大内存和优良的网络性能。

高I/O性能instance的具体参数如下:

· 60.5 GB of memory
· 35 EC2 Compute Units (8 virtual cores with 4.4 EC2 Compute Units each)
· 2 SSD-based volumes each with 1024 GB of instance storage
· 64-bit platform
· I/O Performance: 10 Gigabit Ethernet
· API name: hi1.4xlarge

高I/O instance使用Linux半虚拟化(PV)AMIs,可以提供超过12万4KB的随机读取(IOPS)和10000到85000之间4KB随机写入(IOPS)(取决于活动的逻辑块寻址宽度);对于使用硬件虚拟机(HVM)和Windows AMIs,其性能可达到9万次4KB的随机读取(IOPS)和9000到75000之间4KB的随机写入(IOPS)。
在所有AMIs类型中(Linux PV,Linux HVM, Windows)其每秒最大连续吞吐量大约是2GB读取和1.1GB的写入。不过对于使用Microsoft Windows Server的客户,高I/O的Instance只支持微软的Windows服务器(AMIs)群集实例类型。

详情可见:

Amazon EC2 Instance Types:  http://aws.amazon.com/ec2/instance-types/

Amazon Web Services Blog:  http://aws.typepad.com/aws/2012/07/new-high-io-ec2-instance-type-hi14xlarge.html

分类: cloud, IaaS 标签: ,

Setup Multi Hbase master on Hadoop Cluster

2012年7月24日 评论已被关闭

Hbase

        Setup Multi Hbase master on Hadoop Cluster to avoid single point failure. When active master failed/down for some reason exceed timeout we expected, backup master will be active and take over the role of master, see the value of zookeeper.session.timeout.

$cat /usr/lib/hbase/conf/hbase-site.xml
<name>zookeeper.session.timeout</name>
<value>180000</value>
<description>ZooKeeper session timeout. HBase passes this to the zk quorum as suggested maximum time for a session.In milliseconds.</description>
</property>

  • Test scenarios:

hadoop01.cluster.2hei.net (Hbase master)
hadoop02.cluster.2hei.net (Hbase backup master)

阅读全文…

分类: hbase 标签: ,

换掉Movable Type,启用Worldpress

2012年7月23日 1 条评论

PK  WorldPressMoveable Type

 

博客久未更新,今天来个大动作,下决心换掉Movable Type,启用Worldpress,至于替换的原因有很多,比如MT的模板不很给力、垃圾评论太多,再就是人之固有的喜新厌旧等等~。 目前暂不打算把MT里面旧的文件导过来,只是保留了MT的链接到WP。

一切从头,更换了theme和plugin, 此刻心情大好!

绳命,是剁么的回晃;人生,是入刺的井猜!

 

分类: others 标签:

hbase Daughter regiondir does not exist

2012年5月14日 评论已被关闭
Hbase master logs:
12/05/14 13:33:44 INFO master.LoadBalancer: Skipping load balancing.  servers=10 regions=261 average=26.1 mostloaded=27 leastloaded=26
12/05/14 13:33:44 WARN master.CatalogJanitor: Daughter regiondir does not exist: hdfs://2hei.net:8020/hbase/RecSys_Catalog/7d100af9ac714de605efc9da89a817b3
12/05/14 13:33:44 WARN master.CatalogJanitor: Daughter regiondir does not exist: hdfs://2hei.net:8020/hbase/Track/d87d503a2996200cfa3aae8906767f81
12/05/14 13:33:44 WARN master.CatalogJanitor: Daughter regiondir does not exist: hdfs://2hei.net:8020/hbase/type_subgenre_uniqueId_CatalogIndex/4602f165ca87f345a1f62a48c5677e55
Resons:
May caused by I restart zookeeper first, then stop master and regionserver. when restart Hbase master,it filed init region servers.
other resons:
– region server crashed
– lease timed out
– master starts recovery (can take quite a while to complete)
– region server restarts
– region server sends region server startup message to master
– master waits in rpc handler for old server cleanup (because it
cannot differentiate the new instance from the old).
– ipc from region server to master times out
– region server sends a new startup message. The master thread starts
waiting in the rpc handler for old server cleanup.
– ipc from region server to master times out
Can’t find below folders in HDFS
hadoop fs -ls /hbase/RecSys_Catalog/7d100af9ac714de605efc9da89a817b3
hadoop fs -ls /hbase/type_subgenre_uniqueId_CatalogIndex/4602f165ca87f345a1f62a48c5677e55
hadoop fs -ls /hbase/Track/d87d503a2996200cfa3aae8906767f81
Resolve:
find key from .META., delete them
echo “scan ‘.META.'” | hbase shell|grep 7d100af9ac714de605efc9da89a817b3
echo “scan ‘.META.'” | hbase shell|grep 4602f165ca87f345a1f62a48c5677e55
echo “scan ‘.META.'” | hbase shell|grep d87d503a2996200cfa3aae8906767f81
scan ‘.META.’, {COLUMNS => ‘info:splitA’,TIMESTAMP => 1335245847237}
delete ‘.META.’, ‘Track,,1335245609463.1e40dbd0fb394c05fdaf30ca5f933ea8.’,’info:splitA’ 
delete ‘.META.’, ‘RecSys_Catalog,,1336785348151.1250fbc6334c578629f95113b2a3ba7b.’,’info:splitA’ 
delete ‘.META.’, ‘type_subgenre_uniqueId_CatalogIndex,,1336785356835.d4af2bae84156905367579bc44fcdd97.’,’info:splitA’ 
Hbase logs back to normal:
12/05/14 14:18:44 INFO master.LoadBalancer: Skipping load balancing.  servers=10 regions=261 average=26.1 mostloaded=27 leastloaded=26
12/05/14 14:23:44 INFO master.LoadBalancer: Skipping load balancing.  servers=10 regions=261 average=26.1 mostloaded=27 leastloaded=26
12/05/14 14:28:45 INFO master.LoadBalancer: Skipping load balancing.  servers=10 regions=261 average=26.1 mostloaded=27 leastloaded=26
分类: hbase 标签: