If you show a message
"WARN util.NativeCodeLoader: Unable to load native-hadoop library for
your platform."
you need to build hadoop source
1. download hadoop source
2. edit pom.xml
https://issues.apache.org/jira/browse/HADOOP-10110
3. build hadoop
mvn package -Pnative -DskipTests
4. copy libhadoop.so.1.0.0
cp hadoop-common-project/hadoop-common/target/native/target/usr/local/lib/libhadoop.so.1.0.0 /usr/local/hadoop/lib/native/.
Monday, December 30, 2013
Thursday, December 26, 2013
MySQL Replication Master Server Change without stopping
How to change master server without stop when we have Server Old-Master, Slave, New-Master
I'm not an expert of database administration. If there are better way, share me please.
I. Database no-stop server change
1. New-Master Server Setting
Setup New-Master as slave, also master
1) edit my.cnf
2) Backup data from Old-Master and restore to New-Master by using mysqldump or other utilities.
3) start New-Master as slave
5) stop slave
6) Backup data from Old-Master and restore to New-Master again (for changes)
choose alternative methods.
6-1) dump all data
7) start slave
8) Create new master replication users for slave and grant them.
2. Slave Servers
change master to New-Master dynamically, also change my.cnf
3. Change your application setting.
4. Stop Old-Master server
II. Database stop, easy to change
First, Setup New-Master server (I-1)
Second, shutdown old-master Server
Third, change new-master address to old-master server
Finally, Bring up New-Master Server
I'm not an expert of database administration. If there are better way, share me please.
I. Database no-stop server change
1. New-Master Server Setting
Setup New-Master as slave, also master
1) edit my.cnf
server-id=newidlog_bin=mysql-bin#This option should be enabled for passing replicated data from old-master to slavelog_save_updates=true
2) Backup data from Old-Master and restore to New-Master by using mysqldump or other utilities.
mysqldump -h... -u... -p... --hex-blob --routines --triggers --all-databases | gzip > backup.sql
3) start New-Master as slave
mysql-shell) stop slave; mysql-shell) CHANGE MASTER TO MASTER_HOST = 'Old-Master-Host', MASTER_USER = 'old-master-user', MASTER_PASSWORD = 'old-master-password', MASTER_LOG_FILE = 'old-master-logfile', MASTER_LOG_POS = old-master-pos; mysql-shell) start slave;4) check status
mysql-shell) show slave status\G;
5) stop slave
6) Backup data from Old-Master and restore to New-Master again (for changes)
choose alternative methods.
6-1) dump all data
6-2) dump partial data with mysqlbinlogmysqldump -h... -u... -p... --hex-blob --routines --triggers --all-databases --insert-ignore > backup.sql
mysqlbinlog binlog.lastlog > backup-partial.sql mysql -uroot -p -e "source backup-partial.sql* if there are duplicate key problem
sed -e "s/^INSERT INTO/INSERT IGNORE INTO/" < backup-partial.sql > convert-insertinto.sql
8) Create new master replication users for slave and grant them.
create user 'replicator'@'%' identified by 'password';
grant replication slave on *.* to 'replicator'@'%';
9) flush logs
change master to New-Master dynamically, also change my.cnf
mysql-shell) stop slave; mysql-shell) CHANGE MASTER TO MASTER_HOST = 'New-Master-Host', MASTER_USER = 'New-master-user', MASTER_PASSWORD = 'New-master-password', MASTER_LOG_FILE = 'New-master-logfile', MASTER_LOG_POS = New-master-pos; mysql-shell) start slave;
3. Change your application setting.
4. Stop Old-Master server
II. Database stop, easy to change
First, Setup New-Master server (I-1)
Second, shutdown old-master Server
Third, change new-master address to old-master server
Finally, Bring up New-Master Server
Monday, December 2, 2013
Gitlab Install, Slackware 13.37 or 14.1
Gitlab install, Slackware 13.37, 14.1
copy files
or make initial script
Git (Under 1.7.10)
Download and compile from source
Ruby
System User
Database (MySQL/MariaDB)
Install Redis
Download and compile from source
wget http://download.redis.io/redis-stable.tar.gz
tar xvzf redis-stable.tar.gz
cd redis-stable
make
copy files
cd srcedit initial script
sudo cp redis-server /usr/local/bin/
sudo cp redis-cli /usr/local/bin/
editor /etc/rc.d/rc.local
or make initial script
Git (Under 1.7.10)
Download and compile from source
cd /tmp
curl --progress https://git-core.googlecode.com/files/git-1.8.4.1.tar.gz | tar xz
cd git-1.8.4.1/
make prefix=/usr/local all
Install into /usr/local/bin
sudo make prefix=/usr/local installWhen editing config/gitlab.yml (Step 6), change the git bin_path to /usr/local/bin/git
Ruby
mkdir /tmp/ruby && cd /tmp/ruby
curl --progress ftp://ftp.ruby-lang.org/pub/ruby/2.0/ruby-2.0.0-p353.tar.gz | tar xz
cd ruby-2.0.0-p353
./configure --prefix=/usr/local/
make && make install
Check installed path
login aginwhich ruby
ruby -v
gem install bundler --no-ri --no-rdoc
System User
useradd --system --shell /bin/bash --comment 'GitLab' --create-home --home /home/git/ git
check shadow file
cat /etc/shadow |grep git
git:*:::::::
if it shows git:!::::::: change to git:*:::::::
Git-Shell (v 1.7.9)
su gitedit config.yml
git clone https://github.com/gitlabhq/gitlab-shell.git
cd gitlab-shell
git checkout v1.7.9
cp config.yml.example config.yml
./bin/install
Database (MySQL/MariaDB)
CREATE USER 'gitlab'@'localhost' IDENTIFIED BY 'supersecret';
CREATE DATABASE IF NOT EXISTS `gitlabhq_production` DEFAULT CHARACTER SET `utf8` COLLATE `utf8_unicode_ci`;
GRANT SELECT, LOCK TABLES, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER ON `gitlabhq_production`.* TO 'gitlab'@'localhost'
Gitlab (v6.3)
su - git
Clone GitLab repository
git clone https://github.com/gitlabhq/gitlabhq.git gitlab
Go to gitlab directory
cd /home/git/gitlab
Checkout to stable release
git checkout 6-3-stableSetup files
cp config/gitlab.yml.example config/gitlab.yml
chown -R git log/
chown -R git tmp/
chmod -R u+rwX log/
chmod -R u+rwX tmp/
mkdir tmp/pids/
mkdir tmp/sockets/
chmod -R u+rwX tmp/pids/
chmod -R u+rwX tmp/sockets/
mkdir public/uploads
chmod -R u+rwX public/uploads
cp config/unicorn.rb.example config/unicorn.rb
git config --global user.name "GitLab"
git config --global user.email "gitlab@your_domain_name"
git config --global core.autocrlf input
cp database.yml.mysql database.yml
chmod o-rwx config/database.yml
gem install charlock_holmes --version '0.6.9.4'
bundle install --deployment --without development test postgres puma aws
bundle exec rake gitlab:setup RAILS_ENV=production
Subscribe to:
Posts (Atom)