If you show a message
"WARN util.NativeCodeLoader: Unable to load native-hadoop library for
your platform."
you need to build hadoop source
1. download hadoop source
2. edit pom.xml
https://issues.apache.org/jira/browse/HADOOP-10110
3. build hadoop
mvn package -Pnative -DskipTests
4. copy libhadoop.so.1.0.0
cp hadoop-common-project/hadoop-common/target/native/target/usr/local/lib/libhadoop.so.1.0.0 /usr/local/hadoop/lib/native/.
KotelTech
Monday, December 30, 2013
Thursday, December 26, 2013
MySQL Replication Master Server Change without stopping
How to change master server without stop when we have Server Old-Master, Slave, New-Master
I'm not an expert of database administration. If there are better way, share me please.
I. Database no-stop server change
1. New-Master Server Setting
Setup New-Master as slave, also master
1) edit my.cnf
2) Backup data from Old-Master and restore to New-Master by using mysqldump or other utilities.
3) start New-Master as slave
5) stop slave
6) Backup data from Old-Master and restore to New-Master again (for changes)
choose alternative methods.
6-1) dump all data
7) start slave
8) Create new master replication users for slave and grant them.
2. Slave Servers
change master to New-Master dynamically, also change my.cnf
3. Change your application setting.
4. Stop Old-Master server
II. Database stop, easy to change
First, Setup New-Master server (I-1)
Second, shutdown old-master Server
Third, change new-master address to old-master server
Finally, Bring up New-Master Server
I'm not an expert of database administration. If there are better way, share me please.
I. Database no-stop server change
1. New-Master Server Setting
Setup New-Master as slave, also master
1) edit my.cnf
server-id=newidlog_bin=mysql-bin#This option should be enabled for passing replicated data from old-master to slavelog_save_updates=true
2) Backup data from Old-Master and restore to New-Master by using mysqldump or other utilities.
mysqldump -h... -u... -p... --hex-blob --routines --triggers --all-databases | gzip > backup.sql
3) start New-Master as slave
mysql-shell) stop slave; mysql-shell) CHANGE MASTER TO MASTER_HOST = 'Old-Master-Host', MASTER_USER = 'old-master-user', MASTER_PASSWORD = 'old-master-password', MASTER_LOG_FILE = 'old-master-logfile', MASTER_LOG_POS = old-master-pos; mysql-shell) start slave;4) check status
mysql-shell) show slave status\G;
5) stop slave
6) Backup data from Old-Master and restore to New-Master again (for changes)
choose alternative methods.
6-1) dump all data
6-2) dump partial data with mysqlbinlogmysqldump -h... -u... -p... --hex-blob --routines --triggers --all-databases --insert-ignore > backup.sql
mysqlbinlog binlog.lastlog > backup-partial.sql mysql -uroot -p -e "source backup-partial.sql* if there are duplicate key problem
sed -e "s/^INSERT INTO/INSERT IGNORE INTO/" < backup-partial.sql > convert-insertinto.sql
8) Create new master replication users for slave and grant them.
create user 'replicator'@'%' identified by 'password';
grant replication slave on *.* to 'replicator'@'%';
9) flush logs
change master to New-Master dynamically, also change my.cnf
mysql-shell) stop slave; mysql-shell) CHANGE MASTER TO MASTER_HOST = 'New-Master-Host', MASTER_USER = 'New-master-user', MASTER_PASSWORD = 'New-master-password', MASTER_LOG_FILE = 'New-master-logfile', MASTER_LOG_POS = New-master-pos; mysql-shell) start slave;
3. Change your application setting.
4. Stop Old-Master server
II. Database stop, easy to change
First, Setup New-Master server (I-1)
Second, shutdown old-master Server
Third, change new-master address to old-master server
Finally, Bring up New-Master Server
Monday, December 2, 2013
Gitlab Install, Slackware 13.37 or 14.1
Gitlab install, Slackware 13.37, 14.1
copy files
or make initial script
Git (Under 1.7.10)
Download and compile from source
Ruby
System User
Database (MySQL/MariaDB)
Install Redis
Download and compile from source
wget http://download.redis.io/redis-stable.tar.gz
tar xvzf redis-stable.tar.gz
cd redis-stable
make
copy files
cd srcedit initial script
sudo cp redis-server /usr/local/bin/
sudo cp redis-cli /usr/local/bin/
editor /etc/rc.d/rc.local
or make initial script
Git (Under 1.7.10)
Download and compile from source
cd /tmp
curl --progress https://git-core.googlecode.com/files/git-1.8.4.1.tar.gz | tar xz
cd git-1.8.4.1/
make prefix=/usr/local all
Install into /usr/local/bin
sudo make prefix=/usr/local installWhen editing config/gitlab.yml (Step 6), change the git bin_path to /usr/local/bin/git
Ruby
mkdir /tmp/ruby && cd /tmp/ruby
curl --progress ftp://ftp.ruby-lang.org/pub/ruby/2.0/ruby-2.0.0-p353.tar.gz | tar xz
cd ruby-2.0.0-p353
./configure --prefix=/usr/local/
make && make install
Check installed path
login aginwhich ruby
ruby -v
gem install bundler --no-ri --no-rdoc
System User
useradd --system --shell /bin/bash --comment 'GitLab' --create-home --home /home/git/ git
check shadow file
cat /etc/shadow |grep git
git:*:::::::
if it shows git:!::::::: change to git:*:::::::
Git-Shell (v 1.7.9)
su gitedit config.yml
git clone https://github.com/gitlabhq/gitlab-shell.git
cd gitlab-shell
git checkout v1.7.9
cp config.yml.example config.yml
./bin/install
Database (MySQL/MariaDB)
CREATE USER 'gitlab'@'localhost' IDENTIFIED BY 'supersecret';
CREATE DATABASE IF NOT EXISTS `gitlabhq_production` DEFAULT CHARACTER SET `utf8` COLLATE `utf8_unicode_ci`;
GRANT SELECT, LOCK TABLES, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER ON `gitlabhq_production`.* TO 'gitlab'@'localhost'
Gitlab (v6.3)
su - git
Clone GitLab repository
git clone https://github.com/gitlabhq/gitlabhq.git gitlab
Go to gitlab directory
cd /home/git/gitlab
Checkout to stable release
git checkout 6-3-stableSetup files
cp config/gitlab.yml.example config/gitlab.yml
chown -R git log/
chown -R git tmp/
chmod -R u+rwX log/
chmod -R u+rwX tmp/
mkdir tmp/pids/
mkdir tmp/sockets/
chmod -R u+rwX tmp/pids/
chmod -R u+rwX tmp/sockets/
mkdir public/uploads
chmod -R u+rwX public/uploads
cp config/unicorn.rb.example config/unicorn.rb
git config --global user.name "GitLab"
git config --global user.email "gitlab@your_domain_name"
git config --global core.autocrlf input
cp database.yml.mysql database.yml
chmod o-rwx config/database.yml
gem install charlock_holmes --version '0.6.9.4'
bundle install --deployment --without development test postgres puma aws
bundle exec rake gitlab:setup RAILS_ENV=production
Tuesday, November 26, 2013
Gitlab, Redmine Install with Maria DB in subdirectory, Ubuntu 12.04LTS
Gitlab, Redmine Install with Maria DB in subdirectory, Ubuntu 12.04LTS
1. Maria DB
install packages
sudo apt-get install libmariadbclient-dev libmariadbclient18 mariadb-client-5.5 mariadb-client-core-5.5 mariadb-common mariadb-server mariadb-server-5.5 mariadb-server-core-5.5
if there are dependency problem, you may specify the version of maria db
sudo apt-get install libmysqlclient18=5.5.30-mariadb1~precise mysql-common=5.5.30-mariadb1~precise
sudo apt-get install libmysqlclient18=5.5.33a+maria-1~precise mysql-common=5.5.33a+maria-1~precise
2. Gitlab - Subdirectory
first, follow gitlab.org instruction.Modify 3 files in gitlab home directory
# 1) In your application.rb file: config.relative_url_root = "/gitlab"
# 2) In your gitlab.yml file: relative_url_root: /gitlab
# 3) In your unicorn.rb: ENV['RAILS_RELATIVE_URL_ROOT'] = "/gitlab"
modify gitlab-shell/config.yml
gitlab_url: "http://localhost/gitlab/"
nginx conf
upstream gitlab {
server unix:/home/git/gitlab/tmp/sockets/gitlab.socket;
}
location /gitlab {
proxy_read_timeout 300; # https://github.com/gitlabhq/gitlabhq/issues/694
proxy_connect_timeout 300; # https://github.com/gitlabhq/gitlabhq/issues/694
proxy_redirect off;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_pass http://gitlab;
}
3. Redmine - Subdirectory
install dependent packagessudo install imagemagick libmagickwand-devfollow Redmine install instruction
install unicorn
gem install unicorn
downloads config file
curl -o config/unicorn.rb https://raw.github.com/defunkt/unicorn/master/examples/unicorn.conf.rb
modify config
config/unicorn.rb
working_directory /home/redmine
listen "/home/redmine/tmp/sockets/.unicorn.sock", :backlog => 64
pid "/home/redmine/tmp/pids/unicorn.pid"
config/environment.rb
Redmine::Utils::relative_url_root = "/redmine"
nginx conf
upstream redmine {
server unix:/home/redmine/tmp/sockets/redmine.socket;
}
location /redmine {
proxy_read_timeout 300;
proxy_connect_timeout 300;
proxy_redirect off;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_pass http://redmine;
}
Subscribe to:
Posts (Atom)