September 11th, 2009
With the Sharesight test suite (ruby/rails/mysql && rake test), i tested the performance of my the new MBP on Snow Leopard.
I save 16 seconds per run – its going to take a while to pay for itself…. But about 22% faster, which is not too bad.
Macbook Pro (March 06) 1.83GHz 2G Ram
real 1m29.464s
user 0m50.348s
sys 0m5.812s
Macbook Pro (Sep 09) 2.53 GHz 4G Ram
real 1m13.450s
user 0m43.662s
sys 0m3.280s
Posted in apple, rails | No Comments »
August 5th, 2009
I recently created a couple of Centos virtual machines, and took notes of all the steps. You can use these notes for both installing another VM, or for building a production server.
We used these VMs for running staging and production instances of a Ruby on Rails deployment.
Install Centos 5.3 64 bit
VM configuration settings
- Redhat Linux Enterprise 5 64 bit
- 30 GB drive
- 2 CPU
- 1 GB memory
- Network: bridged
Server install settings:
- use default settings for drive partitions
- set network address to: IP_ADDRESS_GOES_HERE IP version 4
- hostname: HOSTNAME
- gateway: GATEWAY
- primary dns: DNS1
- disable IP version 6
- net mask: 255.255.255.0
- timezone: Pacific/Auckland
- System clock uses UTC: tick
- set root password to PASSWORD_YEAH_RIGHT
- Additional tasks: none
SETUP TASKS
- yum update
- yum install sudo
- adduser nigeluser
- passwd nigeluser
- adduser marcususer
- passwd marcususer
visudo
- add lines at bottom of file:
nigeluser ALL=(ALL) ALL
marcususer ALL=(ALL) ALL
PACKAGES
- sudo yum install ruby
- wget http://rubyforge.org/frs/download.php/58677/ruby-enterprise-1.8.6-20090610.tar.gz
- tar xvfz ruby-enterprise-1.8.6-20090610.tar.gz
- sudo yum install httpd
- sudo yum install gcc-c++
- sudo yum install zlib-devel
- sudo yum install openssl-devel
- sudo yum install readline-devel
- sudo ./installer (ruby tar directory)
- sudo ln -s /opt/ruby-enterprise/bin/cap /usr/bin/cap
- sudo ln -s /opt/ruby-enterprise/bin/capify /usr/bin/capify
- sudo ln -s /opt/ruby-enterprise/bin/erb /usr/bin/erb
- sudo ln -s /opt/ruby-enterprise/bin/gem /usr/bin/gem
- sudo ln -s /opt/ruby-enterprise/bin/integrity /usr/bin/integrity
- sudo ln -s /opt/ruby-enterprise/bin/irb /usr/bin/irb
- sudo ln -s /opt/ruby-enterprise/bin/rails /usr/bin/rails
- sudo ln -s /opt/ruby-enterprise/bin/rake /usr/bin/rake
- sudo ln -s /opt/ruby-enterprise/bin/rdoc /usr/bin/rdoc
- sudo ln -s /opt/ruby-enterprise/bin/ruby /usr/bin/ruby
- sudo yum remove ruby
- sudo yum install httpd-devel apr-devel
- sudo yum install mysql-devel
- sudo /opt/ruby-enterprise/bin/passenger-install-apache2-module
- sudo yum install mysql-server
- sudo /sbin/chkconfig httpd on
- sudo /sbin/chkconfig mysqld on
- sudo yum install ncurses-devel
- wget http://downloads.sourceforge.net/sourceforge/htop/htop-0.8.3.tar.gz?use_mirror=transact
- tar xcfz htop-0.8.3.tar.gz
- cd htop-0.8.3
- ./configure
- make
- sudo make install
- wget http://kernel.org/pub/software/scm/git/git-1.6.3.3.tar.bz2
- bunzip2 files/git-1.6.3.3.tar.bz2
- tar xvf files/git-1.6.3.3.tar
- ./configure
- make
- sudo make install
- sudo gem install mysql –no-rdoc –no-ri — –with-mysql-config=/usr/bin/mysql_config
START APPS
- copy /system/mysql/my.cnf to /etc/my.cnf
- sudo /sbin/service mysqld start
- /usr/bin/mysqladmin -u root password ‘PASSWORD_GOES_HERE’
- sudo /sbin/service httpd start
CAPISTRANO
- cap staging deploy:setup
- cap staging deploy:cold
- cap staging deploy
CONFIGURE
- sudo /usr/sbin/adduser deploy
- sudo passwd deploy
VMWARE TOOLS
- Login as root
- Select from VMware Workstation: go to VM> Install VMware Tools
- mkdir /root/mn
- From the VM: mount the virtual cd drive , this should be done with the guest OS (CentOS ) command line:
mount /dev/cdrom /root/mn
- From the CDROM extract VMware Tools into /tmp/ directory
tar -C /tmp -zxvf /root/mn/VMwareTools-5.5.3-34685.tar.gz
- Running VMware Tools installation/setup script
cd /tmp/vmware-tools-distrib
./vmware-install.pl
- This is VMware Tools installation/setup script, it will ask for confirmation to begin the installation
answer the defaults for each question
- when done, restart the VM (shutdown -r now)
Tags: centos, mac, mysql, passenger, ruby on rails, vmware
Posted in linux | No Comments »
July 15th, 2009
# first remove your copy and paste plugin….
git rm -r vendor/plugins/xxxxxx
git commit
now you are ready to add the submodule
git submodule add git://your_git_repo/xxxxx.git vendor/plugins/xxxxx
git commit
To load the submodule, you need to
git submodule init
git submodule update
To deploy submodules using capistrano, capistrano needs to also do the submodule init and update.
# in deploy.rb
set :git_enable_submodules,1
if you want to update the version of the submodule – or the branch
cd into the plugin, git pull or git checkout branch
then cd back to the containing project and git commit
Tags: git rails
Posted in rails | 4 Comments »
June 21st, 2009
We have a situation where we want one domain which serves a rails app at root and also has a wordpress blog served within the main site.
yours.com/ --> served by rails
yours.com/blog --> served by php
To configure this in apache,
The key is the
PassengerEnabled – else passenger will try and serve the php content.
Posted in rails | 3 Comments »
June 18th, 2009
This is quite trivial and very easy. I wanted to use the same version of ruby on my mac as is running on our production servers (Ruby Enterprise Edition).
Firstly, do a
gem list
and keep this information. You will need to reinstall all your gems once you have successfully installed REE.
Download REE from http://www.rubyenterpriseedition.com/
tar xvfz ruby-enterprise-X.X.X
sudo ./ruby-enterprise-X.X.X/installer
I suggest changing the default location to just /opt/ruby-enterprise – as you are unlikely to need to roll back the symbolic links in your development environment.
Once it is installed, then you need to make sure that you use it for all your gems, ruby, rake tasks. To do this we will sym link in all the executables into /usr/bin
Find any previous versions of ruby
whereis ruby
and
which ruby
Cleanup – backup and remove any old versions and the associated ruby, gems, rake, irb, cap, capify, erb so they are not on your path.
Then sym link from /usr/bin to opt/ruby-enterprise/bin with all the executables that you may require….
sudo ln -s -f /opt/ruby-enterprise/bin/ruby /usr/bin/ruby
sudo ln -s -f /opt/ruby-enterprise/bin/cap /usr/bin/cap
sudo ln -s -f /opt/ruby-enterprise/bin/capify /usr/bin/capify
sudo ln -s -f /opt/ruby-enterprise/bin/erb /usr/bin/erb
sudo ln -s -f /opt/ruby-enterprise/bin/gem /usr/bin/gem
sudo ln -s -f /opt/ruby-enterprise/bin/irb /usr/bin/irb
sudo ln -s -f /opt/ruby-enterprise/bin/rails /usr/bin/rails
sudo ln -s -f /opt/ruby-enterprise/bin/rake /usr/bin/rake
sudo ln -s -f /opt/ruby-enterprise/bin/rdoc /usr/bin/rdoc
Do a gem list to see what ruby-enterprise came bundled with.
Now you will need to install all your gems again – that you noted earlier.
Tags: rails
Posted in rails | No Comments »
June 4th, 2009
Just migrated a legacy database into a rails compliant database – and wrote this nifty DSL script to help out….
The rake task generates the Hash data for the table creation, data insert and rollback scripts.
Once you run the rake task, paste the output into the migration modify the update table and column names – then run the migration.
Posted in rails | 1 Comment »
May 7th, 2009
I just recently spent some time with a client investigating some performance issues with mysql subqueries.
The obviously contrived example below
Depending on data size the query can run very slow. When running two queries and pasting in the actual IDs the query runs extremely fast.
The good news it is looks like it is fixed in version 6.0 - looking forward to it becoming a production release....
Posted in mysql | No Comments »
April 7th, 2009
I have written a Javascript routine which displays a list of the available access keys at the bottom of the page.
Maintaining a list of access keys used for each individual page was going to be a tiresome exercise. As Javascript can interrogate a page for this information, it seemed a little silly manually creating the same result.
So, I give you access_keys, an unobtrusive Javascript routine that searches for and displays access keys that it finds at the bottom of each webpage. Now available on GitHub with full instructions.
Thanks go to Radio New Zealand for inspiring this effort and then releasing it to the world.
Posted in javascript | 1 Comment »
April 6th, 2009
Dual master active-passive is a good way to go for a highly available database server. I can offer the following tips for a more painless dual master experience:
- Make sure all your tables are INNODB – Fixing a broken replication with any MyISAM tables will cause a good deal of pain – or limit you with unnecessary downtime from your production environment.
- If possible, replicate from the beginning. Install your DB’s, configure for replication, create replication user acess rights, then restart your mysql instances. All databases and user rights from that point on will be replicated. If you can’t do this then take a snapshot of the ‘active’ master, and create your passive master from the snapshot. See below for obtaining a consistent snapshot.
- Use log-slave-updates so that replicated data that is processed in master-master is also sent to any slaves. If you are replicating of either of your masters for backup, then set this on the master that is also replicating out to secondary slaves – or set on both so you have the freedom to change topologies.
- Ensure that max_allowed_packet is consistent accross all master slaves. If you don’t you may experience the Slave IO thread simply stopping with no error message, and NULL seconds behind error
- Use sync_binlog=1 so that transactions are written out to the bin logs on each write – relying on the O/S to flush the logs is not satisfactory and wont guard you in the event of a lost server or host. Make sure innodb_flush_log_at_trx_commit is set to 1 (the default)
- If you are not replicating all your databases, be aware that the inserts into one database that is not replicated to a database that is replicated – INSERT into replicated_db_name.tablename will not replicate the insert. This can be used to your advantage in a dual replication set up if you need to fix replication errors – but i would still suggest the snapshot approach below
- Reset slave is not an option – I could not see this as a way to fix a broken slave. You will end up loosing data. Best refer to the snapshot method
- Nagios has some great mysql replication event handlers to warn you in advance of a broken replication setup. You will need this in your production environment – or some other method alerting you of a broken replication
- If your replication is broken – make sure no database writes can make it to the passive database. If this happens you may be losing data – and at the least be in for a whole lot of hurt when putting it back together.
- use
show slave statusG. If you have errors, you can attempt to fix but in dual master remember that your changes will be replicated back to the other masters – and may cause conflicting errors. Use the snapshot method
- Use the
auto_increment_offset and auto_increment_increment to configure your primary key allocation method so odd primary keys are generated on one master, and even keys are generated on the other.
- You can use mysql proxy or a IP load balancer to direct your SQL requests to the active master. If/when your server crashes, the passive master will be up to date and pick up to become the new active master – with no down time – yeah!
The Snapshot Script
export options=' --add-drop-database --triggers --routines --single-transaction --master-data=2 -u mysqluser -p '
export datestamp=`date +%d-%m-%Y`
mysqldump $options --databases database1 database2 database3 > snapshot_$datestamp.sql
The master-data=2 option will create the necessary information to point your new slave at the master and get replication running again – and place it commented out in the first few lines of the file. Use head -40 snapshot.sql to see what it is if your SQL file is large.
The single-transaction option will mean the the database snapshot will be consistent from beginning to end (Innodb only!).
You are pretty safe to run this on a live production server.
You will need to snapshot all the databases that you are replicating in the one command or the master log postion will not be consistent – it is global for all databases.
Once you have the snapshot. Issue a stop slave; on the passive master, load in the database, issue the change master command below, then issue a start slave;
CHANGE MASTER TO
MASTER_HOST='masterXXXXX',
MASTER_USER='mysql_repli_user',
MASTER_PASSWORD='xxxxxxxxx',
MASTER_LOG_FILE='mysql-bin.xxxxxxx',
MASTER_LOG_POS=xxxxxxxxx;
Posted in mysql | 1 Comment »
March 6th, 2009
I had been getting some validation warnings when running some of my specs. The W3 validator confirmed that my xHTML was valid.
After some googling of the problem, it turns out there is a difference between how xhtml and html documents are handled. Essentially, you need to tell the parser that the document is xHTML rather than HTML. More details here, here, and here.
The solution is to tell RSpec that the document is XML based. The options given in the links above work for RUnit based tests. Taking a look at the rspec source, it is checking to see if the response is XML based by noting the Content-Type header record. So, here's the solution I used:
Now, when I run my specs, it correctly parses the response as XHTML and passes validation without any warnings.