Linux

You are currently browsing articles tagged Linux.

VPS hosting is available, affordable, and just a little bit scary.  In order to alleviate some of risk that is taken when moving to a VPS that is not under our benevolent control, we need to set up a reliable data backup solution.  The setup that I incorporated involves a server in the cloud backing up to my box.com account.  To accomplish this on CentOS 6, you can just use the following commands in a script that is executed periodically:

 

#!/bin/bash
HOME=/root
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
echo “Files backup to cloud”
eval $(gpg-agent –daemon)
export SIGN_PASSPHRASE=””
export PASSPHRASE=””
export FTP_PASSWORD=”your password”
echo “Files backup to cloud”
duplicity –use-agent –encrypt-key YOUR_ENCRYPTION_KEY –full-if-older-than 4M /var/spool/duplicity/ webdavs://YOUR_BOX_COM_ACCOUNT@dav.box.com/dav/lynetSky/duplicity && duplicity –use-agent –encrypt-key YOUR_ENCRYPTION_KEY remove-all-but-n-full 4 –force webdavs://YOUR_BOX_COM_ACCOUNT@dav.box.com/dav/lynetSky/duplicity && duplicity –use-agent –encrypt-key YOUR_ENCRYPTION_KEY remove-all-inc-of-but-n-full 2 –force webdavs://YOUR_BOX_COM_ACCOUNT@dav.box.com/dav/lynetSky/duplicity
echo “Database backup to cloud”
duplicity –use-agent –encrypt-key YOUR_ENCRYPTION_KEY –full-if-older-than 4M /var/spool/holland/ webdavs://YOUR_BOX_COM_ACCOUNT@dav.box.com/dav/lynetSky/holland && duplicity –use-agent –encrypt-key YOUR_ENCRYPTION_KEY remove-all-but-n-full 4 –force webdavs://YOUR_BOX_COM_ACCOUNT@dav.box.com/dav/lynetSky/holland && duplicity –use-agent –encrypt-key YOUR_ENCRYPTION_KEY remove-all-inc-of-but-n-full 2 –force webdavs://YOUR_BOX_COM_ACCOUNT@dav.box.com/dav/lynetSky/holland

 

These commands will instruct duplicity to make a full backup every four months (–full-if-older-than 4M).  When this script runs in the in-between times (say 2 months after the last full backup), it will create an incremental backup.  Duplicity will keep four full backups.  This is specified by the remove-all-but-n-full 4 directive in the command.  Specifying remove-all-inc-of-but-n-full 1 tells duplicity to remove all incremental updates except in the case of the last two backup set.  A backup set includes the last full backup and its incremental backups.  Yes, this is a bit complicated.  Yes, it is worth it.

The end result is this: in sixteen months, there will be four full backups (a new full backup is created every four months).  The newest two of these will have the incremental backups as well as the full backups.  This way, file history will be completely preserved for the most recent eight month period.  If we are really desperate, we can recover a file that twelve months old or sixteen months old.

We need to consider what happens at month seventeen.  We will have four full backups, the oldest full backup will be seventeen months old.  We should not expect to be able to recover files that are over sixteen months old.  When we continue on to our twentieth month  the oldest full backup will be removed and we will be back to only having sixteen months of recoverable data.  Always think of your maximum time to recover a file as (–full-if-older-than 4M * remove-all-but-n-full 4 == 4 months * 4 full backups = 16 months of full backups).

Storing four full copies of the data on a backup is (most likely) overkill for what I am doing.  These options were added when I decided that a full backup on another server lacked one thing: files that were deleted on the server were never deleted in the backup location.  Then, of course, my mind wondered into using rsync instead of duplicity.  It’s true, if all we need is a single backup of the files, rsync can provide a better solution (I do this with my personal photos and videos at home).  However, the point in time snapshot duplicity provides can be used for forensics and change tracking.  This solution as scripted above isn’t one size fits all.  Use the tools and settings that work the best for you.

This setup absolutely saved this site and the other sites I run.  The SAN that this VPS (at the time of this writing) is running on became corrupted and all of the data would’ve been lost had I not used an external site for backups.  If you are someone who is worried about the security of your backups, remember that duplicity has an automatic encryption algorithm built in.  Security of your data is there, just make sure to backup your keys so you can read the files.

To conclude: backups can save you important information; the cloud is a great place to store your backups; duplicity is a great tool that can automate this process.

Tags: , ,

Ben Martin’s articles have always been a great resource. In his latest article, Linux.com :: Benchmarking hardware RAID vs. Linux kernel software RAID, he makes a very informative comparison between hardware and software RAID.  It is a must read for the server admins out there.  While you’re at it, also check out his other articles.  They are all great reads.

This is my 1st test of WordPress 2.6’s Blog this Button.  It works, but I loose my information when I move from Photo to Text mode.

Tags: , , ,

OpenSuse 11.0 was released on June 19, 2008. Shortly thereafter, it was installed and running on my laptop.  OpenSuse has many draws, I’ll try and touch on the reasons why I chose it for my laptop and why it will stay on my laptop for the foreseeable future.

  • It has corporate backing from Novell, a company that seems quite determined to put Linux on the business desktop
  • It is less experimental than Fedora
  • The package management is quite good and faster than any other rpm based distribution
  • I wanted to grok KDE 4

OpenSuse Screenshot

KDE4

My KDE4 experience didn’t last long.  I really want to like KDE, but I’m just not productive in it.  This isn’t really the fault of the desktop environment as much as it is the applications.  In gnome, I am used to having all applications use a virtual file system so that media can be played over remote locations such as ssh shares.  I couldn’t find a way to do this in KDE with the available software.  I ended up installing gnome instead.

Gnome Applications

OpenSuse is unique than most other distributions with its application defaults.  Novell is actively developing gnome desktop applications that are based on mono.  Examples of this are the banshee media player, f-spot photo manager, and tomboy note taking application.  It includes these applications by default.

I believe f-spot to be the best photo manager that Linux currently has.  The features that I enjoy the most are how it arranges by photos in a timeline, the speed in which one can browse though photos, and the ability to tag photos easily.

Banshee has one killer feature, video.  Why this was so important to me at the time was that it provided the ability to download and watch video podcasts.  It’s a solid media player, but it isn’t leaps ahead of other applications in the space.

Beagle is the default desktop search engine.  It works ok.  There is a direct competitor in Tracker.  I don’t see any benefit to Beagle over Tracker.

There is also an alternate launch button.  This is like the start button in window.  I really like having my favorite and last launched applications available and in front of me; however, I don’t like having to click on more applications to open a menu to launch any other application.

Packages

OpenSuse absolutely excels in the package space.  The sites that you want to check out are http://software.opensuse.org/search and http://packages.opensuse-community.org. Most of the application searches will result in the package being found and provided though a convent one-click installation.

This was great for when I needed to install things like NetBeans 6.1.  I just searched and clicked the install button on the web page.

Issues

Ah the fun part.  A wise man once said that Linux doesn’t have a lack of drivers, it has a lack of quality drivers.  Most of the issues that I experienced initially were due to driver issues.

The atheros card in the laptop shows up as follows in lspci
04:00.0 Ethernet controller: Atheros Communications Inc. AR242x 802.11abg Wireless PCI Express Adapter (rev 01)
This particular card didn’t work with the new ath5k driver as I would’ve hoped.  There was a patch at the time, but it was only for the 32bit version of Linux.  Tickets for this can be found here http://madwifi.org/ticket/1679 and here http://madwifi.org/ticket/1192.

The second driver issue has to do with the intel video card.  It was shipped with broken s-video ouput.  Even though it showed up in xrandr, it just didn’t work.  Doing a software search and installing an intel driver found there fixed the issue.

Advanced desktop effects were unusable due to slowness issues I had.  Video playback would slow to a crawl when compiz-fusion was enabled.

Putting the computer to sleep does not work.  I attribute this to acer’s acpi more than anything.  It’s not a big deal for my everyday use, but it’s something that Linux struggles with and continues to do so.

There are a few usability issues.  They are annoying and have been covered in other reviews.

Virtualization

Virtualization is a must have for me and many others.  The easies way to get this done in OpenSuse is to install VirtualBox.  It runs without issues and runs on the currently installed kernel.  I tried the Xen installation and configuration through Yast, but the kernel crashed would not boot to a desktop, so I scrapped it and used VirtualBox instead.  It makes more sense to use it in a laptop scenario as Xen kernels don’t support CPU throttleing.

Configuration

OpenSuse uses YAST or Yet Another Setup Tool.  YAST has more configuration capabilities than any other Linux configuration tool I’ve seen.  Most of them even work!  It isn’t the end all configuration too yet, but it is quite good.

Novell

I am choosing to abstain from the Novell vs the free software community battle.  There are many places where this is documented beyond my knowledge.

Bottom Line

The availability of a wide range of software is really the kicker for me.  It’s great.  The updates are less frequent than other distributions and the functionality that works is quite stable.  Typically, I’ll switch distributions due to a time wasting issue that is distribution specific.  I don’t see that happening with OpenSuse.  When an issue arises, it seems to be hardware and Linux specific rather than distribution specific. It is, in a word, usable.  Download the livecd and give it a try.

Tags: , , , , ,

When double clicking on apps don’t seem to open the correct program anymore on almost every file, there is most likely an issue with the MIME database.  The MIME database tells the desktop environment which program should open a file.  Try this to re-associate things.

update-mime-database ~/.local/share/mime

This tip was found after searching the net and landing on this page.

Tags: , ,

Check em out here.  Unfortunitely, it looks like Linux users have to wait until the 5.1 release of MySQL Workbench to get a good free ERD tool.

Tags: , ,

This month, there were a good set of new Linux distribution releases.  The ones I was most excited about was Fedora 9 and OpenSuse 11.  This excitement was driven by a few things:

  1. Improved hardware support with the 2.6.26 kernel
  2. Package manager improvements (especially with OpenSuse)
  3. The new Gnome desktop (2.22) runs snappier than previous versions
  4. KDE 4

My initial project was to install Fedora Core 9 on the arcade system.  The installation was unique in that I decided to install to a flash drive instead of a hard drive.  This was inspired by the Fedora Live USB tools http://fedoraproject.org/wiki/FedoraLiveCD/USBHowTo.

Fedora is in a very interesting point in its life.  It is getting over being trounced in popularity by Ubuntu and learning to implement some of the features that make Ubuntu so appealing to so many.

I used the Windows version of the Live/USB creater tool first and it seemd to complete successfully.  However, the system would not boot fully to the device.  It seemed that the USB stick was assigned /dev/sdb which caused some errors.  I did get it to boot after typing some commands to mount the stick properly.

It was near sighted on my part, but I didn’t reallize that the Live USB stick would want to do hardware configuration on every bootup.  Perhaps a save hardware profile option would be a nice addition.  After realizing this, I decided that a full install to a the the USB Flash drive would be the best route.

The installer that is included with Fedora 9 is lacking when it comes to installing to flash.  There is no option to use jff2 as the file system.  This would’ve helped increase the life of the flash disk.  It took about 10 tries to get the installation going.  There were various problems with the disk partition tool that kept cropping up.  It, of course, didn’t like that I didn’t want to use swap space.  Also, any attempt to use the fat filesystem for the drive resaulted in a failure.

I ended up buying a 4GB drive to be the primary drive.  The Fedora installer fails if the drive does not have enough space (I think it was around 2.3GB) to copy the initial image.  The failure for this happens after the disk partitioning is done, so you have to go all the way back through the installer to correct this.

The Fedora desktop is really good looking and has been for most of there recent releases.  The hardware detection worked well and detected the atheros wireless card and loaded the driver correctly.  It surprised me since that was only recently committed to the Linux kernel.

Fedora package management has been its Achilles heel, in my opinion.  This release is, unfortunately, no exception.  Pup and Pruit have been ditched (yea!!).  The have been replaced with an installer that can only install one package at a time (boo!!).  This is a flaw that may just turn people against the distribution as a whole.  Installing yumex is a good interim solution for this issue.

There are a few ways that Linux distributions separate themselves from other distributions.  Here’s where comparisons should be made.

Type:  Desktop

Release Cycle: 6 months, supported for 1 1/2 years

Package Management:  Still slower than most, the default graphical fronend is missing the feature to install multiple packages.  This is the area where the distribution does the worst.

Feel: Good overall feel.  Theme is pretty and desktop is snappy.

Security:  SELinux is great.  It stayed out of the way while still prividing security.

License: It’s harder to find a “freeer” distribution than Fedora.  They and Red Hat are members in good standing with the open source community

Virualization: I didn’t test this on the arcade, but Fedora 9 does include the new paravirt_ops.

I will still use Fedora and check out the releases as they occur.  They are simply great at moving Linux forward with projects like PackageKit, AIGLX, pulseaudio, and paravirt_opts.  It’s hard not to want to support them.  Just please fix the installer and the package manager.

Tags: , , ,

Screenshot

This is a current screenshot of my desktop. Buuf icons, silver cursor, murrine rounded metacity, and murrina eternal gtk theme.

Tags: ,

Mame is a great thing. The portable SDLMAME is an even greater thing. It’s hard to find where to get it. The official home page is http://rbelmont.mameworld.info/?page_id=163. It really needs a better home page. It’s not even the first site that comes up in a google search.

After downloading it, make sure to read the contents of the readme files. There is a lot of information in SDLMAME.txt that should not be ignored. For example, I missed the fact that I had to change a line in the makefile to compile it for a 64 bit processor. There is also information there about how to compile it on various targets such as the PS3.

Once the exectuable is created, it is time to play some games. You may want to think twice about just playing the games though. There are some options that will help make the games run more smoothly. To create a default mame.ini file run the command ./mame -createconfig. The result will be outputted to the mame.ini file. Look through the file to see if there is anything of interest that needs to be modified.

The multithreading option can really speed things up on machines with multiple cores. I recommend that it be turned on. The autoframeskip option is another one that I turn on. When it is on, mame will automatically add to the frame skip when needed so that the game will run at 100% speed.

There are a lot of games that don’t play fast enough to be playable. Don’t be suprised if you hit a game that just doesn’t run quite right. With the tweaks above, that number is greatly reduced. Have fun 🙂

Tags: ,

So you’ve decided that you want to run gallery in your new chrooted apache install. Well, you’re in for a bitter bitter treat. It seams as though the php exec command requires that /bin/sh exists?!? Here’s how I got there.

Tried the gd option. Even though it checked out ok in the tests it just didn’t work. I didn’t track down why yet. Pages just wouldn’t load with this toolkit selected.

Next, I tried both ImageMagick and NetPBM. There are multiple executables to both systems. NetPBM has about 50 of them. I used this shortcut to copy the executables over.
rpm -ql netpbm-progs | grep bin`; do ln $i /var/www/usr/bin/`basename $i`; done

That command took all of the binaries from the netpbm-progs package and put them into my chroot. Later, I ran the ldd command against the binaries to figure out which library files were required. After finding out that netpbm relies on perl, I excluded that toolkit from the running and focused on getting ImageMagick to work.
About this time, I discovered that gallery has a debug option under config.php. I wish I’d looked there sooner :(.  This actually led me astray.  The error the debug was showing was “file not found”. However, I didn’t know which file wasn’t found (the input file, the output file, /bin/sh, /usr/bin/convert).
Another thing that I ran into was ImageMagick’s reliance on the magick.mgk file. Be sure to copy that to the chroot.

ln /var/www/usr/lib/ImageMagick-6.0.7/config/magic.mgk /usr/lib/ImageMagick-6.0.7/config/magic.mgk.
Thanks to having the debugging turned on, I was able to see the actual commands that gallery was running. The issue was that I could run those commands in the chroot, but gallery wouldn’t run them correctly. I looked through mod_security to see if I wasn’t allowing executables to run, and eventually turned filtering off. Still, it just didn’t work.

Here is an example of running a command in the chroot: chroot /var/www /usr/bin/convert -size 200×200 -geometry 200×200 /html/lynema/gallery2/modules/imagemagick/data/test.gif /html/lynema/gallery2/g2data/tmp/imgk_ZvjJHr
The final piece of the puzzle was moving over /bin/sh to the chroot. This sucks. I don’t like this mostly because of security reasons. Now a hacker has access to a shell to help them crack the site. The site is still chrooted, but I’m not entirely happy with the result.

Time spent on this was about 8 hours. AKA way too long.

Tags: ,

As is the trend with my server box, a hardware failure brought her down.  On Sunday the ups started to complain that the battery was going bad.  I finally caved in and shut the server down a couple days ago.  It was nice to have a silent room for a while, so I left it off till today.

The last time I had to shut the box down was due to the ‘failure’ of a hard disk.   I was going on about 200 days of uptime before that.  It’s amazing that a server at home can have the uptime that I’ve been getting.  Without really trying, I’m capable of 99.9%.

I’m running CentOS 4 BTW.  Thanks for the great product guys.

Tags: ,

« Older entries