I always thought LVM (Linux's Logical Volume Manager) was kind of neat for the flexibility it gives you in adding and removing disks and resizing volumes such. However, in practice, I find it's usually more trouble than it's worth. It adds a layer of complexity between me and my data.
Often I need to mount a disk configured with LVM on another Linux machine or in an Ubuntu live CD environment. Out of the box the logical volumes aren't recognized, so I can't mount them.
It's fairly easy to add LVM support and mount the volumes though.
You install the lvm2 package, load the device mapper kernel module, and then activate any lvm volume groups on your disk.
$ sudo apt-get install lvm2
$ sudo modprobe dm-mod
$ sudo vgchange -a y
(assuming your disk with logical volumes is already connected)
$ sudo mount /dev/mapper/<logical volume name> /mnt
And that's all there is to it.
If you want to deactivate the volume groups (recommended before unplugging a USB disk with logical volumes):
$ sudo vgchange -a n
Warning: the above command will deactivate all volume groups, so check the vgchange manpage first, if that's not what you want.
Showing posts with label linux. Show all posts
Showing posts with label linux. Show all posts
Thursday, March 12, 2009
Thursday, January 08, 2009
How to sudoedit non-interactively
Okay, this one's a bit esoteric, but I think it's pretty cool. How do you use 'sudoedit' non-interactively such as from a script? Just a brief background about sudo: sudo is an authentication mechanism in Unix & Linux that allows unprivileged users to run specific commands (as defined by the system administrator) with root privileges without having the root password. This has several advantages over logging in as root:
So, what if you want to make changes to a system file via a script, and the only access you have to the file is via sudoedit? It isn't useful to have the script call sudoedit and then bring up vi for you to manually make the changes.
Well, the way sudoedit works, is that sudo makes a temporary copy of the file you want to edit, then calls your default editor, giving it the name of the temp file as the first argument. So, say your default editor is "fooedit". You run "sudoedit /etc/systemfile" and sudo makes a copy of /etc/systemfile to /tmp then runs "fooedit /tmp/tempfile". Your changes are saved to the temp file. When your editor exits, sudo copies the temp file over the original, and then removes the temp file.
From your main script, stage the pre-edited version of the system file, then set your default editor to a custom script that will copy that staged file over the temp file. When it exits, sudo will copy the temp file into place as normal.
Your script would go like this:
Your custom script would look like:
This script just cats your pre-staged file over whatever sudo passed in as the first argument ($1).
The only interaction you'll have to do is type your password for sudo, if you haven't already done so in the last few minutes (sudo temporarily remembers your authentication).
- Users can have specific, limited set of root privileges without having the entire set of root privileges.
- Users use their own password, so the root password doesn't have to be shared. If a user's sudo privileges are revoked, the root password doesn't have to be reset.
- Each use of sudo is audited per user, so that each time sudo privileges are invoked, there is an event in the system logs that identifies the specific user and the command they ran.
So, what if you want to make changes to a system file via a script, and the only access you have to the file is via sudoedit? It isn't useful to have the script call sudoedit and then bring up vi for you to manually make the changes.
Well, the way sudoedit works, is that sudo makes a temporary copy of the file you want to edit, then calls your default editor, giving it the name of the temp file as the first argument. So, say your default editor is "fooedit". You run "sudoedit /etc/systemfile" and sudo makes a copy of /etc/systemfile to /tmp then runs "fooedit /tmp/tempfile". Your changes are saved to the temp file. When your editor exits, sudo copies the temp file over the original, and then removes the temp file.
From your main script, stage the pre-edited version of the system file, then set your default editor to a custom script that will copy that staged file over the temp file. When it exits, sudo will copy the temp file into place as normal.
Your script would go like this:
#!/bin/sh
export EDITOR="/bin/sh ./editor.sh";
PRE_STAGE=/tmp/stage.tmp;
echo "my new config"> $PRE_STAGE;
sudoedit /etc/config;
/bin/rm $PRE_STAGE;
exit;
Your custom script would look like:
#!/bin/sh
CAT=/bin/cat
PRE_STAGE="/tmp/stage.tmp"
$CAT $PRE_STAGE > "$1";
exit;
This script just cats your pre-staged file over whatever sudo passed in as the first argument ($1).
The only interaction you'll have to do is type your password for sudo, if you haven't already done so in the last few minutes (sudo temporarily remembers your authentication).
Sunday, December 21, 2008
MythTV, HD-PVR, Trunk
Until Hauppauge released the HD-PVR this past May, there was no way to record in high-definition from your cable or satellite box. Your HD recording options were limited to over-the-air, unencrypted QAM, or, in rare instances, unencrypted firewire from the cable box. In any case there was no way to record the good stuff in HD. I'll spare the details about the HD-PVR itself, and instead include some links at the end.
Of course, Linux support for the HD-PVR wasn't available straight away. However, this is a big breakthrough for homebrew PVRs so the MythTV/Linux community has been hard at work. It there is a Linux driver for the HD-PVR, but you have to check it out and build it yourself (See links at the end). Also, early work has been done on MythTV to support the HD-PVR, but in order to get that support, you have to run MythTV from trunk, rather than using your distribution's packaged version. This means checking out source and building it yourself. The source code changes daily, and at times things get worse before they get better. Whatever you check out on any given day should be in decent shape, but it may not be. You are forging MythTV from the molten center of the Earth, and it's still hot and a little bit on fire.
What is now in trunk will eventually make it into your distribution, but there's generally a 6-12 month lag, depending on your distribution's release cycle. Well, I'm ready to give the HD-PVR a try now, so I decided to give trunk a go. I set up a spare machine on my work bench to be a frontend and backend. After getting the HD-PVR drivers compiled and working, I checked out MythTV from subversion and built it. There are good documents detailing how to do this. It's actually not that hard. The build system for MythTV is very nice. I recommend starting with a MythTV based distribution like Mythbuntu, and removing all the mythtv packages. This will save you a lot of grunt work like setting up MySQL, the mythtv user, init script, etc.
Anyway, I've got the HD-PVR working with MythTV. I haven't used it for any actual TV watching or recording yet. As soon as I can get a frontend set up, I'll be able to actually try using it day-to-day.
Relevant links:
http://www.geektonic.com/2008/06/hauppauge-hd-pvr-unboxing-first-look.html
http://mythtvnews.com/2008/05/30/hauppauge-hd-pvr-support-coming-for-mythtv-devices-are-shipping-now/
http://mythtvnews.com/2008/06/14/hauppauge-hd-pvr-linux-driver-released-alpha-quality/
http://www.mythtv.org/wiki/index.php/Ubuntu_Installation_Guides
http://www.mythtv.org/wiki/index.php/Hauppauge_HD-PVR
Of course, Linux support for the HD-PVR wasn't available straight away. However, this is a big breakthrough for homebrew PVRs so the MythTV/Linux community has been hard at work. It there is a Linux driver for the HD-PVR, but you have to check it out and build it yourself (See links at the end). Also, early work has been done on MythTV to support the HD-PVR, but in order to get that support, you have to run MythTV from trunk, rather than using your distribution's packaged version. This means checking out source and building it yourself. The source code changes daily, and at times things get worse before they get better. Whatever you check out on any given day should be in decent shape, but it may not be. You are forging MythTV from the molten center of the Earth, and it's still hot and a little bit on fire.
What is now in trunk will eventually make it into your distribution, but there's generally a 6-12 month lag, depending on your distribution's release cycle. Well, I'm ready to give the HD-PVR a try now, so I decided to give trunk a go. I set up a spare machine on my work bench to be a frontend and backend. After getting the HD-PVR drivers compiled and working, I checked out MythTV from subversion and built it. There are good documents detailing how to do this. It's actually not that hard. The build system for MythTV is very nice. I recommend starting with a MythTV based distribution like Mythbuntu, and removing all the mythtv packages. This will save you a lot of grunt work like setting up MySQL, the mythtv user, init script, etc.
Anyway, I've got the HD-PVR working with MythTV. I haven't used it for any actual TV watching or recording yet. As soon as I can get a frontend set up, I'll be able to actually try using it day-to-day.
Relevant links:
http://www.geektonic.com/2008/06/hauppauge-hd-pvr-unboxing-first-look.html
http://mythtvnews.com/2008/05/30/hauppauge-hd-pvr-support-coming-for-mythtv-devices-are-shipping-now/
http://mythtvnews.com/2008/06/14/hauppauge-hd-pvr-linux-driver-released-alpha-quality/
http://www.mythtv.org/wiki/index.php/Ubuntu_Installation_Guides
http://www.mythtv.org/wiki/index.php/Hauppauge_HD-PVR
Ditching apt-cacher-ng for Squid
apt-cacher-ng hasn't exactly worked out as well as I had hoped. I kept having a problem where the Releases file from the apt repository would be reported as corrupted. I could go into the cache directory and manually remove that file, forcing it to be downloaded again, but that would only help with the system I was updating at the moment. The next system that I tried to update would have the same problem.
I've read a number of articles about just using Squid as an apt cache. I avoided this at first because Squid isn't really made for that. For example squid has no way of knowing when a specific version of a package has been made obsolete by a newer version. I assume apt-cacher-ng and apt-proxy know how to do this (maybe not). Also Squid isn't intended to cache arbitrarily sized objects for indefinite periods of time. I decided to give it a try though, since a number of people seem to be having success with this.
I had to tweak Squid's default configuration a bit to make it suitable for apt-caching.
The Squid configuration file is well documented, but below are the directives of interest:
refresh_pattern deb$ 1577846 100% 1577846
refresh_pattern Packages.gz$ 1440 100% 1440
cache_dir ufs /var/spool/squid 15000 2 8
maximum_object_size 409600 KB
The first line says to cache anything ending ('$' is the end-of-line anchor) in "deb" for 3 years. There are some packages that rarely get updated if ever, so I want to make they stay in the cache the entire time I'm using a given distribution release. I figure I'll probably be on a particular distribution release for no longer than three years.
The second line says to cache anything ending in "Packages.gz" for one day.
The "cache_dir" line says to put Squid's cache in /var/spool/squid (I believe this is the default). It also says to let it grow no larger than 15,000 megabytes. My system partition gets very little use, so 15Gb is no problem for me. The second two numbers tell Squid how to structure the cache. "2" says to create two level 1 directories and "8" says to create 8 level 2 directories. The default is 16 and 256. I read an article where the author was having a problem with the hard drive never spinning down because Squid was rescanning the cache every few seconds. The author said reducing the number of L1 and L2 directories helped. If anyone can find this article, please post a link in the comments.
The "maximum_object_size" tells squid to cache objects up to 400Mb. There shouldn't be any debs even close to that large. Squid's default is much smaller than this.
In order to get apt to use the proxy, I created a file called "proxy" in /etc/apt/apt.conf.d/. You can call the file whatever you want. The file contains the line:
Acquire::http::Proxy "http://hoth:3128";
"hoth" is the hostname of the Squid server. Make sure there are no "Acquire" directives in any other apt configuration file:
/etc/apt/ # grep -rn Acquire *
I discovered that on some systems there was an "Acquire" directive explicitly telling apt not to use a proxy.
So far, Squid has been working pretty well. There was one instance where apt failed to completely download a package, but after running apt-get dist-upgrade again, the package successfully downloaded.
I've read a number of articles about just using Squid as an apt cache. I avoided this at first because Squid isn't really made for that. For example squid has no way of knowing when a specific version of a package has been made obsolete by a newer version. I assume apt-cacher-ng and apt-proxy know how to do this (maybe not). Also Squid isn't intended to cache arbitrarily sized objects for indefinite periods of time. I decided to give it a try though, since a number of people seem to be having success with this.
I had to tweak Squid's default configuration a bit to make it suitable for apt-caching.
The Squid configuration file is well documented, but below are the directives of interest:
refresh_pattern deb$ 1577846 100% 1577846
refresh_pattern Packages.gz$ 1440 100% 1440
cache_dir ufs /var/spool/squid 15000 2 8
maximum_object_size 409600 KB
The first line says to cache anything ending ('$' is the end-of-line anchor) in "deb" for 3 years. There are some packages that rarely get updated if ever, so I want to make they stay in the cache the entire time I'm using a given distribution release. I figure I'll probably be on a particular distribution release for no longer than three years.
The second line says to cache anything ending in "Packages.gz" for one day.
The "cache_dir" line says to put Squid's cache in /var/spool/squid (I believe this is the default). It also says to let it grow no larger than 15,000 megabytes. My system partition gets very little use, so 15Gb is no problem for me. The second two numbers tell Squid how to structure the cache. "2" says to create two level 1 directories and "8" says to create 8 level 2 directories. The default is 16 and 256. I read an article where the author was having a problem with the hard drive never spinning down because Squid was rescanning the cache every few seconds. The author said reducing the number of L1 and L2 directories helped. If anyone can find this article, please post a link in the comments.
The "maximum_object_size" tells squid to cache objects up to 400Mb. There shouldn't be any debs even close to that large. Squid's default is much smaller than this.
In order to get apt to use the proxy, I created a file called "proxy" in /etc/apt/apt.conf.d/. You can call the file whatever you want. The file contains the line:
Acquire::http::Proxy "http://hoth:3128";
"hoth" is the hostname of the Squid server. Make sure there are no "Acquire" directives in any other apt configuration file:
/etc/apt/ # grep -rn Acquire *
I discovered that on some systems there was an "Acquire" directive explicitly telling apt not to use a proxy.
So far, Squid has been working pretty well. There was one instance where apt failed to completely download a package, but after running apt-get dist-upgrade again, the package successfully downloaded.
Tuesday, September 09, 2008
apt-cacher-ng
I have several machines on my network running Ubuntu Linux which need to be updated regularly to keep up with security updates and the like. Several months ago I decided to try using a caching proxy to get updates from the Ubuntu package repositories in order to reduce load on the Ubuntu servers as well as to speed up the updating process on my systems. After spending some time using apt-proxy, and being generally dissatisfied with its reliability, I decided to give apt-cacher-ng a try.
Unfortunately apt-cacher-ng's documentation is a bit inaccessible. Aside from not being available on the project's website (it's in PDF form in the documentation directory after you install it), the documentation, although complete, is kind of impenetrable. There also really isn't anything that I could find along the lines of HOW-TOs or FAQs on the 'tubes. I eventually got it worked out, and, at least for the common case, it turned out to be more straight-forward than the documentation leads you to believe. Here's how I set mine up.
I put entries in my acng.conf like the following:
Remap-ubuntu: http://us.archive.ubuntu.com/ubuntu
Remap-medibuntu: http://packages.medibuntu.org/
The way this works is you specify names for each Remap-[whatever] where [whatever] is whatever you want to call it--they just have to be unique. Then the part after the ':' is the literal URL of the repository you're caching including the directory path on that server, if there is one.
The documentation describes a third part that follows a ";". If your second part is the actual URL of the repository, you don't need the third part, as you see above.
In my sources.list, I put
deb http://my-server-name:3142/packages.medibuntu.org hardy free non-free
deb http://my-server-name:3142/us.archive.ubuntu.com/ubuntu hardy main restricted universe multiverse
Here, the URL to the apt repositories is your local server name (port is 3142 unless you change it in acng.conf) followed by a directory that is the exact same as whatever the fully qualified domain name of the real apt repository was, followed by the actual directory path that you appended to the actual apt repository in your acng.conf.
There are lots of other options described in the apt-cacher-ng documentation, but I believe this is the common case, and it's working fine so far.
Update:
The directory structure that gets created under /var/cache looks like:
Unfortunately apt-cacher-ng's documentation is a bit inaccessible. Aside from not being available on the project's website (it's in PDF form in the documentation directory after you install it), the documentation, although complete, is kind of impenetrable. There also really isn't anything that I could find along the lines of HOW-TOs or FAQs on the 'tubes. I eventually got it worked out, and, at least for the common case, it turned out to be more straight-forward than the documentation leads you to believe. Here's how I set mine up.
I put entries in my acng.conf like the following:
Remap-ubuntu: http://us.archive.ubuntu.com/ubuntu
Remap-medibuntu: http://packages.medibuntu.org/
The way this works is you specify names for each Remap-[whatever] where [whatever] is whatever you want to call it--they just have to be unique. Then the part after the ':' is the literal URL of the repository you're caching including the directory path on that server, if there is one.
The documentation describes a third part that follows a ";". If your second part is the actual URL of the repository, you don't need the third part, as you see above.
In my sources.list, I put
deb http://my-server-name:3142/packages.medibuntu.org hardy free non-free
deb http://my-server-name:3142/us.archive.ubuntu.com/ubuntu hardy main restricted universe multiverse
Here, the URL to the apt repositories is your local server name (port is 3142 unless you change it in acng.conf) followed by a directory that is the exact same as whatever the fully qualified domain name of the real apt repository was, followed by the actual directory path that you appended to the actual apt repository in your acng.conf.
There are lots of other options described in the apt-cacher-ng documentation, but I believe this is the common case, and it's working fine so far.
Update:
The directory structure that gets created under /var/cache looks like:
apt-cacher-ng
|-- canonical-partner
| `-- dists
| `-- hardy
|-- ubuntu
| |-- dists
| | |-- hardy
| | |-- hardy-backports
| | `-- hardy-updates
| `-- pool
| |-- main
| |-- restricted
| `-- universe
`-- ubuntu-security
|-- dists
| `-- hardy-security
`-- pool
|-- main
|-- multiverse
`-- universe
Subscribe to:
Posts
(
Atom
)