Random Musings

Penguicon 4.0 is here (tomorrow), and I’ll be giving a talk on Git
at 11pm. There should be a bunch of interesting things going on there, and free beer.

This week, at the Michigan User Group meeting, I started playing with a new tool, Vym, that allows you to lay out thoughts, reorganize at will, and get things into somewhat of a structure. So, here’s the mind map I made:

(or here is a full size version).  Also in that directory I have the actual .vym file I made.  This is, in my opinion, and incredibly neat tool for so many things, so I have to rave about it.

Anyone that knows me knows that my generally preferred language is Perl. (I’ll wait for the jeering to die down.)
Generally, this isn’t a bad thing, as Perl is remarkably cross-platform compatible. Unfortunately, Windows is an utter disaster in terms of compatibility with Unix based systems.

Working on a few Git (Introduction) related tools, such as a replacement for “cvs annotate”/”svn blame”, we’ve learned that the current best practices for doing safe pipe opens aren’t very compatible.

For example, to call a program like “ls”, you might do something like this:

open(PIPE, “-|”, “ls”);

while(<PIPE>) { do_something($_); }

close(PIPE);

Unfortunately, that fails on Perl 5.6, and on ActiveState (Any version, I believe.)

So on Perl 5.6, the solution is to instead do the two steps by hand:

my $pid = open my $kid, “-|”;

defined $pid or die “Cannot fork: $!”;

unless ($pid) {

exec “ls”;

die “Cannot exec ls: $!”;

}

while(< $kid>) { do_something($_) }

close($kid);

But this fails on ActiveState, because ActiveState doesn’t handle the forked pipe open well (ok, at all). ActiveState doesn’t appear to have a good solution for this, at all, so at this point, we’re forced to fall back on something like backquotes (“) or the easier to read qx(). (qx() is just an alternate form of backquotes, that gives you some control of how interpolation happens.

The problem with using qx() is that it is a backquote form, so it return a list of lines, not a filehandle. Since that’s a very different way to do things, and potentially a big performance hit on systems that don’t need the qx() work arounds, I forced myself to find a way to just hide the complexity:

sub open_pipe {
if ($^O eq ‘##INSERT_ACTIVESTATE_STRING_HERE##’) {
return open_pipe_activestate(@_);
} else {
return open_pipe_normal(@_);
}
}

sub open_pipe_activestate {
tie *fh, “Git::ActiveStatePipe”, @_;
return *fh;
}

sub open_pipe_normal {
my (@execlist) = @_;

my $pid = open my $kid, “-|”;
defined $pid or die “Cannot fork: $!”;

unless ($pid) {
exec @execlist;
die “Cannot exec @execlist: $!”;
}

return $kid;
}

package Git::ActiveStatePipe;
use strict;

sub TIEHANDLE {
my ($class, @params) = @_;
my $cmdline = join ” “, @params;
my @data = qx{$cmdline};
bless { i => 0, data => \@data }, $class;
}

sub READLINE {
my $self = shift;
if ($self->{i} >= scalar @{$self->{data}}) {
return undef;
}
return $self->{’data’}->[ $self->{i}++ ];
}

sub CLOSE {
my $self = shift;
delete $self->{data};
delete $self->{i};
}

sub EOF {
my $self = shift;
return ($self->{i} >= scalar @{$self->{data}});
}

It should, hopefully, be fairly obvious what is going on there.

For a normal system, we use the Perl 5.6 compatible method, and return a filehandle that works normally.

For an ActiveState system, we tie a glob to a special object that provides some very basic emulation of a filehandle, and internally, calls qx() and indexes across an array returning the data.

So, that’s how “git annotate” is going to be cross-platform compatible, at least, as of today, that is my plan.

Thanks to Randal Schwartz (merlyn) for providing part of the inspiration for this, and the rest of the people on the Git list that helped hash out various approaches to this that didn’t quite seem as clean or as nice as this one, in the long run.

I think this method keeps all the security advantages of the argument list forms, while actually still managing to work on crippled systems, so I’m fairly pleased with it.

I recently was given a Dell Latitude 610 to use for work. So of course I’m running Debian on it. Tonight, I decided to play a DVD, and realized I had never tried to use the DVD player.

Turns out that with SATA drives, you need to give the module option atapi_enabled=1 to make it work. To get this option into the initrd, I had to create a file named /etc/modprobe.d/local as follows:

options libata atapi_enabled=1

and then recreate the initrd with:

mkinitrd.yaird -o /boot/initrd.img-2.6.15-rc5-686 2.6.15-rc5-686

update-grub

and now I’m watching Kill Bill 2.

In a vain attempt to let more people stumble across this, the “deb-pkg” target in the Linux Kernel makefile now needs to be run with fakeroot -u instead of fakeroot, due to a change in the way scripts/setlocalversion tracks down the current git version.

I plan on taking another look at the builddeb script to see about incorporating this into it, internally, but I also need to verify if the build process for a rpm needs to be run as root or if it can run as any user.

In the interests in doubling up some learning this weekend, I spent 5 or 6 hours using LaTeX to do some homework for my graduate-level Databases Class. (Aside: I’m attending Grad School at Oakland University now. I kinda missed going to school. On the other hand, I kinda miss free-time now. but whatever.) So, I really am intrigued by the power LaTeX has. The learning curve on it is a bit daunting at first, but once you get the hang of typing out equations in it, it’s really not so bad.
I don’t think I could imagine using anything else to do equation-laden things in the future…. So, for anyone reading this and pondering how to handle equation-laden stuff - I really don’t think there are any realistic options for you, outside of learning LaTeX. If anyone is interested, post a comment and I’ll put up the raw .tex file I used for this. (Well, after I turn the homework in. Gotta have some ethics.)

The Intel wireless drivers are rather old, and seem to be less stable than the current development drivers.

I don’t really understand why the 1.0.0 driver went in, rather than the 1.0.6 driver, given that the 1.0.6 driver has been out for several months.

In any case, I put a vote in for a more current driver, on the kernel list. Now I wait.

I’ve been hand-building my IPW2200 drivers for a while, but as of today (maybe yesterday), the drivers are part of the main kernel tree.

This makes me very happy. My laptop is now fully functional from the mainline kernel. (Toshiba Satellite A55-[something])

I’ve picked up some web browsing via “planet” sites recently. Most notably, Planet Arslinux, but also Planet Debian and Planet Kernel. On Planet Arslinux today, I saw Jay Wren’s post about products being noticed.. This got me to thinking - and especially, his points about iTunes and Google made me think about why I’m a user of those two things, and also, other things.

As I ponder this, I realize that the products I’m passionate about, all share a common trait. Mostly, I hate their competition because it sucks. I think this is just another way of saying the “10 times” rule Jay postulates.

With Google, I switched because it worked, and found me the information I needed. This was a rather remarkable feature at the time. Mostly because search engines just sucked. With iTunes, I started using it because I bought an iPod. On the other hand, it’s actually a really good mp3 player. And CD ripper. ID3 tag editor….. etc. It does a good job at all these things. None of the other players I’ve used provided all these features in one spot and got more than one of them right. In fact, most failed to get even one right. So, it’s not that iTunes is great. It’s just that the rest suck. (My mp3 player of choice before I stumbled on iTunes was a hand-written Perl script that I called “weightedplay”, that automatically noticed new mp3s and played more recently added stuff slightly more often than older things. It was really quite simple, and other than it’s lack of GUI, it was better than anything else I’d found for just playing songs.)

So those are the things that come to mind immediately, and why I switched away from their competitors. I’ll bring up another one, that’s occurred to me as I write this. A long, long time ago, I switched from using RedHat as my Linux distro of choice, to using Debian. In this case, it wasn’t because Debian was great. I had done some customizations to my RedHat installation in strange places (/etc/inetd.conf, IIRC), and doing an upgrade wiped out those changes without warning. In retrospect, after years of being away, I realize that my customizations *might* have been saved in .rpmsave files. But I was angry, so I found another distro that I was promised would tell me before it nuked my customizations. In the meantime, I’ve realized that apt actually works, and that I don’t really mind my packages being a little bit old if they work, so I’m happy with Debian. But I think this proves that it’s not how good the new choice is, but how much you dislike the old choice that sets these patterns in our minds.

This is the thing MicroSoft doesn’t get on search. For a very long time they thought that it was OK to fool their users by sneaking in paid-for results in their search listings. That’s bad, and hurts the accuracy of their results, in the user’s minds. This is why they will be struggling to recover influence in search for a very long time. (Oddly, I’ve noticed that Google’s advertisements are usually well targetted at my searches, which is pretty nice. I’ve even occassionally clicked on one. The beauty of relevance.)

So, I guess that’s my take on the thought. It’s not how good you are, but how bad you make your competitor look.

I’ve been spending my free time hacking away on “Git”, the source code management tool created by Linus Torvalds. It’s rather absorbing now that I’m really starting to grok how the command line tools all work, and I’m adding my own into the mix to fill the gaps. I guess the trick to everything is to just start using it so you figure out what is missing, either in the tool, or in your own knowledge.

I wish I felt up to trying to convert work to this tool - maybe I’ll do that next year when our BK licenses come up for renewal again. I suppose I should get around to figuring out how to pull all the data out of BK and reimport all of it into Git then.

Ugh, to say the least.

Friday at OLS seems to be almost entirely “Xen” focused. Since I’m rather intrigued by all this stuff - I’ll be sitting in these talks.

Xen is, basically, mainframe quality virtualization on x86.

It’s incredibly cool stuff. In fact, I’m so intrigued by it that I’m going to try rebooting my laptop into Xen in a few minutes and having a virtualized system running on my laptop full-time.

If this works, I think I’ll start converting my user mode linux setups at work into full-time xen systems, and I’ll just limit them to running in 64 meg of ram each, and that should give me much much better performance than I got with uml doing the same thing.

Listening to a talk about Ext3 is going to be improved in the future - it seems that we’ve reached a plateau of the easy and obvious big fixes having occurred, and the next-gen Ext3 changes will hurt.

Extent based allocations (use logical block, physical block, length as a single 64-bit record, instead of older, more granular formats) require file system layout changes, so they will not happen right away. Other changes related to on-disk format changes will most likely sneak in when the extent based allocations happen.

These changes will probably not cause a rename to Ext4 - the system of capability flags means that you will just pick the features your filesystem needs and create it. Older kernel that understand the right flags or that have all the features that are flagged as “incompatible” for backwards functionality, will be able to mount the filesystem and use it (perhaps read-ony), but the compatibility will be rather complicated.

My tentative plans for talks today (after I sleep) are:
nfsim: untested code is buggy code (Netfilter test suite)
Trusted Computing and Linux (Sorry, overslept)
State of Ext3 — see the next entry.

eCryptFS - encrypted filesystem
This is something akin to “unionfs” - it overlays an already mounted directory and provides automatic encryption of the files in it, based upon a set of policies. This is kind of neat, but not really quite ready yet.

Clusterproc (clusterwide process management) or kdump/kexec

Then, BOF sessions:
NFSv4
iSCSI or Security
Spamikaze

I’ll try to update this as I go over the day.

Wednesday - 11:30
Sat in on the BlueTooth talk (BlueZ). Interesting, got a rough overview of hardware and what is going on with the stack - and since I’ve only recently started trying to use this, I have high hopes that it will work as easily as it should - i.e, pair with the my mouse once and never worry about it again.

Wednesday - 13:30
Next up was the ACPI talk - plagued by 30 minutes of projector and laptop problems.
Turns out that most of the ACPI problems were not bad BIOSes, but bugs in the ACPI implementation in Linux. A couple people asked for an independent ACPI tester - to use in manufacturing, and/or when shopping for a laptop. Random other things - the basic theme being “It’s getting better than it was before - we think we’re past the worst point of getting compatibility and support working.” Lots of interesting new features in ACPI 3.0 - most of them haven’t shown up in machines yet so there really isn’t much (any) support for them, either. Support in the OS will happen after features start showing up in the BIOS and machines themselves. Some possible new devices, “operator present detector” (Is there a human in front of the machine?), “ambient light” (automatically dim the backlight in a low-light situation). Various physical connection, dependency, architecture things for CPUs.

Wednesday - 15:00
I went to the wrong room next, and sat in on the summary of how page cache performance was analyzed. (The page cache is the cache of things taht are file-backed, IIRC). Interesting, but hard to write about it without just repeating things - so go to http://linuxsymposium.org/ and download the papers yourself.

Wednesday - 18:00
Last up was the Keysigning - where we were, sadly, forced to implement a human merge-sort since the keysigning sheets were sorted by key-id and not by last name. Oh well. Next up to handle the actual signing, which always sucks. Gotta answer so many questions as part of the process.

Well, it’s 12:20am, on the day I fly to OLS.

I suppose I should pack, but I’m busy goofing off instead.

I’ll be at the Ottawa Linux Symposium for the rest of the week, attending conferences where a bunch of really smart people talk about the impressive (and sometimes crazy) things they’ve done to Linux. I imagine there will be some conversations about Git, too. (Git is the SCM tool that replaced BitKeeper for kernel development.)

So, I’m looking forward to going tomorrow, and I think I’m going to go get packed and crash now.

This week at work, we rolled out an update to our flagship application, including integration into Salesforce.com for our customer service needs.

What strikes me as odd is the apparent lack of feedback from the users that will interact with the system on a daily basis, such as the development team that I am part of.

All I ask for in a ticket/case/bug tracking system is that it can operate entirely via email, so that I don’t need to login to it.

Salesforce.com, well, can’t do that.

So incredibly annoying.

Hell, it can’t even provide the entire details of the case in 1-click from the notification emails, it’s 1-click to view the case overview, then 1-click per ‘detail’ to get at all of it.

I really wish we had evaluated a real product, in use by literally thousands of people first - like, oh, say BugZilla maybe?

Hell, even RequestTracker would be doing a better job at the moment!

I just randomly visited The Register today, to read an article about trojans having trojans in them, and saw a humorous, and hopefully either ironic or satirical, comment to end the article:

At least one security expert says there’s a lesson to be learned from the whole affair. “It obviously says you should always use open-source Trojans,” says Mark Loveless, a senior security analyst with Bindview Corporation. “That’s the moral. You can’t even trust Windows malware.”

The worst part is how accurate it is…

At work, we’ve been switching to using BitKeeper instead of CVS - I’m even more impressed than I was when just using it as a lazy way to download updates to the Linux kernel to play with at home.

It’s odd to feel inclined to do real work just to have an excuse to play with the tool involved…. but then again, my current project is to replace all text strings in our main product (ADX) with calls to gettext…. which is an incredibly boring task.

If only I could figure out why xgettext has no facility to set the character set (encoding? Whatever UTF-8 should be called.)…. I’d love a way to get rid of the crapload of warnings that ends up causing me.