I spent the weekend loading Nexenta on my two home PCs.

Nexenta is basically Ubuntu on an OpenSolaris kernel.  This gives you all the awesomenes of Ubuntu (apt-get, gnome, kde, GNU tools), with all the awesomeness of OpenSolaris (zones, ZFS, DTrace).

Some cool things:
  • The opensolaris packages are available via apt-get.  I wanted to set up some zones, so I did an “apt-get install sunwzoner sunwzoneu”.
  • “zoneadm -z zone1 install” fires off debootstrap and creates a Nexenta Zone.
  • The install is super-easy.  Just boot off the CD and answer a few questions.
So, now I have two Nexenta boxen, with about 1TB of storage online.  Which brings me to why I did this in the first place:  ZFS.

I really hate Linux Software RAID.  I hate it so much that I don’t use it.  This meant that my home storage was more or less unprotected, which sucks.  I don’t have anything irreplacable on these servers, but still, it’s good form to RAID.

Creating a mirror is as easy as:
# zpool create -m /storage zstorage mirror c1d0p0 c2d0p0

Adding another mirrorset to this pool is as easy as:
# zpool add zstorage mirror c3d0p0 c4d0p0

..Which gives me 560GB of RAID10 storage with two commands.   Now I have room for a complete nightly backup of my laptop, including incrementals (thanks, ZFS snapshots!), plus a backup of my “every photo I’ve ever taken” directory.

I was having a problem with Solaris’s Basic Security Module. I’d make changes to the audit_control and run “audit -s”, but it didn’t look like it was having an effect. Or at least it wasn’t taking effect for every process.

It turns out that each process’s audit flags are set at process create time, and that each process it forks off inherit the parent’s audit id.

You can see the audit details of a process by using auditconfig:

  1. auditconfig -getpinfo 23161

audit id = rold(25038)
process preselection mask = lo(0×1000,0×1000)
terminal id (maj,min,host) = 0,41352,gto(192.168.1.98)
audit session id = 23161
..And you can change the mask using auditconfig:

$ auditconfig -setumask 25038 ex,lo,fw,fc,fd

Which enables auditing of execs, login/logouts,file writes, file creates, and file deletes.

I’m jetlagged. It’s not what I expected.

I thought jetlag was primarily a sleep disorder. I thought I’d just have a tough time getting to sleep because of the time difference. This I can understand. You were going to bed at 2300 GMT, now you’re going to bed at 1400 GMT. It’s understandable that the body would go “huh? It’s the middle of the afternoon!”. I thought that I’d just be tired today.

Instead, it’s just bizarre. It’s not that I’m tired, because then I think I could deal with it better. I feel well rested, but fatigued and disoriented. I feel slow, like the rest of the world is travelling very fast. My thoughts aren’t well organized, and my coordination is…. off. It’s has all the symptoms of being sleepy, without actually being tired.

Dear Endocrine System,
I feel I owe you some of you an apology. While you’ve been working so hard to keep my body in balance, I’ve been undercutting your efforts. To those of you I’ve inconvenienced tonight, I’d like to apologize individually.

Pineal Gland. You may be wondering why it is that it’s 6 in the morning and we haven’t gotten a wink of sleep. You may be wondering what the hell is happening to all the melatonin you’ve been pumping into blood all night. You may even see this all-nighter as a personal failure. Don’t feel bad. It’s not your fault. While you’ve been squirting the melatonin, I’ve been chugging the coffee drinks.

That’s also the reason you’ve been working so hard, Pancreas. For all the caffeine those drinks have, they have a lot more raw, adulterated sugar. You’ve been a real champ, pushing out all that insulin. I appreciate it. You’re my favorite gland. Now could you please, PLEASE cut it out with the stomach acid? Isn’t 7 hours of heartburn enough?

Adrenal Glands. No apology for you. You guys freak out at every little thing, and frankly, I’m sick of it. The sudden drop in blood sugar is just the pancreas doing their job. It’s really, REALLY, nothing to worry about. The adrenaline you’re releasing is doing nothing more than wigging me out, and breaking down into more sugar, making the poor pancreas work even harder. Shame on you. Whoa! Whoa! Calm down! That’s exactly what I’m talking about. I’m not attacking you. Just calm the hell down, and cut it out with the fight-or-flight.

Pituitary? Hypothalamus? I don’t know what you guys are doing up there, but I have a sneaking suspicion you guys are the only ones keeping this boat together. Good work guys. Oh, and pituitary? Keep them endorphins coming. They’re awesome.

Finally, Kidneys. Although you’re not always seen as a member of the Endocrine team, I want you to know I appreciate your work the most. You’ve been pulling double duty, removing all this crap from my blood, while doing your very best to keep my blood pressure down. How have I repaid you? By not drinking enough water. My apologies, friend. Have a drink on me.

Your Boss,
-Laen

Idavoll is an implementation of a generic Publish Subscribe component for jabberd in Python.

Supported features:
* Creating nodes (including so-called instant nodes)
* Subscribing and unsubscribing
* Publishing items + subscribers getting notifications
* Requesting a previously published item
* Retracting a previously published item
* Purging all items for a node
* Deleting a node
* Configuring a node
* Requesting current affiliations
* Disco Info and Disco Items support

In the author’s blog, he says:
The Jabber support in Twisted needs some patches here and there. One of the patches is stringprep support for JIDs, a must-have. Twisted development is in a state of flux, as their upcoming 2.0 release will probably be split up in several packages, and the Jabber modules have been moved around. Also, dizzyd, who wrote the Jabber support for Twisted is busy with non-Jabber stuff, and I have to somehow get my patches in Twisted’s codebase. I’ll try and coordinate this with dizzyd in the coming week or so. In the meanwhile, I’ll probably have to distribute my patches to Twisted along in the Idavoll release, but I am not sure how to go about that, yet. For now, just checkout the code from the CVS repository, and give it a whirl. I will be more than happy to do some hand-holding.

Dag Wieers’s Home-Made Tools are some fun sysadmin tools.

In particular, is this bit of coolness: SoapBox. This is an LD_PRELOAD wrapper that monitors and records what changes an application makes to the filesystem. I’ve been looking for something like this for forever. It looks like you can place limits on what the user is allowed to do, too, making this a handy intermediary between a DAC system and going to a full MAC system.

DWall is a shorewall-like iptables front-end. This might be a replacement for shorewall in UMLazi.

DStat is kind of a cross between sar and vmstat/iostat. Might be easier to modify than sar.

Two bits of news from Open-ILS, the uber-cool project aiming to create a distributed, open-source library automation system.

Yesterday they posted an article on using jabber as their communication system:

I’d like to share a word on communication. We’ve decided to start with Jabber (www.jabber.org) as the communication layer between the various components. Jabber is great because it can be as simple as you want while allowing for practically limitless expansion. Given the open nature of Jabber, for example, we could write our own server components that ‘plug in’ to the jabber server and perform additional tasks on messages besides simply routing them through the messaging network.

Today they posted their November Executive Committee Report, which is a “where we’re at” report. The “Teaching a Programmer” section is fun.

The trick to sysadminning is doing the most with the least amount of home-written code. That’s how you make it a fun challenge. Anyone can brute force a problem. The skill– and the fun– is in solving it with style.

Sysadmin Thought Challenge –
Here’s a theoretical problem. How do you solve it?
“Let’s say you have 500 machines, and 5000 users. How do you do authentication?”

Why not take it a step farther? Heck, set up challenges, and provide the machines (UMLs) to do them.

Peter Saint-Andre talks about making Jabber more Secure:

  1. Get all the Jabberd’s using SASL and TLS (as per RFC)

    I think plumpy’s jabberd already does this.
  2. Get XMPP addresses into standard certificates.
  3. Set up a Cert authority

    2 and 3 are… well.. I don’t know. I think cert authorities are dumb. Can’t we do a web of trust instead of a tree?
  4. Start with s2s, then do s2c.

    Aren’t we pretty much all the way there on s2c? I think s2s is the last remaining challenge.
It’s bigger than this, actually, but it’s a start.

Jeremy Bowers (author of iRights) is doing great work on connecting Jabber to Radio. Lots of mind bombs here. Most of the IM crowd can’t see beyond simple chat etc., but the real gold is in making connections possible. Connecting desktop Web apps is the future of Jabber. [John Robb's Radio Weblog]

We’re actually going to hook lots of non-web apps together, too. Jabber is what DCE, CORBA, RMI, etc., could have been if they were open, simple, and had a natural ability to span firewalls, yet still be secure.

Now, IM is a way that we get our platform promulgated, but it’s also a key feature that other application-integration approaches don’t have. If users are running an application to chat with their friends, and tell if their friends are online, applications can use the same services to interact with users…

Jabber brings users and applications together with applications and users.

Is there a perl module for generating sendmail-style QID strings? That would be useful for any queuing system, especially the Jabber message queuer. If you encoded the time in there (in hex?) you could insure chronological order. I’m thinking jabber queue client here.

If you want an unprivileged process to be able to signal a privileged process in a well defined way, you can create a directory of “switches”. Files that the unprivileged can touch, and that the privileged can see. If you watch that directory with DNOTIFY, then the privileged process would know about the change in real time.

Logsurfer would make a great event correlation engine. Send all events into a log file (or set of log files) on a single host, and write logic to trigger events if events happen in order, or don’t happen within time periods of each other. Send as many events as possible through it. Start off with simple logic, then write tools to generate more complicated logic.

While we’re underestimating the amount of work involved in writing an event correlator, write a set of rules for every daemon. Write event generators for every daemon. Make “init” spawn events when processes start and stop! Write logsurfer rules on each client to generate events that go to The Correlator.

In Host Configuration Directories, like UMLazi uses, we don’t have a way of “commenting out” values. We can rename them to .files though!

Fluxbox lets you tab together xterms and other windows. What if it gave you the ability to mirror xevents to windows that were tabbed together? What’s stopping us from writing an X application that does that does that?

It would be freaking awesome if “screen” and “xterm” were aware of each other. So that you could use the xterm scrollbar (and thus, a mousewheel) to scroll back through screen history.

Okay, I’ve got four main projects going on right now.

  1. UMLazi – It’s almost time for another release. Wahoo!
  2. UMLFaq – This is going nowhere fast. I want this to be a compendium of UML knowledge, but thus far I’ve only posted a single entry. It takes a lot of energy to write FAQs.
  3. UMLForge – I don’t know what to do with this. I pictured it as a place where people could just request a UML. The problem with that is that they could be used for evil, and that I would rapidly run out of processor and memory for new UMLs. The answer to this might be to set up a timesharing system. Request a UML, and it’ll create one and stick your Public Key on it. You get it for an hour. After that, it suspends to disk, and you have to request more time. That could be fun. I don’t know how useful it would be. Hey, what if this was a UMLForge system, where anyone could set up one of their machines as a UMLForge Node, and contribute UMLs to the common good? Hmm.. Again with the usefulness test: What good is a temporary UML? What problem does this solve?
  4. The Jabber Monitoring and Management System – Sysadmins write tons of custom scripts and processes in order to automate their jobs, but the monitoring around those scripts and processes is almost always either nonexistent or sub-par. We need a better way to discover problems, notify the appropriate personnel, and automatically fix when possible.

November 1st, I start working for Nike in Beaverton, OR. I have very little idea of what I’ll be doing (outside of normal sysadmin stuff), but I do know that every time I’m paged out of bed, I’ll be getting paid for it. Yay contracting.

This marks the beginning of my return to Portland, which means, in no particular order:
* Selling the house in Tampa.
* Figuring out the bus system between home and work
* Making a good impression on my coworkers
* Distilling my possessions down to the absolute minimum, and shipping those to Portland.
* Finding a place to live in Portland.
* Re-examining my investment goals to take into account my new income.

I had a wisdom tooth removed Tuesday, and dang did that suck. Not so much the procedure, but the recovery. It wasn’t as bad as it might’ve been (pain was tolerable without Vicodin in just 8 hours), but I still have this oozing open wound in my mouth that I don’t know what to do with. I don’t dare chew with that side of my mouth. I don’t know how long I’m supposed to go easy on it. They say the stitches will dissolve in 9-14 days, and I’ll be fully recovered in 6-9 weeks. I guess I just have to be careful for a while!

Oh, the procedure for a tooth extraction (when the tooth is already all the way in) goes something like this:
* X-Ray
* Lidocaine
* Novocaine
* More novocaine
* Blindfold
* A rubber block in your mouth, to keep you from biting the dentist.
* A bunch of yanking
* Stitches
* About an hour of numbness, after which you damn well better have some vicodin in your system.