Sandbox and ACLs or: Why I Want to French Kiss Mikey-San

So I recently had a freelance project that involved my first experience with, among other things, Access Control Lists (which we will, from here on out, refer to as ACLs, 'cause I'm not typing that shit a hundred more times today). Briefly, ACLs allow more fine-grained control over file and folder permissions by defining multiple rules for multiple groups and/or users for any given object. ACLs are exceptionally cool. But they're also incredibly complex.

For my freelance project I was assigned the odious task of setting up a share on a Mac server, and this share would be readable by all, but its contents would not, unless explicitly allowed, be readable by members of a particular group. Let me try to clarify this, 'cause it took me a few days to get my head around it: Essentially we had three groups, which we'll refer to as Artists, Producers and Freelancers. Artists and Producers needed full access to everything on the share (which we'll call Share), and Freelancers needed read access. Freelancers, however, needed to be denied access to all subfolders of Share, including newly created folders. Later, if a member of Freelancers needed access, someone from one of the other groups would grant them access to the folder in question on Share.

Complicated? You betcha! Impossible? Surprisingly, no. Though, if you try to do this with Apple's Workgroup Manager (which we'll just call WGM) you'll never be able to. The key lies with two concepts: rule ordering and inheritance. See, ACLs are ordered lists, and the first rule in the list wins. Additionally, "deny" rules always trump "allow" rules, with one crucial exception: explicit allow rules trump inherited deny rules. This was the magic that made our little scenario work. I did not see any mention of this in the Mac OS X Server documentation (though it's entirely possible I missed it). And I certainly did not see any way to reorder rules in the WGM application, inherited or not. Rather than from Apple, I got all my info on rule ordering and precedence from my new best-est pal in the whole wide universe, mikey-san of sightless and born.

Enter: Sandbox.

Sandbox is a cool, free, little utility whose sole purpose in life is to enable ACL management on non-server versions of Mac OS X. It even has some pretty informative documentation. Amazingly, however, Sandbox gives you way more and better control over your ACLs than Mac OS X Server's WGM app, particularly with regards to rule ordering. With Sandbox, you can explicitly set a rule as "inherited," and you can explicitly set the order of rules. You can't do either of these things in WGM, which is quite infuriating considering how important ordering is in ACLs.

Like the Command-Line: This is How it Looks in Sandbox

(click for larger view)

So, using Sandbox, I created a rule for Share that gave full access to Share for Artists and Producers, and that forced inheritance of these permissions to any subfolders of Share (so Share and its subfolders are always accessible to Artists and Producers). Then I created an "allow" rule which granted Freelancers read permissions on Share, but which did not propagate those permissions to subsequent subfolders (so Share is readable by Freelancers, but none of its subfolders are — if this rule is not propagated, which it's not, there is no rule on subfolders, and POSIX permissions take over, granting "world," and thus Freelancers, read access to subfolders of Share, so we need a deny rule next). Finally, I created a "deny" rule for Freelancers that disallowed them access to Share and any of its subfolders. This rule is inherited and occurs before the "allow" rule (so denial of access is propagated to subfolders of Share, essentially acting like a umask, but does not apply to the actual Share folder). The key here is that, in order for this deny rule to not override the allow rule, it must be inherited (remember, explicit "allow" rules trump inherited "deny" rules), and, once it has been set to "inherited," it must occur before the deny rule. Neither of these astonishing feats can be accomplished with WGM. Both can with Sandbox.

That "Deny" is Out of Order: Here's How it Looks in WGM

(click for larger view)

And that is why I want to French kiss mikey-san. Whoever he is.

UPDATE:

All the above was done on a test machine. (Thanks be to the joys of education work that actually allows me to build test servers!) Putting this plan into practice was one more, extra, added layer of tricky. You see, the share on which we wanted to set this up was already in use by the client. Yup, it was already populated with all manner of data, none of which had any ACLs applied.

The final nail in this coffin was to propagate the ACLs in WGM. But this had to be done such that all subfolders and files had the inherited properties we wanted, but the top-level "Share" had it's original properties. Remember: our top-level Share should be readable by Freelancers, but it's descendants should not, so the top-level permissions are different from all the subfolders and files. Had we simply propagated our "Share" permissions through the entire tree, Freelancers would have read-access to everything, and this was verboten. The trick was to remove the "allow:read" rule from "Share" and then propagate permissions. After the propagation was finished (about 10-15 minutes later — it was a large amount of data) we reapplied the "allow:read" rule to "Share" and all seems cool.

This may seem like an obvious solution, but I wanted to make a note of it as ACLs — at least complex ones like these — seem to befuddle even the most logical minds, and certainly mine. So for thoroughness' sake, there you have it.

Sync Firefox Bookmarks with Google Extension

A few days ago I wrote about a method for publishing iCal calendars sans .Mac. In that vein, here's a quick plug for the Google Browser Sync extension for Firefox. The extension will allow you to save your Firefox cookies, saved passwords, bookmarks, history and tabs and windows — or any combination of these — to Google's servers. For the more security conscious, you can even opt to encrypt this information. The extension will then sync this information with any other Firefox browser that has the plug in installed and configured.

Google Browser Sync: Finally!
(click for larger view)

I've been using this for a few days now, and I'm loving it. For the first time in my life my home and work bookmarks are finally in sync. It's a thing of beauty, and something I've longed for for quite a while. Of course, if you're a Safari user, you're out of luck. But in my opinion, this is just one more reason you should really consider switching to Firefox. While Firefox may be slower than the others — and bear in mind that the Google Browser Sync will slow launch times down just a wee bit while it syncs your info — it's far and away the most full-featured and stable browser I use. It loads virtually everything and never, ever crashes. And now it syncs with itself via Google.

Don't bother getting me a Christmas gift; I have everything I need.

Remote Network (and More!) Management via the Command-Line

There's a lot you can do in the Terminal in Mac OS X. It's often the most expedient method of achieving a given task. But I always get stuck when it comes time to make certain network settings via the command-line. For instance, in the lab all our machines are on Built-In Ethernet, and all are configured manually with IP addresses, subnet masks, routers, DNS servers and search domains. But sometimes we need to make a change. While I suppose it's easy enough to go over to the computer, log in, open the System Preferences, click on the Network pane, and configure it by hand in the GUI, this becomes a problem when you have to interrupt a staff member busily working on whatever tasks staff members tend to busily work on. Also, it doesn't scale: If I have to make a change on all 25 Macs in the lab, the above process becomes tedious and error-prone, and exponentially so as we add systems to the network. When performing such operations on multiple computers, I generally turn to Apple's wonderful Remote Desktop. But ARD (I'm still using version 2.2) lacks the ability to change network settings from the GUI. Fortunately, there is a way.

ARD comes with command-line tools. They're buried deep in the ARD application package, for some reason, making them a serious pain to get at, but they're there and they're handy as hell in situations like the above. You can use these tools to set up virtually anything normally accessible via the System Preferences GUI. Including network settings. There are two main commands for doing this:
systemsetup for making general system settings, and networksetup for making network settings.

Both commands are located in the folder:

/System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Support

You'll need to type the full path to use the commands, or cd into the directory and call them thusly:

./networksetup

Or you can do what I do and add an alias to them in your .bash_profile:

alias networksetup='sudo /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Support/networksetup'

There are no man pages for these commands, but to get an idea of what they do and how they work, just run them without any arguments and they'll print a list of functions. Or you can run the commands with the -printcommands flag to get an abbreviated list of functions. There is also a PDF reference online. The syntax for these tools is not always straightforward, but by and large looks something like this:

networksetup -option <"network service"> <parameter 1> <parameter 2> <etc...>

For "network service" you'll want to enter the name of the interface as it's called in the preference pane, surrounded by quotes. You also need to run these commands as root. Here's an example that makes network settings (IP address, subnet mask and router, respectively) for Built-In Ethernet:

sudo networksetup -setmanual "Built-In Ethernet" 192.168.1.100 255.255.255.0 192.168.1.1

Here's an example for setting multiple search domains:

sudo networksetup -setsearchdomains "Built-In Ethernet" systemsboy.blog systemsboy.blog.net

Why these tools aren't included in the GUI version of ARD is almost as confounding as the fact that they're buried deep within the guts of the app. The wordy syntax doesn't help either (I'd much prefer typing "en0" to "Built-In Ethernet" any day). But despite being difficult to access, I've found these commands to be unbelievably useful from time to time. And though I certainly do hope to edify TASB readers as to the existence and use of these tools, I must confess, my main reason for this post is as a reminder to myself: I'm always forgetting where the damn things are and what the hell they're called.

So there you have it. Remote network (and more!) management via the command-line.

Please resume admin-ing.

UPDATE:
So a scenario occurred to me which might be a bit tricky with regards to the above information: Say you want to change the router setting for your 25 lab macs. Nothing else, just the router. The obvious way to do this would be to send a networksetup command to all of your Macs via ARD. The problem is that the networksetup command has no way of specifying only the router address, without setting the IP address and subnet mask as well. The only way to set the router address is with the -setmanual flag, and this flag requires that you supply an IP address, subnet mask and router address, respectively:

sudo networksetup -setmanual "Built-In Ethernet" 192.168.1.100 255.255.255.0 192.168.1.1

If you send this command to every Mac in your lab, they'll all end up with the same IP address. This will wreak havoc in numerous ways, the worst of which will be the fact that you now have lost all network contact with said lab Macs and will have to go and reset all the IP addresses by hand on each machine. The solution is to specify the current IP address on each machine, which you could do by sending an IP-specific version of the command to each machine individually. But that sort of defeats the purpose of using ARD in the first place. A better way is to substitute a variable in place of the IP address option in the above command. The variable will get the IP address of the machine it's run on. This is what I use:

ifconfig -m en0 | grep "inet " | awk {'print$2'}

This command sequence will get the IP address for the en0 interface, which is what Built-In Ethernet uses. If your machines are on wireless, you can use en1 in place of en0:

ifconfig -m en1 | grep "inet " | awk {'print$2'}

So, instead of entering the IP address in the networksetup command, we'll enter this command sequenc e in backticks. The command we'd send to our lab Macs would look something like this:

sudo networksetup -setmanual "Built-In Ethernet" `ifconfig -m en0 | grep "inet " | awk {'print$2'}` 255.255.255.0 192.168.1.1

This will set the subnet mask and router address — both of which would generally be consistent across machines on any given network — but will leave the unique IP address of each machine intact. You can try this on your own system first to make sure it works. Open the Network System Preference pane, and then run the command in Terminal. You'll see the changes instantly in the preferences pane, which will even complain that an external source has changed your settings. If all goes well, though, your IP address will remain the same while the other settings are changed. You should now be able to safely change the router address of all your machines using this command in ARD (remember to specify the full command path though). Save that sucker as a "Saved Task" and you can do this any time you need to with ease.

Huzzah!

Publish iCal Calendars on the Internet for Free

Addendum: Aug 21, 2010

I wanted to let folks know that the information in this post is pretty old, and these days there are better options for sharing calendars. Most notably, Google Calendar sharing. The method used in the post you're about to read is a one-way share in which the publishing computer is the only one that can make changes, and all subscribing calendars have read-only access. This is limited, I know, but at the time it was readily available and plenty of functionality for most people who just wanted a free option for calendar sharing from their main machine. Nowadays, though, Google Calendar sharing gives the same level of ease-of-use and true, server-based, two-way calendar syncing, allowing you to edit you calendars from any computer (including iPhones). It truly is superior, and I highly recommend it.

So, if you're still interested in the one-way Box.net method, I leave it up for posterity. Otherwise, Google Calendar sharing is probably the way to go.

Happy calendaring!

Original Post

Ever want to share your iCal calendars on the 'net? Well, I have. Unfortunately, iCal publishing requires a server that runs the WebDAV protocol. Apple's .Mac offers such a service, and .Mac, indeed, uses WebDAV. But .Mac costs money, and I've never seen a compelling enough reason to pay for it. Setting up iCal on a Macintosh server is also fairly easy, but it's an awful lot of trouble to go to and a waste of resources if all you're interested in is sharing your iCal calendars. But today I've discovered an easier — and free — way to do just that.

Box.net offers 1 GB of online file storage for free. The data is accessible via the web, so you can get to it from any connected computer. They have a web interface for accessing you files. On Windows they also have client software that lets you access your data as though it were on a shared volume, right from the Desktop. But on the Mac you can connect to it right from the Finder using the ever-trusty "Connect to Server..." command (or command-k) in the "Go" menu. How is this possible you ask? Because the Mac Finder has WebDAV accessibility built right in. And because box.net shares it's data via the WebDAV protocol.

If you haven't put it together yet, this means you can use your free box.net account to host your iCal calendars. The process would go something like this:

1. Sign up for a free box.net account. It's easy as pie, and faster than a bat out of hell. Give 'em your email, a password, and you are d-o-n-e, done. (NOTE: Obviously, to share the calendar with others, they'll need the URL and the box.net username and password, so you might want to make sure you set this account up with information you don't mind giving out. Use a second, private account for stuff you want to keep secure.)
2. To access your box.net account via the Finder, click "Connect to Server..." and type "https://www.box.net/dav" into the URL field of the connection window. (NOTE: Some folks are unable to connect using the "https" protocol. If this fails for you, try it with "http" instead. Either one works for me.) You will be prompted for username (the email you used to register the account) and your box.net password. Enter these and your online data will appear on the Desktop in a volume called "dav" which will be pre-populated with a few default folders.
3. But to publish your iCal calendar, just open iCal, select the calendar you want to share and choose "Publish" from the "Calendar" pulldown menu.
   

   (click for larger view)

4. In the dialog box that follows, name your calendar (or keep the existing name if it suits you), for "Publish on:" select "Private Server," enter "https://www.box.net/dav/Documents" for the "Base URL:" and enter your box.net username and password in the "Login:" and "Password:" fields respectively. (NOTE: Here, too, secure http sometimes fails, so if you have trouble try using "http" instead of "https".)
   

   (click for larger view)

5. You might also want to check or uncheck some of the checkboxes at the bottom, depending on your needs.
6. Finally, hit "Publish" to publish the calendar. You'll see your calendar now sports the "I'm Published" icon just to its right.
7. To subscribe to this calendar (say, from another computer), choose "Subscribe" in the"Calendar" pull-down menu.
8. In the URL field enter:

   "http://www.box.net/dav/Documents/YourCalendar.ics"

   (where "YourCalendar" is the name of the calendar you just published).

   

   (click for larger view)

   Be careful NOT to enter "https" here. Just use the "http" protocol to subscribe. Enter your box.net username and password at the prompt and you'll suddenly find yourself subscribed to your own calendar.

   

   (click for larger view)

9. A final note: Updates to the calendar take some time. Hitting refresh too often (like every minute or so) on a subscribed calendar will generate an error. Just wait and let iCal do it's thing and it will keep everthing up to date. No pun intended.

I'm not sure if many people will find this useful. But to me it seems like a good way to share you iCal calendars, either between remote locations (like work and home) or between groups of people, like friends, family or members of an organization. And hey, it's completely free. How cool is that?

UPDATE 1:

A lot of people have written in with great suggestions, alternatives to and variations on this process, including a page for similarly configuring Sunbird. Please read the comments for more useful information.

UPDATE 2:

I have added a note to the publishing instructions in this article regarding the use of "https". In a nutshell, both publishing and subscribing sometimes fail because of problems with secure http. This is something I receive a lot of comments about and I just wanted to clear things up. If you are having trouble either publishing or subscribing, try doing so using "http" instead of "https". While somewhat less secure, it tends to be far more reliable.

On Backups

Let me just say up front, historically I've been terrible about backing up my data. But I'm working on it.

As far as backups go, I've tried a lot of things. I am responsible for backups of staff data at work, and here is where the bulk of my trials have occurred. For my personal data I've always just archived things to CD or DVD as my drive got full, or as certain projects wrapped up, but I've never had any sort of emergency backup in case of something like a drive failure or other catastrophe. Both at home and at work, though, the main problem I've faced has been the ever-expanding amount of data I need to backup. Combined staff data typically takes up a few hundred gigabytes of disk space. And at home my Work partition (I store all user data on a partition separate from the System) currently uses 111 GB. This does not even take into account the multiple firewire drives attached to my system at any given time. All tolled, we're talking several hundred gigabytes of data on my home system alone. I don't know what "the best" way is to back all this up, but I think I have a pretty good solution both at home and at work.

The Olden Days
Back in the day, staff backups were performed with Retrospect to a SCSI DAT drive. This was in the OS9 days. The tapes each held about 60GBs, if memory serves, and this worked fine for a while. But with the high price of tapes, a limited tape budget, and ever-increasing storage needs, the Retrospect-to-tape route quickly became outmoded for me. It became a very common occurrence for me to come in on any given morning only to find that Retrospect had not completed a backup and was requesting additional tapes. Tapes which I did not have, nor could I afford to buy. Retrieving data from these tapes was also not always easy, and certainly never fast. And each year these problems grew worse as drive capacities increased, staff data grew and tape capacities for our $3000 tape drive remained the same. The tape solution just didn't scale.

Enter Mac OS X
When Mac OS X arrived on the scene, I immediately recognized the opportunity — and the need — to revise our staff backup system. First off, Retrospect support was incredibly weak for OS X in those early days. Second, even when it did get better, there continued to be many software and kernel extension problems. Third, SCSI — which most tape drives continue to use to this day — was on the way out, annoying as hell, and barely supported in OS X. Fourth, the tape capacity issue remained. On the other hand, from what I was reading, Mac OS X's UNIX underpinnings would provide what sounded like a free alternative, at least on the software side: rsync. My two-pronged revision of our backup system consisted of replacing Retrospect with rsync and replacing tape drives with ever-cheaper, ever-larger hard drives.

RsyncX
The only problem with the UNIX rsync was that it famously failed to handle HFS+ resource forks (as did, incidentally, Retrospect at the outset). This situation was quickly remedied by the open source community with the wonderful RsyncX. RsyncX is a GUI wrapper around a version of rsync that is identical in most respects to the original UNIX version except that it is capable of handling resource forks. Once I discovered RsyncX, I was off to the races, and I haven't found anything to date — incuding the Tiger version of rsync — that does what I want better.

My Process
These days I do regular, weekly staff backups using RsyncX over SSH to a firewire drive. For my personal data, I RsyncX locally to a spare drive. This is the most economical and reliable data backup solution I've found, and it's far more scalable than tape or optical media. It's also been effective. I've been able to recover data on numerous occasions for various staff members.

My system is not perfect, but here's what I do: Every day I use RsyncX to perform an incremental backup to an external hard drive. Incremental backups only copy the changes from source to target (so they're very fast), but any data that has been deleted from the source since the last backup remains on the target. So each day, all new files are appended to the backup, and any changes to files are propagated to said backup, but any files I've deleted will remain backed up. Just in case. Eventually, as I'm sure you've guessed, the data on my backup drive will start to get very large. So, at the end of each month (or as needed) I perform a mirror backup, which deletes on the target any file not found on the source, essentially creating an exact duplicate of the source. This is all run via shell scripts and automated with cron. Finally, every few months or so (okay, more like every year), I backup data that I want as part of my permanent archive — completed projects, email and what not — to optical media. I catalog this permanent archive using the excellent CDFinder.

Almost Perfect
There are some obvious holes in this system, though: What if I need to revert to a previous version of a file? What if I need a deleted file and I've just performed the mirror backup? Yes. I've thought about all of this. Ideally this would be addressed by having a third hard drive and staggering backups between the two backup drives. A scenario like this would allow me to always have a few weeks worth of previous versions of my data, while still allowing me to keep current backups as well. Alas, while I have the plan, I don't have the drives. Maybe someday. But for now this setup works fine for most of my needs and protects me and the staff against the most catastrophic of situations.

Consider Your Needs
Still, when devising a backup scheme, it's important to understand exactly what you need backups to do. Each situation presents a unique problem and has a unique set of requirements. Do you need a permanent, historical archive that's always available? Or do you simply need short-term emergency backup? Do you need versioning? What data needs to be backed up and what doesn't? For my needs previous versions are less important; emergency backups are critical. Also you need to consider how much data you have and what medium is most appropriate for storage with an eye towards the future. In my case I have a lot of data, and I always will. Hard drives are the most economical way for me to store my large backups — as data needs grow, so too do drive capacities — but they are also the most future-proof. In a few years we may not be using DVDs anymore, or tapes. But drives will be around in some form or another for the foreseeable future, and they'll continue to get bigger and bigger. And since I'm not so much worried about having a permanent archive of my backup data (except in the case of data archived to optical media), I can continually and easily upgrade my storage by either purchasing new drives every so often, or by adding additional drives as needed. And transferring the data to new media — to these new drives — will be faster than it will with any other media (tape and optical media are slow). This system scales. And while it may be less reliable over the long term than optical or tape, it's plenty reliable for our needs and easily upgradeable in the future.

Lately everyone seems to talking about backup solutions. Mark Pilgrim recently wrote an intriguing post asking how to archive vast amounts of data over the next 50 years. I don't think there's an easy answer there, and my solution would not help him one bit. But it did inspire me to share my thoughts on the matter of backups, and my own personal system. It's certainly not the be-all-end-all of backup systems, and if others have thoughts on this complex and important topic, feel free to post them in the comments. I'd be curious to hear.