Archiving and storing email

Folks who are concerned about email retention (as I am) might find this interesting:

 

http://www.mothsoftware.com/

 

Like MailSteward (MS), <http://mailsteward.com/> this software archives emails to an external database.

 

MailSteward does it automatically, at a scheduled interval you can set (or you can invoke it manually). All mail goes into either its own internal SQLite database, or into your machine’s larger mySQL database. It works with Apple’s Mail only.

 

Mothsoftware’s Mail Archiver X (MAX) does not (yet – it’s planned) support scheduled backups, requiring that you do it yourself, although the process, once setup is trivial.

 

The advantages however are significant, at least to me:

 

• Archives emails from the following formats: Entourage, Eudora, Mail, Outlook, Postbox, Powermail, Thunderbird, and standard mbox.

• Exports emails to the following formats: Valentina (native), Filemaker, PDF, mbox, mySQL, Text, or XML.

 

MAX has a built-in browser for its own Valentina DB and comes with a free browser for FileMaker. (PDF, mBox, mySQL et al all have easily obtainable and free browsers.)

 

That internal browser for Valentina makes sticking with the MAX Valentina DB the most convenient solution.

 

I only just got MAX and so don’t have lots of experience with it. I did verify that it performs as advertised above however.

 

I’ve been using MS, and it’s been 100% reliable and very useful when I need to find an old email, but when you hit 60,000 emails or so, you’ll start reaching the limits of SQLite, and have to pony up for the “real database” version of MS, which uses mySQL and costs $100 ($50 upgrade fee, I think.) (MS prices are $25, $50 and $100.)

 

Either one, MAX or MS,  will allow you to keep the mail client’s own database small (because once you’ve archived email, you can delete it from the mail program), and the email program will therefore be more responsive, and less trouble to navigate.

 

Either one will export the database, so you can easily back it up (although restoring mySQL is a bit of a pain.)

 

The other advantage to MAX over MS is the price: it’s only $35.

 

Once the developer gets scheduling going, it seems that it’s going to be hard to justify MS much longer… but I’ll have to play with MAX for a while to be sure about that.

 

As it now stands, the MAX advantages are significant enough for me to seriously consider it… if for no other reason than that it will let me play with other email clients with impunity.

 

Bottom line between the two: MS is geeky and not “Mac-like” while MAX does a better job of being more friendly, and versatile.

 

hth

 

Tracy

USB 3 on a Mac? Sorta…

Here’s something you’ll find interesting.

While Mac’s don’t support USB 3 yet, USB 3 is backwards compatible with USB 2, which your Mac has.

So?

Well, USB 3 used in under USB 2 conditions (which I’ll call USB 3/2 to save typing) is much faster than USB 2.

For example, the ADATA S102/16GB USB 3 memory stick is about 50% faster than even the fastest USB 2 stick I’ve found.

And for more of a surprise, how about this:  the $19 Transcend USB 3 card reader (TS-RDF8K) is nearly twice as fast as my fastest USB 2 card reader (500MB copied in 11 sec vs 20 sec.)

So, if you’re into moving data from your camera faster, get a USB3 card reader even if your computer doesn’t support USB 3 yet.

As usual, however, YMMV.

 

 

What is a “block-level” copy?

Think of it this way: blocks are “containers for data”.  Or, let’s pretend they are dominos, and the “data” are the dots painted on each one.

line up 20 dominos. See the little dots on the tops of them?

a file-level copy will read those dots, and go to the destination, and write the same dots. Think copy/paste.

A block level copy pays no attention whatsoever to the dots. It takes the individual dominoes and creates them anew on the destination. The dots happen to come along for the ride.

The upshot of this is that if you use a file-level copier to create a clone to a blank drive, the result will be a very neat, nice continuous placement of all the files on the destination.  If you use a block-level copy, the files will be in the same physical place on the destination as they were on the source.

If your source drive files look like this:
***     ——–    **************          8          &&&&&     ^^^^^^^^^^^       $$   3

then when you use file-level copying, the destination drive will look like this:
***——–**************8&&&&&^^^^^^^^^^^$$3

but the block-level will look like this:
***     ——–    **************          8          &&&&&     ^^^^^^^^^^^       $$   3

Block-level copies also take longer, because they are coping ALL the blocks, regardless of whether or not there’s any data stored in them.

Goodbye, Steve

I bought my first Apple in 1978; sat down with Woz’s Red Book (which I still have) and taught myself programming. In December of that year, I was awakened by a woman, unidentified, who has seen my advertisement for “Apple programming.” She asked what I thought of Apple’s own accounting software. I replied that I was not in competition with Apple, and that I frequently recommended their software. Still anonymous, she asked nothing else; simply said “Thank you” and hung up.

An hour later, the phone rang again, and I was offered a job as the first outside contractor ever hired by Apple. From that day to this, even in semi-retirement, I’ve made my living with Apple products. Programming; writing for Mac Home Journal; editing TechNotes; consulting. Today I continue to program albeit for the iPhone. My Macs are used for video editing and photography. My day is still spent before a glowing screen.  I estimate that I’ve spent over 150,000 hours “at the wheel.”

It was coincidence of some sorts: I went to school with Bill Atkinson; was taught by Jef Raskin. My sister was in Woz’s plane when it went down, along with Jack and Candy. I was introduced to Apple just after they moved from the Red House to the Bandley Street warehouse. I remember looking over the cubicle walls: “That’s Woz office. Job’s is over there… and back  there is where we glue the rubber feet on the computers.”

Over the years I worked on DOS; the ROM for the Apple //c; the Apple ][ GS; the original AppleWorks and much more. My work, because of Apple, took me on some of my greatest adventures, including whale research in Alaska; work for NASA, and PBS. I met with both Steves several times, and when I “officially” retired from Mac programming a couple of years ago, Steve J called me to chat about the old days. We were not “friends” but long-time acquaintances, and that he’d phoned me spoke volumes about him.

Almost exactly half my life has involved Apple, its people and products. It has been, in a very large sense, my reality.

A couple of weeks ago, I wrote to Steve. I got the chance to tell him some of these things, and that he could rest knowing that he not only changed the world, but changed it for the better.

Goodbye, Steve.

Thank you.

Tracy Valleau

Move your user folder (Macintosh OSX )

Got a new SSD, or a Raptor? Need to squish down that boot drive so it will fit?

Most likely your User (Home) folder is the single largest item on your drive. For example, of my 520 GB of files, 205 GB of it was my home folder. Moving that to a different drive (and a few more files moved & replaced using symbolic links) got my boot drive down to 160 GB.

Sounds scary, and it is… scary, that is. It’s not particularly difficult: really just two steps.

Requirement for what follows: Snow Leopard and SuperDuper.*

OK: three, if you take mandatory step one – MAKE A CLONE FIRST!!! (In fact, you might want to run DiskWarrior just to make sure everything on your drive is ship-shape. Up to you…) Make absolutely certain you’ve used SuperDuper to make a bootable clone BEFORE you go mucking about with your boot drive.

And, just because some fool will call me on it: I am, in no way, shape or form, even -remotely- responsible for you following these instructions and completely trashing a lifetime of work. You’re on your own here buck-o. If you agree to that, and you’ve made a clone, then here we go:

A quick overview:

1) Run SuperDuper and choose the script that will copy your home folder.  (See below for the specific steps to choose a script, or RTM.)

2) in your System Preferences/Accounts, select the account you’re moving, and control click on it. In Advanced Options, choose the newly copied user folder, and accept the changes. (There is a convenient “choose” button.) You’ll be forced to reboot.

DO NOT delete your previous Home folder yet!!!

Basically, that’s it. Perhaps an item or two will complain about the new location, but that’s probably all. (I had to point DropBox again.)

Now, IF you reboot and  things are obviously, seriously wrong, just return to step 2, and change advanced options back to your original home folder. (See why you shouldn’t delete it right away now?) If worse comes to worst, you can replace the boot drive with the clone you made, as instructed above.

And if it’s a total disaster, don’t say I didn’t warn you!  (That’s just CYA on my part… everything was simple and easy to do for me, and I don’t expect it will be any different for you, either.)

Good luck!

 

*SuperDuper. Why? Because it works, and many Mac users have it. If you don’t, you’ll have to choose some other way of copying the user folder. (Drag and drop is likely to fail, BTW – you need something that will copy properly for the OS.) No, I don’t have other recommendations. This is a quick and dirty guide, explaining what worked for me. YMMV. GIYF.

 

Here are the specific steps to move your user folder using SuperDuper:

 

  1. new copy script
  2. included scripts
  3. + button
  4. standard scripts
  5. backup – user files.dset
  6. close
  7. save
  8. save as “Home folder”
  9. —now on the main SD screen, select the source drive and the destination drive
  10. in “using” click on “Home folder” under custom scripts
  11. Begin the copy.
  12. when done, see step 2), above.

Making a “real” clone

I’m a huge fan of SuperDuper! and I’ve used it for years. “Fan” because it’s saved my bacon several times. When a drive dies, I just remove it and use the SuperDuper “clone” and I’m up and running in under 5 minutes.

That said, it’s a “file-level” clone, fully bootable of course, but it’s not an absolutely identical clone, block-for-block.

To do that, you need to do a “block-level” clone. Such a clone ignores the contents of a drive and simply duplicates every low-level block from one drive to the other. You could have one file on the drive or a 15 million; makes no difference – it will take the same amount of time to clone the drive either way.

Carbon Copy Cloner does this, as does DriveGenius and a few others. My personal choice is CopyCatX, since it adds the feature of not requiring the drives to be exactly the same size. (Think about it: if you’re copying all the blocks on one drive, to another drive, that destination drive had better have _exactly_ the same number of blocks as the source, or things are going to get wonky.)

(I believe that if the drives are the exact same size, Disk Utility’s “restore” will also do a block level copy.)

CopyCatX will do a block level clone to a larger destination drive, and then apply the proper resize information to the boot block so that the drive is seen at full size, and not the size of the source.

One quick word here. You’ll often see blanket statements on the internet that “block-level copies are faster than file copies.”

Nonsense. Take a simple example: if you’re copying a 750 GB drive, it will take about 4.5 hours to copy all the blocks. If the source drive has one file on it, a file-level copy will take a few seconds, so the blanket statement that “block-level copies are faster” simply doesn’t hold water. The theory is that the OS has to read the file system and write directories and etc on a file-by-file basis, and that will slow it down. Yeah… but.. these days that adds millionth’s of a second per file. Look: if it takes 4.5 hours to block-level copy a given drive, it will always take 4.5 hours. Period.

The only time a block-level copy will be faster than a file level copy is when the source drive is nearly full… and that refers only to when the destination drive is empty.

If you’re doing a file-level copy using SuperDuper, and the destination already has most of the files from the source (as it would if you’re using SD for backups) then the update of the destination is going to take minutes, not hours.

So: why would you want to do a block-level copy?

1) because it will preserve the exact state of the drive. If you have a drive disaster, the first thing to do is a block-level copy of the damaged drive. Then do your repair attempts on one of them, leaving the other alone. If your repairs only make the situation worse; clone again, and try again.

2) because it will preserve authorizations on that annoying software that uses “activations” and similar approaches. I do this with my daily backup drives. That is: I first do a block-level clone to the destination drive… and from then on, I use SuperDuper to do file-level copies for daily backups. Having done the block-level first insures that the activations are properly copied, and file-level copies won’t alter them.

Finally this tip: why don’t I use Carbon Copy Cloner instead of SuperDuper? Because SuperDuper is nearly twice as fast for file-level clones.

On installing Lion

If you have not run DiskWarrior against your boot drive in a while, it probably wouldn’t hurt to do it now.
One thing I always do before installing a new OS, or update, is run Repair permissions.

Downloading the 3.76 GB file took 45 minutes on my (fairly fast cable) connection, and resulted in an application in my root Applications folder. The installer launched, but I quit it.
Next I copied the entire file to a backup drive.

(You want to do this because the installer will disappear after you have used it.)

Making a bootable DVD:
Then I went to the Lion installer application, and control-clicked on it, choosing “show package contents”
in the resulting contents folder, I opened the SharedSupport folder, to find the file InstallESD.dmg.
I option-clicked on that and dragged a copy of it out to the desktop. (If you do this, MAKE SURE that you option-click, and that the cursor has a green + on it. You want to COPY the file, NOT MOVE it out of the SharedSupport folder!)
Next, I launched Disk Utility and clicked on the burn button. I selected the InstallESD.dmg that I’d copied to my desktop, and let it burn a DVD of that file.

This gives me a bootable DVD for installing Lion. I’m carefully filing this DVD away. Then I tossed out the InstallESD.dmg that was on my desktop.

Once that was done…

I ran Repair Permissions on my main drive; quit all running applications, and ran the Lion installer.
The installer will take a minute to do some setup, and then restart your machine.
The actual installation takes about 30 minutes. (If you are doing this on a laptop, remember to plug in the power supply.)
The installation is hands-off, with the usual “Time remaining: less than a minute” taking about 5 minutes.

Don’t freak out if the screen goes black… it’s just the energy saver kicking in. You can click the mouse button to restore it, and watch the grass grow… er… watch the blue progress bar grow…

I’m not going to repeat things which have already been written, so if you want an overview of what’s new, try these links:

http://www.electronista.com/reviews/mac-os-x-10.7-lion.html
http://www.macworld.com/article/161026/2011/07/osx_lion_review.html

if you get a message saying: “Some features of Mac OS X Lion are not supported for the disk (volume name)”, it means that a repair partition could not be created on your hard drive. see this page: http://support.apple.com/kb/HT4649

I’ll report back, as will others here, I’m sure, on what I find that’s most interesting.

Oh… once you install everything and reboot, be sure to check software updates in the Apple menu. iTunes and other items have been updated for Lion.

Tracy

“They couldn’t say it on the internet if it wasn’t true.”

Is Apple headed for trouble?

Just the observations of an old guy, but one who has been watching and working with Apple since 1978…

FCP-X launch is a disaster.

iCloud reception is luke-warm; hostile in some cases.

Android is a serious contender.

Lion dumbs down the OS.

Apple reduces iAd rates as advertisers flee.

Apples loses “App Store” tm… (maybe)

Apple is “negotiating” with Lodsys.

Google and Amazon have lockers that didn’t cost them a dime to the RIAA (et al.)

 

“We know what you want better than you do.”  The first sign of problems just around the corner is when you start believing your own PR.

We’ll see how Lion pans out, but if it gets the luke-warm response I expect, I’ll be seriously considering selling my stock. $358 today (7/7/11)… either I”ll be eating crow, or sadly correct… give it a few months.

Honestly, after nearly 65 years on earth, and paying attention all the time,  it’s beginning to look like there’s some tarnish around the edges.

 

I hope I’m wrong.

Warning re: DiskWarrior and SuperDuper’s clone-on-mount

Please don’t get me wrong: both of these are superior products, and I have and use and love both.

That said, SuperDuper (SD) offers a “clone-on-mount” (auto-clone) option that can wreak havoc when and if you decide you want to run Diskwarror (DW) on the destination drive.

That is, if you mount the destination drive, and SD starts cloning something to it, you may have problems if you try to run DW on that drive to repair it.

Here’s the issue:

Source drive S, and Destination drive D

User wants to have DW repair drive D.
Drive D is also the target of an automount which clones S to D.

Drive S has files A, B, C, D and E.
Destination drive D has files A, B, F and Q.

User inserts D
SD clone starts and User cancels clone.

DW looks at drive D, and sees A, B, F and Q.
DW unmounts the drive to work on it.
DW creates a replacement directory listing A, B, F and Q… remounts drive D, and then pauses. DW does NOT write the directory, because it’s waiting for the user to OK it.
Seeing that D has remounted,  the auto-clone kicks in and SD copies drive S to drive D.

here’s the crux of it:

after the SD clone, D now has files A, B, C, D and E. (not A, B, F and Q, which is what DW thinks it has)
User says to DW, “Yep, install the new directory.” and DW dutifully replaces the directory to say that D has files A, B, F and Q.

Drive D directory is now totally hosed.

There is basically no way around this except to start up SD and turn off all the auto-clone scripts and then quit SD before you run DW.. Now you can run DW without issues… just remember to re-enable the scripts in SD after DW is done.

On the death of the Finder

OK: the title is a bit over the top….. the finder is not dead… but it’s in its later years. I’m prompted to write this as the release of Lion is impending.

Many of us understand the Finder/Desktop metaphor as the “filing cabinet” it’s intended to emulate. Others (although this blows my mind) don’t understand filing/organizing at all. I suppose organization is a trait of some folks, while for others it’s literally beyond comprehension. My dad, for example, filed things in cabinets obsessively, but could not grasp the same concept on the Mac.

Apparently, that second group of people is the majority. For them, the Finder was a nightmare, and virtually everything in their computer was on their desktop.

For those of us who pride ourselves on our ability to organize and retrieve information, our directory (folder) structure are not only a source of pride, but something we eventually begin to obsess over. We become uncomfortable when something isn’t file properly and promptly. And rightly so, since when retrieving an item, that obsession has served us well over the years.

But for both the filing-obsessed and the filing-phobic, there is, indeed, a better way. The latter will have already embraced it. This blog-entry is for those filing-champions who start jerking and twitching at not knowing exactly where a document is filed.

The point of the Finder is, um, to “find” things… so that we can “use” them. The point of any filing system is to make it easier to retrieve a given item … hence the extra effort required up-front to do the filing in the first place.

Yes, it’s much easier and faster to simply throw everything you own into one large box… but picking out one given item (especially after memory has faded) can then take hours to do… or even become virtually impossible. So instead we take a bit of extra time to use multiple storage places, and assign a meaning that we personally find useful to each one. Then we file the item therein.

If those boxes are too large (the categories too broad) then it’s still time-consuming to sort thru them to find a given item… so we use sub-categories – (boxes within boxes).

My point is that this “do the work up-front filing system” is highly personal, and reasonably effective.

Enter the computer.  We simply carried over that old physical filing system technique to the desktop metaphor … and it works … that is… it works just as well as the physical one does… but with all the attendant shortcomings along with it. (Folders fail if they contain too much; the titles are not descriptive; you put something in the wrong place; you forget the name of the folder, et al.)

The computer however, can make the process more efficient, both up-front (just toss everything in one box) and on retrieval.

The difference is that we are just specifying the search criteria after the fact instead of during the filing process.

In the database world, we can find something if we remember almost anything at all about it. We can increase our comfort/familiarity level by using “smart folders” which look like our old familiar directory/folders, but are vastly superior.

We can drag a document to the “main box” and drop it (equivalent of putting everything into one box) or we can drag and drop it on a set of keywords (with exactly the same effort) and further enhance our search (just like sub-folders, but with less effort.)

Further, a database approach is not only faster on the input side, and the retrieval side; not only does it contain “sub-folders (keywords)” but it offers something no folder/directory setup can: items can be “in multiple places” at the same time.

You can have a smart folder for clients; one for projects; one for billing… and each one can reference a single document.

The whole point of this however is that we spend 95% of our time -using- a document, and where it is physically located is relevant only to retrieving it (so that we can use it.)

And, if you get jittery, you can simply use the contextual menu to select “reveal in finder” (yes: even in iTunes) and you’ll be taken directly there… you don’t even need to navigate the sub-directories manually.

Having been at this for 36 years, I too was a bit hesitant, even uncomfortable, at “giving up” my carefully constructed and personally meaningful directory structures for many things.

However, over the past several years, I’ve not only over-come those feelings, but actually embraced the database paradigm. I use DevonThink Office for text documents; LightRoom for photos; iTunes for music and so on.

When/if I need things organized into a project folder (for backup to an optical disc, for example) I simply choose the smart folder that contains it, and either select “export” or better yet “burn.”

Where the data is located is important only in the context of being able to retrieve it, and in that respect, databases are leagues ahead of the multiple shoebox approach.

 

Post Script:

I guess I should have pointed out that a directory/folder -is- a record in a database. There are no “places” on a hard drive that are “folders” that “contain” your files. Your files are likely scattered in pieces all over the place on your drive; it’s much more like trail-mix than a box of rasins and a box of nuts.

Directories (aka folders) exist only in the database that is called the “catalog” and “B-Tree” (among others.) When you click on a file in the Finder, that database is consulted, the constituent bits of your file are found, and presented.

The actual, technical different between iTunes/DevonThink et al, and the Finder is far less than one might think, and is mostly just the user interface.

30 years sitting in front of Apple computers. I’m fat and my butt’s sore.