Not long ago I had a Finnish friend asking about what, if any, difference there is between “that” and “which.”  Some people think that the only difference is that “which” seems more formal, so when they want to sound fancy, they use “which” but use “that” in conversation.  The truth is that most native English speakers don't know and just assume they are perfectly interchangeable.  Given this fact alone, you can safely keep plodding along, using them interchangeably. 

However, if you are a precise writer/speaker, you will want to know the difference.  It really is quite simple.  In terms of basic meaning, they are quite the same—they both further qualify the idea being conveyed—but “which,” when properly used, implies that what follows is not integral to the understanding of the idea being communicated.  If you use “that,” it means that you cannot really understand what is being conveyed without that information.

For example:

I am typing on this keyboard, which just happens to be black.

In this case, you don’t really need to know that the keyboard is black.  It is just supplemental information that I think you might want to know.  The critical information is that I am typing on the keyboard. 

On the other hand, consider this:

This is the keyboard that I am typing on.

In this case, you really need to know the information “that I am typing on” because otherwise I’m not really conveying what I intend to convey, which is that I am typing on this particular keyboard.  If I left off that clause, it would simply say “This is the keyboard.”  What keyboard?  Why do you care?  You need to know that it is the one I am typing on.

In grammatical terms, we are talking about restrictive and non-restrictive subordinate clauses.  Restrictive clauses are those that convey information that is essential to understanding.  Non-restrictive clauses just supply extra information that is non-essential to the overall meaning of the communication.

So the rule of thumb is, if you can take out the subordinate clause and still get across what you want to get across, use a comma followed by “which.”  If your sentence loses essential meaning when you take the clause out, use “that” without any commas. 

Having said all that, there are times where I find that, for variation or for effect (because people really do perceive “which” as more formal), I will choose to abuse “which” in a restrictive clause.  However, I would never use “that” in a non-restrictive clause.  See the following example to get an idea of why this just doesn’t do.

This post is about writing, that I like to talk about.

Now the alternative:

This post is about writing, which I like to talk about.

If you don’t see the difference, it’s okay.  I just think that it sounds really strange, and heck, even Word’s grammar checker catches that one.  If Word catches a grammar error, it means that the error is really grievous or that Word is just confused.  In this case, I’d go with that it is really bad.

Oh and I should also note that we are only talking about the usage of “that” and “which” where they introduce a subordinate clause.  There are of course other uses of these words not covered here.