PS tutorial 7: masks

Masks, Masks, Masks!

Masks in Photoshop have a rep as complex and scary, fit only for truly advanced users. And yes, they are somewhat complex, in a sense that they are powerful and versatile. But they're really not scary or particularly difficult to use, you just have to understand the basic concept. The purpose of this tutorial is to show you both how to use masks (which you can also learn from the Photoshop help files) and then how to use masks to do stuff you will probably want to do in the course of creating composite images, collages, wallpapers, blends, CD labels, web designs, etc. There is some overlap with my earlier blending tutorial, but lots of new stuff, and lots more detail. (I seem to have written a book!* Hopefully it's not so much detail that it's overwhelming!)

*JUL 2008 UPDATE! I have written a book! Well, 66 pages, is that a short book or a long pamphlet? At any rate, it's an extended version (3x as long) of this tutorial, with all my own photographs plus additional topics such as: using the pen tool to mask, retouching dusty photos, colorizing black and white photos, creating decorative edges, using clip templates and other mask-based methods to make interlocking layouts and a cheat-sheet to help you keep track of it all. Sound good? Well, the bad news is that since it took me the better part of a year to pull it together, and since I would *really* like a shiny new dSLR round about the time my first baby is due to make her appearance this fall, this is the shameless commerce part of the site. So here's the part where I bat my eye-lashes and say: PLEASE BUY MY BOOK! and tell all your friends to buy my book, too. Or just enjoy the freebee version below, I totally understand. But if you happen to have some spare cash lying around, or a friend/family member with a birthday coming up, etc, etc....


7.1 - Photographic Background

But to start with, let's forget Photoshop entirely, and take a little trip down memory lane, back to high school year book class, where I learned (in addition to the fact that there are some people entirely too annoying to be stuck in darkroom with) about how masks work in actual RL photography. The basic concept is the same, so I'm hoping an understanding of that will make everything a little more concrete and easy to grasp.

To make a print from a negative, you shine light through the negative onto light-sensitive photopaper. Everywhere the light hits the paper, the paper darkens. The degree/color to which it darkens depends on the intensity/color of the light shining on it. Sometimes, however, you don't want to print the whole negative, just part of it. That's what you need a mask for.

In the diagram to the left, the mask is the piece of cardboard with a hole in it. Where the hole is, the light shines all the way from the light bulb, through the negative, through the hole, to the paper, creating the image. But the cardboard blocks the edges of the light shining through the negative, so that it never reaches the photo paper. Hence the photo paper in those areas remains its original color (white) creating an image that is the same shape as the hole in the cardboard. [in yearbook, we also used to mask things by stick masking tape on the negative. Again, where the masking tape was stuck, light couldn't shine through to the paper, thus blocking out -- masking! -- those parts out from the finished print.]

A mask in Photoshop works the same way. The negative in the above diagram would be the layer that you have the complete image of AD (Alexis Denisof AKA Wes on Angel the Series) on. The cardboard is the layer mask attached to that layer, that tells Photoshop what part of the image should "shine through" and what part should be "blocked" (hidden from view). And what you see when you've applied the mask is the finished photographic layer, the circular version of the pic.

7.2 - Photoshop Masks, explained

Rather than being a piece of cardboard, a mask in Photoshop is a layer completely filled with black. To cut a hole out of the mask, we paint on it with white. A completely white mask would thus allow *all* of the layer to show through, whereas a completely black mask doesn't allow *any* of it through. Thus, to make the Photoshop mask equivalent of the cardboard mask shown in the diagram below, we'd have a black layer with a white oval on it. If you prefer the masking tape analogy when you paint black shapes on your mask, those black areas are the equivalent of masking tape on your negative. The neat thing about Photoshop masks is that whereas cardboard/masking-tape masks have only two states -- solid or not-there -- digital masks have shades of gray in between white and black. Any part of the mask painted 50% (medium) gray would allow 50% of the image to show through. So, a mask filled entirely with 50% gray would be just like setting a layer to 50% transparency. Meanwhile 25% gray (light gray) would block only 25% of the image, leaving us with 75% transparency whereas 75% gray (dark gray) would block 75% of the image, leaving us with only 25% showing through.

Confused yet? Let's make it a little clearer with some examples:

This example assumes that the background layer (the layer directly beneath the layer with AD's image on it) has been filled with that turquiosy blue color. So in the first example, the mask is black, completely blocking the image, leaving you looking at the background (blue). In the second example, the mask is white, so the entire image is visible, which means that it's completely covering up the blue background. In the third example, the mask is 50% gray, so it's only half-blocking the AD image, so the blue partially shows through the image. In the final example, the white circular part of the mask allows that part of the AD image to show, but the black parts block the edges, so that it looks like we've cut a circular bit of the AD image out and set it floating over the background.

Let's take a moment on that fourth example. You could accomplish exactly same effect by making a circular selection, and then either copy-pasting it to a new layer, or inverting the selection and deleting the excess. So why use a mask? One word: versatility. Let's say later on you decide you want to change that circle to an oval. Or feather the edges. Or make it a square instead. If you've deleted the stuff outside the circle, you've got no choice but to open your original image and copy-paste it back in to your layered Photoshop file and start all over. But if you did with a mask, it's easy, because you haven't actually lost that image information that's hidden. It's still there, and all you have to do bring it back is paint with white on the mask.

7.3 - Photoshop Masks, technical how-to's

Okay, enough of the theoretical, time for some nitty gritty: Here's what the layer palette looks like PS5.0. Other versions should look similar.

When a layer has a mask, you'll see an thumnail representing that mask just right of the thumbnail representing the layer. You click on the layer thumbnail when you want to paint on the layer. You click on the mask thumbnail when you want to paint on the mask. To add a mask to an existing layer, you click on the "new mask" icon at the bottom of the layers palette right next to the "new layer" icon. If you happen to have a selection active when you click on the new mask icon, everything that you've selected will be white in mask, and everything that you haven't selected will be black (ie, everything not currently selected will be made invisible). If there is no active selection, you'll get a white mask (which, remember, means your entire layer is 100% visible). Start painting with black to "erase" (hide, really) parts of the layer. [In the example masks shown in the palette screen-cap above, the masks are mostly black with "holes" poked in them so that the subject part of the cap in question can show through, but with edges erased. The edges of each "hole" are blurry because I want the edges to fade out, feathered.]

Oh, one more thing: if you ever want a better look at your mask than is provided by looking at that tiny little thumbnail, alt-click on the thumbnail. The mask will be displayed in the canvas window. Alt-click again to go back to regular view.

7.4 - Getting Ready to Play

Okay, with that out of the way, let's start playing. First, download these 2 images:

Open them both in Photoshop. Make each into a layer in a new Photoshop document, with the textury one on bottom and the AD image on top. [here's one method: with the texture image window active, control-A to select, control-C to copy, control-N to open a new doc, control-V to paste, switch to the AD window, control-A to select, control-C to copy, switch to the new doc, control-V to paste, close the two original windows. ] Label each layer appropriately. You should end up with something like:

Save a copy of this somewhere so you can revert to it. Some of the following examples will require you to go back and start from this state, whereas some will just build on what you did last time. Make sure you start with the AD layer active, as shown in the screen cap above. I'll refer to this state as the "2-layer, no masks" or "original version", etc.

7.5 - Simple Cropping:

Use the marquee tool to select a square/rectangle or circle/ellipse over AD's face. Click the "add layer mask" icon. [Alternate method: before making your selection, add a mask to the layer. Make your selection, select inverse, make sure the mask layer is active, fill with black.] You should get something like this:

Your mask would look something like this:

Now for the fun. With the AD layer still active, select "effects" from the Layers menu, and add a drop-shadow or outer glow. Note how the drop shadow follows the curve of the *MASK* and not the original image's edges. Pretty slick, huh?

7.6 - Fun with Cropping

Okay, try this: now make another selection that partially overlaps your original selection, and with the mask layer active, fill with black. (I did a vertical rectangle, intersecting the oval on the right) And then do a new little selection that DOESN'T overlap any of your previous selections, but DOES overlap the AD image, and fill that with white (I did a box down in the right corner). [Rather than using the marquee to make selections and fill, you can also use the brush/airbrush to simply paint some black onto the currently white areas of your mask, and some white onto the currently black areas. For extra fun, try using a star brush -- or some other chunky custom brush -- to stamp solid shapes.] My mask now looks like:

and my image looks like this:

See how the newly black part has gone transparent And how the part you just filled with white is has recovered a bit of the original image? When you use a mask to "delete" part of an image, no deletion is ever permanent. Remember that, it's *very* useful when you're building complex images!

7.7 - Moving/Resizing

Now, activate the move tool. Drag your AD layer around a bit. See how the mask and the image move together? That's because they are linked. You can tell, because in the layers palette between the layer thumbnail and the mask thumbnail for each masked layer there's a little link icon. Now click on that link icon (interlinked chain) to UNLINK. Now, with the regular layer (not the mask) activated, try using the move tool. You should be able to do something like this:

Conversely, you can also move the mask around while leaving the image alone. Make the mask active by clicking on its icon, and now drag around. You can do something like this:

Now, try this: with the mask and image still unlinked, and the mask active, do control-T. Move the corners around to resize the mask, while leaving the image alone.

You can also do the reverse, activate the image layer and resize it in relation to the mask. If you want to resize both the mask and the image at the same time, or move them around in tandem again, link them up by clicking between the two icons on the layers pallette, where the little link symbol was to begin with.

7.8 - Blurring

Okay, back to the basic 2 layers, no masks yet. Now use the lasso tool to draw a loop around AD's head, and click the "add mask" icon. You should get something like:

The mask looks like this:

Now, with the mask layer still active, call up the Gaussian Blur Filter (Filter->Blur->Gaussian Blur). Play with the slider, and watch what happens. I decided "20" was good for me. This gave me a mask that looks like this:

And an image that looks like this:

See, where on the mask the color around the edges trails off from white to black, the transparency of the image trails off in exactly the same way. Which gives us the feathered effect so useful for getting different images to blend smoothly together. The nice thing is that if it's not perfect, you can go in and touch up with an airbrush. [This is how I do pretty much 100% of my collages. I find that it's usually much easier to get adjacent images to blend smoothly together if there's a halo of semi-transparant background from each hovering around the edges, such as created by using a blurred mask, esp. if the background colors between the two images are similar, or at least, both light or both dark. I typically use close-cropping (see below) instead only when the background colors/values are dissimilar.]

One more step just to show you the mirror image of this technique. Download this image, and paste it as a new layer on top of the existing layers, so that all you see is it:

Now add a mask to this layer by clicking the "add mask" icon. Nothing should change on the canvas, since you didn't have a selection active. Select a large soft brush and with the airbrush at a low opacity setting, and black as the foreground color, brush where you think AD's face should be. [NOTE: this would also be an excellent time to play with some grungy custom brushes instead of the soft round I use in my example] You should be able to do something like this:

Call up the gaussian blur dialogue on that mask, and slide out to 77, and you get something like this:

7.9 - Carefully Cutting Out

If you want a really close cut like you're used to doing with the eraser to really cling to the lines around a person or object, layer-masks have you covered there too. Delete the purple basket layer and go back to the mask on the AD layer. You can go back in the history to the un-blurred version of the mask, or just work with it as is. Now, spend some time fine-tuning your mask, using the paintbrush or airbrush and a small brush size. Set your foreground to black, and your background to white, and start painting around the edges, drawing near the line where face meets background. If you go a little too far, hit "x" on your keyboard to flip flop background and foreground colors. Now when you paint (in white), you'll be *un-erasing*. When you've corrected your mistake, flip back and continue. I was able to produce the following in a few minutes:

7.10 - Patterns

Okay, try this now. Download the following stock image and add it as a new layer on top of everything else:

Now create a new document, 20 pixels by 20 pixels, fill it with black and add a small white square to the top corner. Or just use mine:

Select all and define it as a pattern (under 'edit'). Now, flip back to your photoshop file and select that animal-print layer. Add a mask to it, control-A to select all, then fill with pattern. You should get something that looks like this:

Now use the airbrush and a large roundish brush to clear out the middle by painting black on it so that you get something more like:

This mask for this, incidently, ends up looking like this:

You can definite all sorts of interesting things as patterns, either more black than white as in this case if you just want little sparks of some texture to show up, or more white than black if you want just sparks of some *underlying* texture to show up. Lots to experiment with here. For example, here's the same mask, but this time I applied it to the AD layer, and then reversed the colors (control-I) so that it was more white than black.

And checkout what I got by using this mask:

And then applying a white, screened, drop-shadow to the layer:

7.11 - Using Images as patterns

Go back to the original, sans-mask version. Hide the AD layer so that all you can see is the textury background. With that layer active, control-A to select all, and then again under the "edit" menu, select "define pattern". Hide the texture layer and unhide the AD layer. Add a mask to the AD layer. Now with the mask active, fill with the pattern in memory. You'll get something like this:

Then I dabbed a little white onto the mask over the face to bring AD's features out from under the texture, sprayed black along the edges of the AD image to get it to blend a bit, and filled the background layer with a light blue:

7.12 - Gradients

Back to the "basic 2 layers, no masks yet" version again. Add a blank mask to the AD layer by clicking the new mask icon. Set the gradient tool to "foregound to transparent" (or foreground to background) and use the "normal style" to create a mask that looks like this:

This will make your image look like this:

You can also use the radial filter for a quick version of the "make a circular mask with the marquee tool and blur it a lot".

7.13 - Filters

Back to the "basic 2 layers, no masks yet" version yet again. This time, use the marquee tool to make a selection and make that into a mask, giving something like:

Now, with the mask layer still active, go to the filters menu and select one of the distort filters. I used ripple, with "large" waves and the slider kicked way up to distort my mask so that it looked like:

And this makes my image look like:

You can do all sorts of funny things to the edges of your mask using various distort filters. Just about any filter can do something interesting to a mask, so lots to experiment with. The possibilities here are endless, since you don't have to stop with one filter, you can apply as many as you want to the mask.

7.14 - Custom Brushes

As I've hinted at above, custom brushes can be very fun to use in conjunction with masks. To try it, go back to the original 2 layer version, and add a blank mask to the AD layer. Hide the texture layer, it'll make it easier to see what's going on. Now select a custom brush. [If you don't have any, I recommend downloading and installing the classic Nocturna brushes, or the fantastic Roshiweb brushes I use in my example] and use it to paint with black on the mask. My mask looks like this:

Which yields:

Try filling that white background layer with some different colors. I did a mustardy yellow sort of thing to get:

You *could* get the same effect by adding a new layer over the top of everything and painting on that with the custom brushes. But it wouldn't be nearly as easy to switch the colors. Plus, if you over-spray, you have to switch to the eraser, select an appropriate brush, and erase what you've done, but if you're using a mask, just tap x on your keyboard to switch black to white and spray out your over-exuberance with the same brush & tool you used to create it.

7.15 - Quick Masking

Now, it's time for a word about Quick Masking. Quick Masking is exactly what it sounds like, a quick version of masking. Maybe it would be more accurate to call it "temp masking". It's for those times when you want to use one of the paint tools to help you make a selection, but you don't need it to hang around for any subsequent tweaks like you would if you were using a regular mask.

Access the quick mask mode by hitting Q on your keyboard. Go back to regular by hitting it again. If you have a selection active when you hit Q, that selection will be reflected by the new Quick Mask, just like it is if you have a selection active, and then add a layer mask to a layer. The only difference is that in Quick Mask mode, instead of black, you get red, and instead of white, you get transparent.

You can do any of the examples above with Quick Mask instead of regular mask. Wherever the instructions talk about clicking on the "add mask icon", type Q instead. You're still painting with white & black, but when you do, it'll show up as "red" or "not red". When you're done tweaking the mask with paint tools, filters, gradients etc, hit Q again, and you have a selection. Hit shift-control-I to invert your selection and then delete *or* just control-c/control-V to copy/paste to a new layer, and then go back and erase your original, un-masked layer. Of course, if you do it this way, you lose the ability to go back and make changes to the mask, like you do with regular masks. That's why I mostly use quick mask for levels tweaks, as explained in the glow tutorial.

7.16 - Clip-Masking

Far more interesting than Quick Masking is Clip Masking, or as the PS help calls it, Clipping Groups. When you use a clipping group, your mask is really just a regular layer. One or more layers are "clipped" to this layer, and that/those additional layers are masked based on the properties of the base layer (aka, the "clipping layer"). This is a lot harder to explain in words than it is to show, so let's just hop to the showing.

Go back to your basic 2-layer, no-mask version. Now, create a new layer above the texture layer, but below the AD layer. On that layer, use the marquee tool to create a circle, and fill with white. You won't actually be able to see your circle yet, because the AD layer is hiding it. Now go over the layers palette and hold down the alt-key and move the cursor over the line that separates the circle layer from the AD layer. A new little cursor with circles on it will appear. Click. You've just clipped those two layers together. Notice that the AD image is now being masked by your circular layer:

Unlike regular masks, clipping masks don't use black and white. For a clipping mask, *any* color painted onto the layer is the equivalent of white in a layer mask. Transparant areas, ie those created by using the eraser or the delete on the clipping mask layer, are the areas that are made "invisible" on the overlying layer.

Now, try this. Get that funky animal print stock out again, and place it as new layer above everything else. Use the move tool to position it like so:

Now, clip it to the AD layer.

That's right folks, you can clip as many layers as you want to that clipping layer. And each layer can also have its own individual layer mask. Like so: add a layer mask to the animal print layer. Now use the gradient tool to put a black-to-white blend across it. You should get something like this:

You can link all these layers together by clicking on the link circles for the appropriate layers, and then move them about en-masse, or you can move each individually. Same goes for size transformations. This is how I do my CD labels; I have a mask shaped like a CD, and then I clip screen caps to that layer, each with its own individual mask to feather out the edges.

Oh, and try dropping the opacity on the layer with the circular mask on it. See how all the images on top drop their opacity too, so that the whole group fades out as you reduce the visibility of the clipping mask? Now try just moving the slider on the AD layer, or the animal print layer. See how that works?

7.17 - Clip-Masking & Text

The real beauty of clip-masking comes in when we get to text. Try this: Go back to the original 2-layer, no mask version. Use the Text tool to make text something like this:

Now, move that text layer beneath the AD layer in the layers pallette, and clip the AD to it. Move the face a bit to the left, and you get:

And the neat thing is that you can edit that text all you want. If you'd done this by selecting the shape of the letters and then deleting the inverse off the image, you'd have to start all over again if you noticed a typo and had to modify the text, or wanted to change fonts.

Not only do clipping groups allow you to put images within text, it also allows you to use whatever stock image or texture you want to spice up your fonts. Just clip the image to your text, and instant pizazz. Here, I've clipped the animal skin layer to the text layer, added a layer effect (drop shadow) to the text layer and moved the pattern around until only the zebra part is showing.

One last thing I use clipping-layers in conjunction with text layers for a lot is to add texture or color variations to text. Even a very subtle variation in color on text can add a lot of excitement to it... In this example, I've deleted the animal pattern layer, replacing it with a blank layer clipped to the text (the text color is a medium blue, picked from the background). I then used a custom brush (you could also use any round brush) to randomly spray some darker blue across the text. Then I set the layer to color-dodge. Because I've clipped the layers together, I don't have to be careful to spray only on the letters. Wherever I miss and go over the edge is just invisible. So basically I have a licence to be messy here!

Sometimes, instead of using the airbrush, I just put a gradient on a layer, and clip that layer to text. This can give a nice metallic effect, depending on the colors you use.

7.18 - Putting it all together

Here's a little challenge piece I put together using only techniques explained on this page and the images provided for the tutorial:(no coloring tweaks or filters or cutting or glow techniques allowed! just masks! and one roshiweb brush.)

The floating boxes are square masks applied to various copies of the AD image, with white drop shadows (7.5) The ghostly Wes in the bottom left corner is masked layer in which I blurred the mask until it faded off around all the edges (7.8). The AD on the right was carefully cut out using a mask and small brushes (7.9). The text at the top is blue with a layer sprayed with white and set to color-dodge clipped to it (7.17). The text at the bottom has a copy of the purple basket weave clipped to it, plus an outer glow (7.17). I added a pattern, this time little circles (instead of squares in the demo above) to the animal print layer (7.10). Then I put yet another copy of the animal print layer on top of everything and applied a diagonal gradient to it so that it's visible in the bottom right corner and trails off as it reaches toward the center (7.12). I also put a copy of the purple baskety stock image over everything and used a custom brush to scuff most of it out (7.14). Not by any means a brilliant work of art (15 minutes!), but just an example of how you can combine multiple techniques and a very few images to do something pretty complex looking.

You may download a shrunken, but still layered, version of the photoshop file used to create this image. I've labeled all the layers, so hopefully if you poke around in it, you should be able to see how I did each thing described in the paragraph above. Remember that all you have to do to view the mask for a given layer is alt-click on its thumbnail in the layers palette. *Very* useful! The original version is 10 megs, so I had to reduce the dimensions quite a bit to get something reasonable (1.9 MB). Hopefully, it should be just large enough to allow you to see what's going on.


More of my Photoshop Tutorials   |     view my art (home page)

Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)