Is it possible for you to do something to help your passive aggressive husband? It takes courage to go beyond the frustration and anger to see him as a good person trapped in his own old fears...and ignore the thousand frustrations he makes you go through every day.
If you are not extremely angry with him, and have kept a healthy sense of curiosity ("Why in hell is he doing this to himself"?) perhaps you have discovered the pattern: the moments in which he goes back to being a scared child, hiding from commitment and adult love.
Your own brain is telling you: "when X happens, he does Z"...and this is the pattern you are looking for. Try doing this behavior once more, on purpose, and see if he goes in hiding automatically.
So, now you know something else: his pattern, or the sequence of grown up situations that force him to retreat.
The most difficult part is how to share this information with him in a casual, non accusatory way: can help him see the pattern, which is usually the toughest part.
If you can explain the pattern to him just when it is happening and not be too judgmental, or taking him too seriously, that would help. Something in the vein of: "Have you noticed that situations like (facing my whole family together at the holidays) make you cringe? It also happened last year, can you see that? ...what can we do to improve this situation?
I'm following here Louis Clichot "The whole psychological point of being passive-aggressive is to spare oneself the messy implications of experiencing one's anger. And it's not easy getting a grown-up man to say he's feeling like a child inside. Their feeling of repressed anger is so intense; it has never been dealt with in the family of origin and he could never bring it out, so it feels like a mountain of anger for him."
If you show him that you are not accusing him, and that this behavior is something both of you can watch, evaluate and improve, it gives you a different point of view in this situation. It's empowering for both the idea that passive aggressive behavior could be perceived and demolished with support and humor.