Wasps...please help

For the past few months we have noticed a wasps nest in our bathroom window and have done nothing about it.  Now we a huge family living in our window.  Check out the pictures below.  We don't have a way to kill them right now, I think we need to call Billy the Exterminator and have him get rid of these nasty things.