You know how in every crime/mob/gangster mvoie/TV show some poor chump totally stirs up shit and than, even though he knows he should skip town, stays in his mom's basement or visits his girfriend and gets wacked? Hell, look at what happened to Fafhrd and the Gray Mouser's girlfriends?
Don't be that guy. When you end up kicking over the hornet's nest and you know that the shithammer may very well be swinging your way, don't mess around, don't tie up loose ends, instead steal some horses or a boat that very night and get the hell out of town.
And don't shack up two villages over or anything else half-assed; beat the hoof hard and get your butts on another continent if you can. There's more of them than their are of you; and divination magic, speak with dead, etc. makes it really hard to cover your tracks in a bulletproof fashion.
"Getting out of Dodge" is a cliche for a reason...